2024-04-22T06:57:02.900 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T06:57:02.901 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/norstats}} --name leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160 --owner scheduled_leonidus@teuthology --verbose -- /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/orig.config.yaml 2024-04-22T06:57:02.958 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T06:57:03.058 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/norstats}} email: null first_in_suite: false job_id: '7668160' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync rbytes: false syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4362 sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 sleep_before_teardown: 0 subset: 222/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6g0JTthriIoXN5kBBGseqoqWy1JOXp45uI8u2MFVYG9zizTzKOY+zEkhKBbMelmYFy26mSKGWmrXvYDlSEp+E= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPk9A0djFdO5+ax7FyFdebc7JerxPkuam0fVNNPzK3lJD3HZmwBMDE9OTT8HGce9NbeAQWWiQB2nHL/7E0NuJTg= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGBpNK4PdUCor6qTZfXUX28WjERiFuxHXPSv/o2ZVN9dqn1K1O5AnqNlRUduwt3ESyYoKq1ZOOQW9Sin5xhXWsA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_05:52:53 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T06:57:03.058 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa; will attempt to use it 2024-04-22T06:57:03.059 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks 2024-04-22T06:57:03.059 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T06:57:03.061 INFO:teuthology.task.internal:Checking packages... 2024-04-22T06:57:03.083 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '86f7587a5a09af35f0895e1a2d08527638fae697' 2024-04-22T06:57:03.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T06:57:03.083 INFO:teuthology.packaging:ref: None 2024-04-22T06:57:03.083 INFO:teuthology.packaging:tag: None 2024-04-22T06:57:03.083 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-22T06:57:03.084 INFO:teuthology.packaging:sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:57:03.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-22T06:57:03.342 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.g86f7587a 2024-04-22T06:57:03.343 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T06:57:03.349 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T06:57:03.349 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T06:57:03.375 INFO:teuthology.task.internal:Saving configuration 2024-04-22T06:57:03.395 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T06:57:03.400 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T06:57:03.426 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.658753', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6g0JTthriIoXN5kBBGseqoqWy1JOXp45uI8u2MFVYG9zizTzKOY+zEkhKBbMelmYFy26mSKGWmrXvYDlSEp+E='} 2024-04-22T06:57:03.456 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.662081', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPk9A0djFdO5+ax7FyFdebc7JerxPkuam0fVNNPzK3lJD3HZmwBMDE9OTT8HGce9NbeAQWWiQB2nHL/7E0NuJTg='} 2024-04-22T06:57:03.482 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.660459', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGBpNK4PdUCor6qTZfXUX28WjERiFuxHXPSv/o2ZVN9dqn1K1O5AnqNlRUduwt3ESyYoKq1ZOOQW9Sin5xhXWsA='} 2024-04-22T06:57:03.482 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T06:57:03.488 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-22T06:57:03.488 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-22T06:57:03.488 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-22T06:57:03.488 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T06:57:03.627 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2694f4d0d0>, signals=[15]) 2024-04-22T06:57:03.627 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T06:57:03.633 INFO:teuthology.task.internal:Opening connections... 2024-04-22T06:57:03.633 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T06:57:03.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:57:03.720 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-04-22T06:57:03.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:57:03.804 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-04-22T06:57:03.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:57:03.888 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T06:57:03.894 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-04-22T06:57:03.912 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-04-22T06:57:03.912 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:57:03.969 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:57:03.970 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:57:03.970 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:57:03.970 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-04-22T06:57:04.003 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-04-22T06:57:04.021 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-04-22T06:57:04.021 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-04-22T06:57:04.078 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:57:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:57:04.080 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-04-22T06:57:04.103 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-04-22T06:57:04.121 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-04-22T06:57:04.121 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:57:04.179 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:57:04.180 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-04-22T06:57:04.211 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T06:57:04.221 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T06:57:04.241 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T06:57:04.241 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:57:04.244 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:57:04.246 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:57:04.262 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T06:57:04.267 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T06:57:04.267 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:57:04.301 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:57:04.303 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:57:04.486 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T06:57:04.584 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T06:57:04.597 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-22T06:57:04.598 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-22T06:57:04.598 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-22T06:57:04.598 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:57:04.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:04.615 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-04-22T06:57:04.672 INFO:teuthology.orchestra.run.smithi113.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:57:04.672 INFO:teuthology.task.kernel:Running kernel on smithi113: 5.14.0-437.el9.x86_64 2024-04-22T06:57:04.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum install -y kernel 2024-04-22T06:57:08.093 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-22T06:57:10.119 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-22T06:57:14.488 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-22T06:57:15.870 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-04-22T06:57:17.780 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-22T06:57:22.862 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-22T06:57:24.710 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:57:24.710 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:57:24.802 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-22T06:57:24.803 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-04-22T06:57:24.803 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-22T06:57:24.915 DEBUG:teuthology.orchestra.run.smithi113:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:57:25.409 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:57:22 AM UTC. 2024-04-22T06:57:25.531 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalling: 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T06:57:25.543 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-22T06:57:25.544 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-22T06:57:25.544 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T06:57:25.544 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 255 k 2024-04-22T06:57:25.544 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 0 2024-04-22T06:57:25.544 INFO:teuthology.orchestra.run.smithi113.stderr:Operation aborted. 2024-04-22T06:57:25.600 INFO:teuthology.orchestra.run.smithi113.stdout:Is this ok [y/N]: 2024-04-22T06:57:25.600 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum reinstall -y kernel || true 2024-04-22T06:57:26.089 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 06:57:22 AM UTC. 2024-04-22T06:57:26.204 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalling: 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:================================================================================ 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 255 k 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:Installed size: 0 2024-04-22T06:57:26.205 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-22T06:57:26.839 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-437.el9.x86_64.rpm 492 kB/s | 255 kB 00:00 2024-04-22T06:57:26.839 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:57:26.839 INFO:teuthology.orchestra.run.smithi113.stdout:Total 402 kB/s | 255 kB 00:00 2024-04-22T06:57:26.854 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-22T06:57:27.022 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-22T06:57:27.022 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-22T06:57:27.027 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-22T06:57:27.028 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-22T06:57:27.169 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-22T06:57:27.240 INFO:teuthology.orchestra.run.smithi113.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:27.241 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:27.997 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:27.998 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:28.698 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:28.698 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T06:57:28.698 INFO:teuthology.orchestra.run.smithi113.stdout:Reinstalled: 2024-04-22T06:57:28.698 INFO:teuthology.orchestra.run.smithi113.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:28.698 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T06:57:28.698 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-22T06:57:28.863 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:57:28.910 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:28.911 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:57:28.911 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:57:28.911 DEBUG:teuthology.parallel:result is None 2024-04-22T06:57:28.911 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T06:57:28.920 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T06:57:28.920 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:28.922 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:28.925 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:28.943 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T06:57:28.956 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T06:57:28.973 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T06:57:28.973 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:28.977 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:28.983 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:29.107 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T06:57:29.113 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T06:57:29.113 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:57:29.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:29.129 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:57:29.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:29.146 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:29.148 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:29.171 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:29.172 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:29.186 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:29.208 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:29.223 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:29.225 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:29.250 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:29.251 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T06:57:29.259 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T06:57:29.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:29.261 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:29.267 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:29.319 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T06:57:29.327 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T06:57:29.327 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:29.330 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:29.333 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:29.376 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:29.426 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:29.470 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:29.559 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:29.589 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T06:57:29.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:29.652 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:29.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:29.741 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:29.834 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:29.864 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T06:57:29.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:29.928 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:29.983 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:30.019 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:30.108 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:30.145 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T06:57:30.145 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:30.210 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-04-22T06:57:30.213 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-04-22T06:57:30.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-04-22T06:57:30.243 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:30.243 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:30.290 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:30.620 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T06:57:30.630 INFO:teuthology.task.internal:Starting timer... 2024-04-22T06:57:30.630 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T06:57:30.676 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T06:57:30.691 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-22T06:57:30.692 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-04-22T06:57:30.747 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-04-22T06:57:30.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-04-22T06:57:30.818 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-04-22T06:57:30.819 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-04-22T06:57:30.871 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-04-22T06:57:30.874 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T06:57:30.874 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-04-22T06:57:30.897 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-04-22T06:57:30.897 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-04-22T06:57:30.937 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-04-22T06:57:30.937 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-04-22T06:57:30.973 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-04-22T06:57:30.973 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi113.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive'} 2024-04-22T06:57:30.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:30.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:30.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:31.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:31.026 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:31.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:31.053 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T06:57:31.053 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:31.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:31.106 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:31.137 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T06:57:31.146 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-22T06:57:31.150 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-22T06:57:31.294 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T06:57:31.309 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T06:57:31.311 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi113.front.sepia.ceph.com,smithi133.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T07:02:04.242 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi113.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2024-04-22T07:02:04.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-22T07:02:04.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:02:04.332 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-22T07:02:04.404 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-22T07:02:04.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-22T07:02:04.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:02:04.489 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-04-22T07:02:04.564 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-22T07:02:04.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-22T07:02:04.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:02:04.651 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-04-22T07:02:04.731 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-04-22T07:02:04.732 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T07:02:04.744 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T07:02:04.744 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:02:04.744 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:02:04.747 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:02:04.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:02:04.749 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:02:04.749 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:02:04.773 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:02:04.774 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:02:04.787 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:02:04.787 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:02:04.804 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:02:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-04-22T07:02:04.810 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-04-22T07:02:04.820 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:02:04.820 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-04-22T07:02:04.820 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-04-22T07:02:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:02:04.833 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:02:04.844 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-04-22T07:02:04.846 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:02:04.846 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:02:04.855 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-04-22T07:02:04.869 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:02:04.882 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:02:04.900 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:02:04.903 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:02:04.903 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-22T07:02:04.903 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.903 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.904 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.904 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:02:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-04-22T07:02:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.905 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.937 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:02:04.940 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:02:04.940 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-22T07:02:04.940 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.940 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.940 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.940 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:02:04.942 INFO:teuthology.run_tasks:Running task install... 2024-04-22T07:02:04.950 DEBUG:teuthology.task.install:project ceph 2024-04-22T07:02:04.950 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}} 2024-04-22T07:02:04.950 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'} 2024-04-22T07:02:04.950 INFO:teuthology.task.install:Using flavor: default 2024-04-22T07:02:04.956 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T07:02:04.956 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-22T07:02:04.956 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:02:04.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:02:04.958 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:02:04.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:02:04.960 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:02:04.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:02:05.135 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:02:05.136 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:02:05.196 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:02:05.196 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:02:05.214 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:02:05.214 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:02:05.264 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:02:05.264 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:02:05.264 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:02:05.295 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:02:05.295 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:02:05.359 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:02:05.381 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:02:05.381 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:02:05.381 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:02:05.396 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:02:05.396 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:02:05.396 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:02:05.406 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:02:05.406 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:02:05.423 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:02:05.423 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:02:05.432 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:02:05.471 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:02:05.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:02:05.495 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-04-22T07:02:05.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-04-22T07:02:05.544 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:02:05.563 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:02:05.607 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-04-22T07:02:05.609 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-04-22T07:02:05.625 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-04-22T07:02:05.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-04-22T07:02:05.723 INFO:teuthology.orchestra.run.smithi133.stdout:48 files removed 2024-04-22T07:02:05.750 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:02:05.833 INFO:teuthology.orchestra.run.smithi113.stdout:48 files removed 2024-04-22T07:02:05.854 INFO:teuthology.orchestra.run.smithi154.stdout:48 files removed 2024-04-22T07:02:05.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:02:05.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:02:06.420 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-04-22T07:02:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 247 kB/s | 83 kB 00:00 2024-04-22T07:02:06.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 184 kB/s | 83 kB 00:00 2024-04-22T07:02:06.795 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-22T07:02:06.839 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-22T07:02:06.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-22T07:02:07.171 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-22T07:02:07.265 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-22T07:02:07.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-22T07:02:08.156 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-22T07:02:08.550 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-22T07:02:10.123 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-04-22T07:02:10.592 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-22T07:02:13.830 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-04-22T07:02:16.543 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-04-22T07:02:17.935 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-22T07:02:19.788 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.2 MB 00:02 2024-04-22T07:02:22.681 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 9.3 kB/s | 16 kB 00:01 2024-04-22T07:02:27.877 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 2.1 MB/s | 21 MB 00:09 2024-04-22T07:02:32.851 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-22T07:02:34.454 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 772 kB/s | 19 MB 00:25 2024-04-22T07:02:34.968 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:02:34.969 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:02:34.979 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:02:34.980 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:02:34.981 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:02:34.981 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:02:34.983 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:02:34.985 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:02:34.986 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:02:35.045 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-04-22T07:02:35.053 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-22T07:02:35.053 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-04-22T07:02:35.053 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:02:35.054 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:02:35.055 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:02:35.056 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-04-22T07:02:35.057 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:02:35.058 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:02:35.059 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:02:35.060 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:02:35.061 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:02:35.062 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:02:35.063 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:02:35.064 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:02:35.065 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:02:35.066 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout:Install 148 Packages 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 2 Packages 2024-04-22T07:02:35.067 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:02:35.068 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 218 M 2024-04-22T07:02:35.068 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-04-22T07:02:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-22T07:02:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 4.2 MB/s | 890 kB 00:00 2024-04-22T07:02:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 2.8 MB/s | 145 kB 00:00 2024-04-22T07:02:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-22T07:02:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:(5/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-04-22T07:02:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:(6/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 4.7 MB/s | 5.4 MB 00:01 2024-04-22T07:02:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:(7/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 11 MB/s | 4.7 MB 00:00 2024-04-22T07:02:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:(8/150): ceph-radosgw-19.0.0-3156.g86f7587a.el9 11 MB/s | 11 MB 00:00 2024-04-22T07:02:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 275 kB/s | 25 kB 00:00 2024-04-22T07:02:38.705 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.2 MB 00:00 2024-04-22T07:02:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:(10/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x8 7.9 MB/s | 17 MB 00:02 2024-04-22T07:02:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:(11/150): ceph-common-19.0.0-3156.g86f7587a.el9 6.1 MB/s | 21 MB 00:03 2024-04-22T07:02:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:(12/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 962 kB/s | 225 kB 00:00 2024-04-22T07:02:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:(13/150): cephfs-shell-19.0.0-3156.g86f7587a.el 465 kB/s | 27 kB 00:00 2024-04-22T07:02:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:(14/150): libcephfs-devel-19.0.0-3156.g86f7587a 541 kB/s | 32 kB 00:00 2024-04-22T07:02:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 1.9 MB/s | 164 kB 00:00 2024-04-22T07:02:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:(16/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 4.6 MB/s | 747 kB 00:00 2024-04-22T07:02:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:(17/150): librados-devel-19.0.0-3156.g86f7587a. 1.6 MB/s | 127 kB 00:00 2024-04-22T07:02:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:(18/150): libradosstriper1-19.0.0-3156.g86f7587 5.3 MB/s | 500 kB 00:00 2024-04-22T07:02:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 604 kB/s | 45 kB 00:00 2024-04-22T07:02:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 1.4 MB/s | 135 kB 00:00 2024-04-22T07:02:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 1.7 MB/s | 162 kB 00:00 2024-04-22T07:02:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:(22/150): python3-rados-19.0.0-3156.g86f7587a.e 3.1 MB/s | 320 kB 00:00 2024-04-22T07:02:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:(23/150): python3-rbd-19.0.0-3156.g86f7587a.el9 2.9 MB/s | 298 kB 00:00 2024-04-22T07:02:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:(24/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 8.1 MB/s | 5.4 MB 00:00 2024-04-22T07:02:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:(25/150): python3-rgw-19.0.0-3156.g86f7587a.el9 629 kB/s | 100 kB 00:00 2024-04-22T07:02:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:(26/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-22T07:02:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-22T07:02:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-22T07:02:40.339 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-22T07:02:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 1.5 MB/s | 142 kB 00:00 2024-04-22T07:02:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:(30/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 6.5 MB/s | 3.1 MB 00:00 2024-04-22T07:02:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 7.9 MB/s | 3.5 MB 00:00 2024-04-22T07:02:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 2.4 MB/s | 249 kB 00:00 2024-04-22T07:02:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 454 kB/s | 49 kB 00:00 2024-04-22T07:02:41.341 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-22T07:02:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:(34/150): ceph-test-19.0.0-3156.g86f7587a.el9.x 14 MB/s | 49 MB 00:03 2024-04-22T07:02:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.g8 31 kB/s | 16 kB 00:00 2024-04-22T07:02:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:(36/150): ceph-volume-19.0.0-3156.g86f7587a.el9 4.6 MB/s | 273 kB 00:00 2024-04-22T07:02:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:(37/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 439 kB/s | 18 kB 00:00 2024-04-22T07:02:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:(38/150): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-04-22T07:02:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:(39/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 3.7 MB/s | 763 kB 00:00 2024-04-22T07:02:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 260 kB/s | 80 kB 00:00 2024-04-22T07:02:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 867 kB/s | 195 kB 00:00 2024-04-22T07:02:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-22T07:02:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-22T07:02:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-04-22T07:02:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:(45/150): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-22T07:02:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:(46/150): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-22T07:02:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:(47/150): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-22T07:02:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-22T07:02:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 917 kB/s | 107 kB 00:00 2024-04-22T07:02:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-22T07:02:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-04-22T07:02:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-22T07:02:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-22T07:02:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:(54/150): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-04-22T07:02:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-22T07:02:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-22T07:02:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-22T07:02:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-04-22T07:02:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-22T07:02:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 767 kB/s | 25 kB 00:00 2024-04-22T07:02:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:(61/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.9 MB/s | 3.0 MB 00:00 2024-04-22T07:02:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-04-22T07:02:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:(63/150): libtool-2.4.6-45.el9.x86_64.rpm 2.9 MB/s | 582 kB 00:00 2024-04-22T07:02:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-22T07:02:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-22T07:02:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-04-22T07:02:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-04-22T07:02:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-22T07:02:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:(69/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 19 MB/s | 4.1 MB 00:00 2024-04-22T07:02:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-22T07:02:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-22T07:02:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-22T07:02:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-22T07:02:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-22T07:02:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:(75/150): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-22T07:02:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-22T07:02:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-22T07:02:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-22T07:02:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-22T07:02:43.390 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 21 MB 00:02 2024-04-22T07:02:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-04-22T07:02:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-22T07:02:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-22T07:02:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-22T07:02:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-22T07:02:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-22T07:02:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-22T07:02:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-22T07:02:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-22T07:02:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-22T07:02:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-22T07:02:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-22T07:02:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-22T07:02:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-04-22T07:02:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-22T07:02:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-22T07:02:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-22T07:02:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-22T07:02:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-22T07:02:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-22T07:02:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-22T07:02:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-22T07:02:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:(102/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-22T07:02:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-04-22T07:02:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-04-22T07:02:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 582 kB/s | 19 kB 00:00 2024-04-22T07:02:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:(106/150): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2024-04-22T07:02:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 779 kB/s | 26 kB 00:00 2024-04-22T07:02:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:(108/150): libarrow-9.0.0-10.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-04-22T07:02:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-04-22T07:02:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-22T07:02:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-22T07:02:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-22T07:02:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-04-22T07:02:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-04-22T07:02:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-04-22T07:02:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-22T07:02:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-22T07:02:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-22T07:02:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-22T07:02:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-04-22T07:02:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 4.0 MB/s | 203 kB 00:00 2024-04-22T07:02:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-04-22T07:02:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-22T07:02:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-22T07:02:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-22T07:02:45.054 INFO:teuthology.orchestra.run.smithi113.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-22T07:02:45.079 INFO:teuthology.orchestra.run.smithi113.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-22T07:02:45.105 INFO:teuthology.orchestra.run.smithi113.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-22T07:02:45.130 INFO:teuthology.orchestra.run.smithi113.stdout:(129/150): protobuf-compiler-3.14.0-13.el9.x86_ 791 kB/s | 863 kB 00:01 2024-04-22T07:02:45.155 INFO:teuthology.orchestra.run.smithi113.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-04-22T07:02:45.180 INFO:teuthology.orchestra.run.smithi113.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-22T07:02:45.206 INFO:teuthology.orchestra.run.smithi113.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-22T07:02:45.231 INFO:teuthology.orchestra.run.smithi113.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-22T07:02:45.256 INFO:teuthology.orchestra.run.smithi113.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-22T07:02:45.281 INFO:teuthology.orchestra.run.smithi113.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-22T07:02:45.307 INFO:teuthology.orchestra.run.smithi113.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-22T07:02:45.332 INFO:teuthology.orchestra.run.smithi113.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-04-22T07:02:45.357 INFO:teuthology.orchestra.run.smithi113.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-22T07:02:45.382 INFO:teuthology.orchestra.run.smithi113.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-22T07:02:45.408 INFO:teuthology.orchestra.run.smithi113.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-04-22T07:02:45.442 INFO:teuthology.orchestra.run.smithi113.stdout:(141/150): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-04-22T07:02:45.467 INFO:teuthology.orchestra.run.smithi113.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-22T07:02:45.492 INFO:teuthology.orchestra.run.smithi113.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-22T07:02:45.517 INFO:teuthology.orchestra.run.smithi113.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-22T07:02:45.543 INFO:teuthology.orchestra.run.smithi113.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-22T07:02:45.651 INFO:teuthology.orchestra.run.smithi113.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-22T07:02:45.743 INFO:teuthology.orchestra.run.smithi113.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2024-04-22T07:02:45.785 INFO:teuthology.orchestra.run.smithi113.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-04-22T07:02:46.455 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 8.6 kB/s | 1.7 kB 00:00 2024-04-22T07:02:46.518 INFO:teuthology.orchestra.run.smithi113.stdout:(149/150): librados2-19.0.0-3156.g86f7587a.el9. 3.9 MB/s | 3.4 MB 00:00 2024-04-22T07:02:46.902 INFO:teuthology.orchestra.run.smithi113.stdout:(150/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-04-22T07:02:46.907 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:02:46.910 INFO:teuthology.orchestra.run.smithi113.stdout:Total 18 MB/s | 218 MB 00:11 2024-04-22T07:02:47.944 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-04-22T07:02:48.070 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-04-22T07:02:48.070 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-04-22T07:02:48.515 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-22T07:02:48.538 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:02:48.539 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:02:48.549 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:02:48.550 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:02:48.551 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:02:48.552 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:02:48.553 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:02:48.555 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:02:48.556 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:02:48.615 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-22T07:02:48.635 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-04-22T07:02:48.635 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-22T07:02:48.635 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-04-22T07:02:48.635 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-22T07:02:48.635 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:02:48.635 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:02:48.636 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:02:48.637 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:02:48.638 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:02:48.639 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:02:48.640 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:02:48.641 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:02:48.642 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:02:48.643 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:02:48.644 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:02:48.645 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:02:48.646 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Install 148 Packages 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 2 Packages 2024-04-22T07:02:48.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:02:48.648 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 218 M 2024-04-22T07:02:48.648 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-22T07:02:49.631 INFO:teuthology.orchestra.run.smithi133.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-22T07:02:49.777 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-04-22T07:02:49.777 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-04-22T07:02:49.848 INFO:teuthology.orchestra.run.smithi133.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-22T07:02:49.898 INFO:teuthology.orchestra.run.smithi133.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 2.8 MB/s | 145 kB 00:00 2024-04-22T07:02:50.173 INFO:teuthology.orchestra.run.smithi133.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-22T07:02:50.348 INFO:teuthology.orchestra.run.smithi133.stdout:(5/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-04-22T07:02:50.632 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:02:50.633 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:02:50.643 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:02:50.644 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:02:50.645 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:02:50.646 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:02:50.647 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:02:50.649 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:02:50.650 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:02:50.711 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-04-22T07:02:50.751 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:02:50.752 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:02:50.753 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:02:50.754 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:02:50.755 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:02:50.756 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:02:50.757 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:02:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:02:50.759 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:02:50.760 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:02:50.761 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:02:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout:Install 148 Packages 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 2 Packages 2024-04-22T07:02:50.763 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:02:50.764 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 218 M 2024-04-22T07:02:50.764 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-04-22T07:02:50.907 INFO:teuthology.orchestra.run.smithi133.stdout:(6/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-04-22T07:02:51.282 INFO:teuthology.orchestra.run.smithi133.stdout:(7/150): ceph-common-19.0.0-3156.g86f7587a.el9. 12 MB/s | 21 MB 00:01 2024-04-22T07:02:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:(8/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-04-22T07:02:51.566 INFO:teuthology.orchestra.run.smithi133.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 116 kB/s | 25 kB 00:00 2024-04-22T07:02:51.650 INFO:teuthology.orchestra.run.smithi154.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-22T07:02:52.083 INFO:teuthology.orchestra.run.smithi133.stdout:(10/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 13 MB/s | 11 MB 00:00 2024-04-22T07:02:52.183 INFO:teuthology.orchestra.run.smithi133.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 2.2 MB/s | 225 kB 00:00 2024-04-22T07:02:52.258 INFO:teuthology.orchestra.run.smithi133.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 360 kB/s | 27 kB 00:00 2024-04-22T07:02:52.308 INFO:teuthology.orchestra.run.smithi154.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-22T07:02:52.317 INFO:teuthology.orchestra.run.smithi133.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 546 kB/s | 32 kB 00:00 2024-04-22T07:02:52.417 INFO:teuthology.orchestra.run.smithi133.stdout:(14/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 7.3 MB/s | 747 kB 00:00 2024-04-22T07:02:52.433 INFO:teuthology.orchestra.run.smithi154.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-04-22T07:02:52.484 INFO:teuthology.orchestra.run.smithi133.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 2.4 MB/s | 164 kB 00:00 2024-04-22T07:02:52.660 INFO:teuthology.orchestra.run.smithi133.stdout:(16/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x8 9.6 MB/s | 17 MB 00:01 2024-04-22T07:02:52.685 INFO:teuthology.orchestra.run.smithi133.stdout:(17/150): librados-devel-19.0.0-3156.g86f7587a. 633 kB/s | 127 kB 00:00 2024-04-22T07:02:52.735 INFO:teuthology.orchestra.run.smithi133.stdout:(18/150): libradosstriper1-19.0.0-3156.g86f7587 6.5 MB/s | 500 kB 00:00 2024-04-22T07:02:52.836 INFO:teuthology.orchestra.run.smithi133.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 449 kB/s | 45 kB 00:00 2024-04-22T07:02:52.936 INFO:teuthology.orchestra.run.smithi133.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 1.3 MB/s | 135 kB 00:00 2024-04-22T07:02:53.028 INFO:teuthology.orchestra.run.smithi133.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 1.7 MB/s | 162 kB 00:00 2024-04-22T07:02:53.142 INFO:teuthology.orchestra.run.smithi154.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 3.3 MB/s | 2.4 MB 00:00 2024-04-22T07:02:53.170 INFO:teuthology.orchestra.run.smithi133.stdout:(22/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-22T07:02:53.195 INFO:teuthology.orchestra.run.smithi133.stdout:(23/150): python3-rados-19.0.0-3156.g86f7587a.e 1.9 MB/s | 320 kB 00:00 2024-04-22T07:02:53.245 INFO:teuthology.orchestra.run.smithi133.stdout:(24/150): python3-rbd-19.0.0-3156.g86f7587a.el9 3.9 MB/s | 298 kB 00:00 2024-04-22T07:02:53.279 INFO:teuthology.orchestra.run.smithi133.stdout:(25/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.2 MB/s | 100 kB 00:00 2024-04-22T07:02:53.313 INFO:teuthology.orchestra.run.smithi133.stdout:(26/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-22T07:02:53.317 INFO:teuthology.orchestra.run.smithi154.stdout:(5/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-22T07:02:53.396 INFO:teuthology.orchestra.run.smithi133.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-22T07:02:53.488 INFO:teuthology.orchestra.run.smithi133.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-22T07:02:53.542 INFO:teuthology.orchestra.run.smithi154.stdout:(6/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-22T07:02:53.588 INFO:teuthology.orchestra.run.smithi133.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 1.4 MB/s | 142 kB 00:00 2024-04-22T07:02:53.639 INFO:teuthology.orchestra.run.smithi133.stdout:(30/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 8.6 MB/s | 3.1 MB 00:00 2024-04-22T07:02:54.031 INFO:teuthology.orchestra.run.smithi133.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 8.0 MB/s | 3.5 MB 00:00 2024-04-22T07:02:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 2.4 MB/s | 249 kB 00:00 2024-04-22T07:02:54.215 INFO:teuthology.orchestra.run.smithi133.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 593 kB/s | 49 kB 00:00 2024-04-22T07:02:54.298 INFO:teuthology.orchestra.run.smithi133.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3156.g8 197 kB/s | 16 kB 00:00 2024-04-22T07:02:54.376 INFO:teuthology.orchestra.run.smithi154.stdout:(7/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-04-22T07:02:54.382 INFO:teuthology.orchestra.run.smithi133.stdout:(35/150): ceph-volume-19.0.0-3156.g86f7587a.el9 3.2 MB/s | 273 kB 00:00 2024-04-22T07:02:54.799 INFO:teuthology.orchestra.run.smithi133.stdout:(36/150): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-04-22T07:02:54.832 INFO:teuthology.orchestra.run.smithi133.stdout:(37/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 1.7 MB/s | 763 kB 00:00 2024-04-22T07:02:54.857 INFO:teuthology.orchestra.run.smithi133.stdout:(38/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 312 kB/s | 18 kB 00:00 2024-04-22T07:02:55.000 INFO:teuthology.orchestra.run.smithi133.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-04-22T07:02:55.083 INFO:teuthology.orchestra.run.smithi133.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-22T07:02:55.117 INFO:teuthology.orchestra.run.smithi133.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-22T07:02:55.167 INFO:teuthology.orchestra.run.smithi133.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-22T07:02:55.226 INFO:teuthology.orchestra.run.smithi133.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-22T07:02:55.285 INFO:teuthology.orchestra.run.smithi133.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-22T07:02:55.343 INFO:teuthology.orchestra.run.smithi133.stdout:(45/150): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-22T07:02:55.368 INFO:teuthology.orchestra.run.smithi133.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-22T07:02:55.394 INFO:teuthology.orchestra.run.smithi133.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-22T07:02:55.500 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-04-22T07:02:55.571 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:02:55.596 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:02:55.638 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:02:55.652 INFO:teuthology.orchestra.run.smithi133.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-22T07:02:55.686 INFO:teuthology.orchestra.run.smithi133.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-04-22T07:02:55.711 INFO:teuthology.orchestra.run.smithi133.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-22T07:02:55.753 INFO:teuthology.orchestra.run.smithi133.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-22T07:02:55.787 INFO:teuthology.orchestra.run.smithi133.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-22T07:02:55.812 INFO:teuthology.orchestra.run.smithi133.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-22T07:02:55.870 INFO:teuthology.orchestra.run.smithi133.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-22T07:02:55.884 INFO:teuthology.orchestra.run.smithi154.stdout:(8/150): ceph-common-19.0.0-3156.g86f7587a.el9. 4.7 MB/s | 21 MB 00:04 2024-04-22T07:02:55.929 INFO:teuthology.orchestra.run.smithi133.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-22T07:02:55.934 INFO:teuthology.orchestra.run.smithi154.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 507 kB/s | 25 kB 00:00 2024-04-22T07:02:55.969 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:02:55.988 INFO:teuthology.orchestra.run.smithi133.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-22T07:02:56.038 INFO:teuthology.orchestra.run.smithi133.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-22T07:02:56.063 INFO:teuthology.orchestra.run.smithi133.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-04-22T07:02:56.097 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:56.097 INFO:teuthology.orchestra.run.smithi133.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-22T07:02:56.101 INFO:teuthology.orchestra.run.smithi154.stdout:(10/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 6.1 MB/s | 11 MB 00:01 2024-04-22T07:02:56.139 INFO:teuthology.orchestra.run.smithi133.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-22T07:02:56.164 INFO:teuthology.orchestra.run.smithi133.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-04-22T07:02:56.168 INFO:teuthology.orchestra.run.smithi154.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 3.3 MB/s | 225 kB 00:00 2024-04-22T07:02:56.198 INFO:teuthology.orchestra.run.smithi133.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-22T07:02:56.201 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:56.210 INFO:teuthology.orchestra.run.smithi154.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 651 kB/s | 27 kB 00:00 2024-04-22T07:02:56.223 INFO:teuthology.orchestra.run.smithi133.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-22T07:02:56.231 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:56.252 INFO:teuthology.orchestra.run.smithi154.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 757 kB/s | 32 kB 00:00 2024-04-22T07:02:56.257 INFO:teuthology.orchestra.run.smithi133.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-22T07:02:56.282 INFO:teuthology.orchestra.run.smithi133.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-04-22T07:02:56.306 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:56.348 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:56.377 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:02:56.386 INFO:teuthology.orchestra.run.smithi154.stdout:(14/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 5.4 MB/s | 747 kB 00:00 2024-04-22T07:02:56.402 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:02:56.445 INFO:teuthology.orchestra.run.smithi154.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 2.7 MB/s | 164 kB 00:00 2024-04-22T07:02:56.504 INFO:teuthology.orchestra.run.smithi154.stdout:(16/150): librados-devel-19.0.0-3156.g86f7587a. 2.1 MB/s | 127 kB 00:00 2024-04-22T07:02:56.524 INFO:teuthology.orchestra.run.smithi133.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 17 MB/s | 4.1 MB 00:00 2024-04-22T07:02:56.553 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:02:56.591 INFO:teuthology.orchestra.run.smithi133.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-22T07:02:56.604 INFO:teuthology.orchestra.run.smithi154.stdout:(17/150): libradosstriper1-19.0.0-3156.g86f7587 4.9 MB/s | 500 kB 00:00 2024-04-22T07:02:56.616 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:02:56.645 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:56.666 INFO:teuthology.orchestra.run.smithi133.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-22T07:02:56.700 INFO:teuthology.orchestra.run.smithi133.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-22T07:02:56.725 INFO:teuthology.orchestra.run.smithi133.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-22T07:02:56.748 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:56.771 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:56.817 INFO:teuthology.orchestra.run.smithi133.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-22T07:02:56.834 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:56.839 INFO:teuthology.orchestra.run.smithi154.stdout:(18/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x8 5.1 MB/s | 17 MB 00:03 2024-04-22T07:02:56.856 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:56.879 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:56.914 INFO:teuthology.orchestra.run.smithi154.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 603 kB/s | 45 kB 00:00 2024-04-22T07:02:56.918 INFO:teuthology.orchestra.run.smithi133.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-04-22T07:02:56.971 INFO:teuthology.orchestra.run.smithi154.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 2.3 MB/s | 135 kB 00:00 2024-04-22T07:02:57.002 INFO:teuthology.orchestra.run.smithi133.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-22T07:02:57.022 INFO:teuthology.orchestra.run.smithi154.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 3.2 MB/s | 162 kB 00:00 2024-04-22T07:02:57.039 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:57.072 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:57.073 INFO:teuthology.orchestra.run.smithi133.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-22T07:02:57.115 INFO:teuthology.orchestra.run.smithi154.stdout:(22/150): python3-rados-19.0.0-3156.g86f7587a.e 3.4 MB/s | 320 kB 00:00 2024-04-22T07:02:57.136 INFO:teuthology.orchestra.run.smithi133.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-22T07:02:57.182 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:02:57.185 INFO:teuthology.orchestra.run.smithi133.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-22T07:02:57.207 INFO:teuthology.orchestra.run.smithi154.stdout:(23/150): python3-rbd-19.0.0-3156.g86f7587a.el9 3.2 MB/s | 298 kB 00:00 2024-04-22T07:02:57.267 INFO:teuthology.orchestra.run.smithi154.stdout:(24/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.7 MB/s | 100 kB 00:00 2024-04-22T07:02:57.269 INFO:teuthology.orchestra.run.smithi133.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-22T07:02:57.283 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:02:57.286 INFO:teuthology.orchestra.run.smithi133.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-22T07:02:57.323 INFO:teuthology.orchestra.run.smithi154.stdout:(25/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-22T07:02:57.325 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:02:57.333 INFO:teuthology.orchestra.run.smithi133.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-22T07:02:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-22T07:02:57.355 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:02:57.403 INFO:teuthology.orchestra.run.smithi133.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-22T07:02:57.427 INFO:teuthology.orchestra.run.smithi154.stdout:(26/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 6.6 MB/s | 5.4 MB 00:00 2024-04-22T07:02:57.429 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:02:57.463 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:02:57.470 INFO:teuthology.orchestra.run.smithi133.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-22T07:02:57.473 INFO:teuthology.orchestra.run.smithi154.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-22T07:02:57.506 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:02:57.529 INFO:teuthology.orchestra.run.smithi133.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-22T07:02:57.532 INFO:teuthology.orchestra.run.smithi154.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 683 kB/s | 34 kB 00:00 2024-04-22T07:02:57.539 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:02:57.567 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:02:57.582 INFO:teuthology.orchestra.run.smithi154.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 2.4 MB/s | 142 kB 00:00 2024-04-22T07:02:57.595 INFO:teuthology.orchestra.run.smithi133.stdout:(84/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-04-22T07:02:57.621 INFO:teuthology.orchestra.run.smithi133.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-04-22T07:02:57.636 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:02:57.654 INFO:teuthology.orchestra.run.smithi133.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-22T07:02:57.688 INFO:teuthology.orchestra.run.smithi133.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-22T07:02:57.698 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:02:57.714 INFO:teuthology.orchestra.run.smithi133.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-04-22T07:02:57.728 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:02:57.747 INFO:teuthology.orchestra.run.smithi133.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-04-22T07:02:57.763 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:02:57.772 INFO:teuthology.orchestra.run.smithi133.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-04-22T07:02:57.831 INFO:teuthology.orchestra.run.smithi133.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-22T07:02:57.841 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:02:57.858 INFO:teuthology.orchestra.run.smithi154.stdout:(30/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 5.8 MB/s | 3.1 MB 00:00 2024-04-22T07:02:57.875 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:02:57.890 INFO:teuthology.orchestra.run.smithi133.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-04-22T07:02:57.926 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:02:57.955 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:02:57.957 INFO:teuthology.orchestra.run.smithi133.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-22T07:02:58.015 INFO:teuthology.orchestra.run.smithi133.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-22T07:02:58.027 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:02:58.069 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:02:58.073 INFO:teuthology.orchestra.run.smithi133.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-22T07:02:58.116 INFO:teuthology.orchestra.run.smithi154.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 6.6 MB/s | 3.5 MB 00:00 2024-04-22T07:02:58.145 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:02:58.157 INFO:teuthology.orchestra.run.smithi133.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-22T07:02:58.175 INFO:teuthology.orchestra.run.smithi154.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 4.2 MB/s | 249 kB 00:00 2024-04-22T07:02:58.216 INFO:teuthology.orchestra.run.smithi133.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-04-22T07:02:58.233 INFO:teuthology.orchestra.run.smithi154.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 849 kB/s | 49 kB 00:00 2024-04-22T07:02:58.278 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:02:58.292 INFO:teuthology.orchestra.run.smithi154.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3156.g8 282 kB/s | 16 kB 00:00 2024-04-22T07:02:58.330 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:02:58.359 INFO:teuthology.orchestra.run.smithi154.stdout:(35/150): ceph-volume-19.0.0-3156.g86f7587a.el9 4.0 MB/s | 273 kB 00:00 2024-04-22T07:02:58.408 INFO:teuthology.orchestra.run.smithi133.stdout:(98/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-22T07:02:58.433 INFO:teuthology.orchestra.run.smithi133.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 432 kB/s | 94 kB 00:00 2024-04-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi133.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-22T07:02:58.492 INFO:teuthology.orchestra.run.smithi133.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 759 kB/s | 45 kB 00:00 2024-04-22T07:02:58.510 INFO:teuthology.orchestra.run.smithi154.stdout:(36/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 5.0 MB/s | 763 kB 00:00 2024-04-22T07:02:58.551 INFO:teuthology.orchestra.run.smithi133.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-22T07:02:58.568 INFO:teuthology.orchestra.run.smithi154.stdout:(37/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 313 kB/s | 18 kB 00:00 2024-04-22T07:02:58.601 INFO:teuthology.orchestra.run.smithi133.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-22T07:02:58.751 INFO:teuthology.orchestra.run.smithi133.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.7 MB/s | 548 kB 00:00 2024-04-22T07:02:58.760 INFO:teuthology.orchestra.run.smithi154.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-04-22T07:02:58.777 INFO:teuthology.orchestra.run.smithi133.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-22T07:02:58.802 INFO:teuthology.orchestra.run.smithi133.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-22T07:02:58.852 INFO:teuthology.orchestra.run.smithi133.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-22T07:02:58.856 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:02:58.894 INFO:teuthology.orchestra.run.smithi133.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-22T07:02:58.921 INFO:teuthology.orchestra.run.smithi154.stdout:(39/150): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-22T07:02:58.944 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:02:58.954 INFO:teuthology.orchestra.run.smithi154.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-04-22T07:02:58.972 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:02:58.994 INFO:teuthology.orchestra.run.smithi133.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-04-22T07:02:58.996 INFO:teuthology.orchestra.run.smithi154.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-22T07:02:59.015 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:02:59.053 INFO:teuthology.orchestra.run.smithi133.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-22T07:02:59.053 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:02:59.086 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:02:59.121 INFO:teuthology.orchestra.run.smithi154.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-22T07:02:59.141 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:02:59.180 INFO:teuthology.orchestra.run.smithi154.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-22T07:02:59.205 INFO:teuthology.orchestra.run.smithi154.stdout:(44/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-04-22T07:02:59.215 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:02:59.247 INFO:teuthology.orchestra.run.smithi154.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-22T07:02:59.253 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:02:59.273 INFO:teuthology.orchestra.run.smithi154.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-22T07:02:59.290 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:02:59.323 INFO:teuthology.orchestra.run.smithi154.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-22T07:02:59.338 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:02:59.356 INFO:teuthology.orchestra.run.smithi154.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-04-22T07:02:59.373 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:02:59.378 INFO:teuthology.orchestra.run.smithi133.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-04-22T07:02:59.416 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:02:59.423 INFO:teuthology.orchestra.run.smithi154.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-04-22T07:02:59.449 INFO:teuthology.orchestra.run.smithi154.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-22T07:02:59.458 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:02:59.492 INFO:teuthology.orchestra.run.smithi154.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-22T07:02:59.503 INFO:teuthology.orchestra.run.smithi133.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-22T07:02:59.516 INFO:teuthology.orchestra.run.smithi154.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-22T07:02:59.545 INFO:teuthology.orchestra.run.smithi133.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-22T07:02:59.558 INFO:teuthology.orchestra.run.smithi154.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-22T07:02:59.587 INFO:teuthology.orchestra.run.smithi133.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 775 kB/s | 32 kB 00:00 2024-04-22T07:02:59.596 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:02:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-04-22T07:02:59.642 INFO:teuthology.orchestra.run.smithi154.stdout:(54/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-22T07:02:59.647 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:02:59.667 INFO:teuthology.orchestra.run.smithi154.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-04-22T07:02:59.685 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:02:59.688 INFO:teuthology.orchestra.run.smithi133.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-22T07:02:59.709 INFO:teuthology.orchestra.run.smithi154.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-22T07:02:59.727 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:02:59.734 INFO:teuthology.orchestra.run.smithi154.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-22T07:02:59.771 INFO:teuthology.orchestra.run.smithi133.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-22T07:02:59.776 INFO:teuthology.orchestra.run.smithi154.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-04-22T07:02:59.810 INFO:teuthology.orchestra.run.smithi154.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-22T07:02:59.823 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:02:59.852 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:02:59.860 INFO:teuthology.orchestra.run.smithi154.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-22T07:02:59.880 INFO:teuthology.orchestra.run.smithi133.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-22T07:02:59.885 INFO:teuthology.orchestra.run.smithi154.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-22T07:02:59.892 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:02:59.917 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:02:59.922 INFO:teuthology.orchestra.run.smithi133.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 850 kB/s | 35 kB 00:00 2024-04-22T07:02:59.927 INFO:teuthology.orchestra.run.smithi154.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-04-22T07:02:59.942 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:02:59.952 INFO:teuthology.orchestra.run.smithi154.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-22T07:02:59.989 INFO:teuthology.orchestra.run.smithi133.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-22T07:02:59.994 INFO:teuthology.orchestra.run.smithi154.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-22T07:03:00.012 INFO:teuthology.orchestra.run.smithi154.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 600 kB/s | 35 kB 00:00 2024-04-22T07:03:00.016 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:03:00.145 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:03:00.162 INFO:teuthology.orchestra.run.smithi154.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-22T07:03:00.175 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:03:00.203 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:03:00.206 INFO:teuthology.orchestra.run.smithi133.stdout:(121/150): libarrow-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-04-22T07:03:00.237 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:03:00.238 INFO:teuthology.orchestra.run.smithi154.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 18 MB/s | 4.1 MB 00:00 2024-04-22T07:03:00.264 INFO:teuthology.orchestra.run.smithi133.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-22T07:03:00.281 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:03:00.289 INFO:teuthology.orchestra.run.smithi154.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-22T07:03:00.298 INFO:teuthology.orchestra.run.smithi133.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 321 kB/s | 11 kB 00:00 2024-04-22T07:03:00.319 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:03:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 529 kB/s | 18 kB 00:00 2024-04-22T07:03:00.350 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:03:00.354 INFO:teuthology.orchestra.run.smithi154.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-22T07:03:00.380 INFO:teuthology.orchestra.run.smithi154.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-22T07:03:00.381 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:03:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-22T07:03:00.415 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:03:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 585 kB/s | 19 kB 00:00 2024-04-22T07:03:00.448 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:03:00.472 INFO:teuthology.orchestra.run.smithi154.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-22T07:03:00.474 INFO:teuthology.orchestra.run.smithi133.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-22T07:03:00.490 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:03:00.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:03:00.558 INFO:teuthology.orchestra.run.smithi133.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-04-22T07:03:00.564 INFO:teuthology.orchestra.run.smithi154.stdout:(72/150): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-22T07:03:00.589 INFO:teuthology.orchestra.run.smithi154.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 747 kB/s | 87 kB 00:00 2024-04-22T07:03:00.591 INFO:teuthology.orchestra.run.smithi133.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.3 MB/s | 46 kB 00:00 2024-04-22T07:03:00.613 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:03:00.644 INFO:teuthology.orchestra.run.smithi133.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-22T07:03:00.664 INFO:teuthology.orchestra.run.smithi154.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-22T07:03:00.680 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:03:00.685 INFO:teuthology.orchestra.run.smithi133.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-04-22T07:03:00.711 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:03:00.759 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:03:00.761 INFO:teuthology.orchestra.run.smithi133.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-22T07:03:00.765 INFO:teuthology.orchestra.run.smithi154.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-22T07:03:00.794 INFO:teuthology.orchestra.run.smithi133.stdout:(133/150): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-04-22T07:03:00.820 INFO:teuthology.orchestra.run.smithi133.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-22T07:03:00.845 INFO:teuthology.orchestra.run.smithi133.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-22T07:03:00.857 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:03:00.870 INFO:teuthology.orchestra.run.smithi133.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-22T07:03:00.895 INFO:teuthology.orchestra.run.smithi133.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-04-22T07:03:00.904 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:03:00.921 INFO:teuthology.orchestra.run.smithi133.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-22T07:03:00.962 INFO:teuthology.orchestra.run.smithi133.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-04-22T07:03:00.988 INFO:teuthology.orchestra.run.smithi133.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-04-22T07:03:01.038 INFO:teuthology.orchestra.run.smithi133.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-22T07:03:01.063 INFO:teuthology.orchestra.run.smithi133.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-22T07:03:01.089 INFO:teuthology.orchestra.run.smithi133.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-22T07:03:01.122 INFO:teuthology.orchestra.run.smithi133.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-22T07:03:01.147 INFO:teuthology.orchestra.run.smithi133.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-22T07:03:01.173 INFO:teuthology.orchestra.run.smithi133.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-22T07:03:01.199 INFO:teuthology.orchestra.run.smithi154.stdout:(76/150): ceph-test-19.0.0-3156.g86f7587a.el9.x 9.3 MB/s | 49 MB 00:05 2024-04-22T07:03:01.224 INFO:teuthology.orchestra.run.smithi154.stdout:(77/150): python3-jinja2-2.11.3-5.el9.noarch.rp 443 kB/s | 248 kB 00:00 2024-04-22T07:03:01.248 INFO:teuthology.orchestra.run.smithi133.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-22T07:03:01.249 INFO:teuthology.orchestra.run.smithi154.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 53 kB/s | 26 kB 00:00 2024-04-22T07:03:01.274 INFO:teuthology.orchestra.run.smithi154.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-22T07:03:01.300 INFO:teuthology.orchestra.run.smithi154.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-22T07:03:01.325 INFO:teuthology.orchestra.run.smithi154.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-04-22T07:03:01.384 INFO:teuthology.orchestra.run.smithi154.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-22T07:03:01.500 INFO:teuthology.orchestra.run.smithi154.stdout:(83/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-22T07:03:01.517 INFO:teuthology.orchestra.run.smithi154.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 724 kB/s | 77 kB 00:00 2024-04-22T07:03:01.548 INFO:teuthology.orchestra.run.smithi133.stdout:(148/150): librados2-19.0.0-3156.g86f7587a.el9. 9.0 MB/s | 3.4 MB 00:00 2024-04-22T07:03:01.568 INFO:teuthology.orchestra.run.smithi154.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 42 kB 00:00 2024-04-22T07:03:01.593 INFO:teuthology.orchestra.run.smithi154.stdout:(86/150): python3-oauthlib-3.1.1-5.el9.noarch.r 830 kB/s | 222 kB 00:00 2024-04-22T07:03:01.618 INFO:teuthology.orchestra.run.smithi154.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-22T07:03:01.640 INFO:teuthology.orchestra.run.smithi133.stdout:(149/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-04-22T07:03:01.643 INFO:teuthology.orchestra.run.smithi154.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-22T07:03:01.669 INFO:teuthology.orchestra.run.smithi154.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-22T07:03:01.694 INFO:teuthology.orchestra.run.smithi154.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-22T07:03:01.719 INFO:teuthology.orchestra.run.smithi154.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-04-22T07:03:01.726 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:03:01.768 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:03:01.769 INFO:teuthology.orchestra.run.smithi154.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-04-22T07:03:01.795 INFO:teuthology.orchestra.run.smithi154.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-04-22T07:03:01.802 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:03:01.837 INFO:teuthology.orchestra.run.smithi154.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-22T07:03:01.871 INFO:teuthology.orchestra.run.smithi154.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 475 kB/s | 36 kB 00:00 2024-04-22T07:03:01.902 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:03:01.912 INFO:teuthology.orchestra.run.smithi154.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-04-22T07:03:01.952 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:03:01.954 INFO:teuthology.orchestra.run.smithi154.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-22T07:03:01.997 INFO:teuthology.orchestra.run.smithi154.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 690 kB/s | 58 kB 00:00 2024-04-22T07:03:02.022 INFO:teuthology.orchestra.run.smithi154.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-22T07:03:02.042 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:03:02.079 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:03:02.090 INFO:teuthology.orchestra.run.smithi133.stdout:(150/150): ceph-test-19.0.0-3156.g86f7587a.el9. 4.7 MB/s | 49 MB 00:10 2024-04-22T07:03:02.094 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:03:02.094 INFO:teuthology.orchestra.run.smithi133.stdout:Total 16 MB/s | 218 MB 00:13 2024-04-22T07:03:02.205 INFO:teuthology.orchestra.run.smithi154.stdout:(100/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-22T07:03:02.222 INFO:teuthology.orchestra.run.smithi154.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-04-22T07:03:02.248 INFO:teuthology.orchestra.run.smithi154.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 199 kB/s | 45 kB 00:00 2024-04-22T07:03:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:03:02.273 INFO:teuthology.orchestra.run.smithi154.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-04-22T07:03:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:03:02.372 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:03:02.407 INFO:teuthology.orchestra.run.smithi154.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-22T07:03:02.640 INFO:teuthology.orchestra.run.smithi154.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-04-22T07:03:02.832 INFO:teuthology.orchestra.run.smithi154.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 938 kB/s | 548 kB 00:00 2024-04-22T07:03:02.932 INFO:teuthology.orchestra.run.smithi154.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-22T07:03:02.999 INFO:teuthology.orchestra.run.smithi154.stdout:(108/150): gperftools-libs-2.9.1-2.el9.x86_64.r 425 kB/s | 309 kB 00:00 2024-04-22T07:03:03.023 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:03:03.041 INFO:teuthology.orchestra.run.smithi154.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-04-22T07:03:03.100 INFO:teuthology.orchestra.run.smithi154.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-22T07:03:03.115 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-22T07:03:03.143 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:03:03.150 INFO:teuthology.orchestra.run.smithi154.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-22T07:03:03.241 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-22T07:03:03.241 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-22T07:03:03.258 INFO:teuthology.orchestra.run.smithi154.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-22T07:03:03.401 INFO:teuthology.orchestra.run.smithi154.stdout:(113/150): libarrow-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-04-22T07:03:03.426 INFO:teuthology.orchestra.run.smithi154.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 260 kB/s | 43 kB 00:00 2024-04-22T07:03:03.451 INFO:teuthology.orchestra.run.smithi154.stdout:(115/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-04-22T07:03:03.501 INFO:teuthology.orchestra.run.smithi154.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-04-22T07:03:03.527 INFO:teuthology.orchestra.run.smithi154.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-22T07:03:03.560 INFO:teuthology.orchestra.run.smithi154.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-22T07:03:03.636 INFO:teuthology.orchestra.run.smithi154.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-22T07:03:03.661 INFO:teuthology.orchestra.run.smithi154.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-22T07:03:03.686 INFO:teuthology.orchestra.run.smithi154.stdout:(121/150): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-04-22T07:03:03.745 INFO:teuthology.orchestra.run.smithi154.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-04-22T07:03:03.787 INFO:teuthology.orchestra.run.smithi154.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-22T07:03:03.854 INFO:teuthology.orchestra.run.smithi154.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-22T07:03:03.888 INFO:teuthology.orchestra.run.smithi154.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2024-04-22T07:03:03.954 INFO:teuthology.orchestra.run.smithi154.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-22T07:03:03.988 INFO:teuthology.orchestra.run.smithi154.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-22T07:03:04.055 INFO:teuthology.orchestra.run.smithi154.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-22T07:03:04.106 INFO:teuthology.orchestra.run.smithi154.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-22T07:03:04.164 INFO:teuthology.orchestra.run.smithi154.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-22T07:03:04.206 INFO:teuthology.orchestra.run.smithi154.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-04-22T07:03:04.273 INFO:teuthology.orchestra.run.smithi154.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-22T07:03:04.306 INFO:teuthology.orchestra.run.smithi154.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-22T07:03:04.374 INFO:teuthology.orchestra.run.smithi154.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-22T07:03:04.388 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:03:04.407 INFO:teuthology.orchestra.run.smithi154.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-04-22T07:03:04.449 INFO:teuthology.orchestra.run.smithi154.stdout:(136/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-22T07:03:04.473 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:03:04.474 INFO:teuthology.orchestra.run.smithi154.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-22T07:03:04.501 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:03:04.508 INFO:teuthology.orchestra.run.smithi154.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-22T07:03:04.534 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:03:04.558 INFO:teuthology.orchestra.run.smithi154.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-22T07:03:04.575 INFO:teuthology.orchestra.run.smithi154.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-04-22T07:03:04.600 INFO:teuthology.orchestra.run.smithi154.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-04-22T07:03:04.651 INFO:teuthology.orchestra.run.smithi154.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 552 kB/s | 51 kB 00:00 2024-04-22T07:03:04.684 INFO:teuthology.orchestra.run.smithi154.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-22T07:03:04.710 INFO:teuthology.orchestra.run.smithi154.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-04-22T07:03:04.793 INFO:teuthology.orchestra.run.smithi154.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-22T07:03:04.835 INFO:teuthology.orchestra.run.smithi154.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-04-22T07:03:04.848 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:03:04.888 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:03:04.961 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-22T07:03:04.962 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-22T07:03:04.979 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:03:04.994 INFO:teuthology.orchestra.run.smithi154.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 674 kB/s | 192 kB 00:00 2024-04-22T07:03:05.010 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:03:05.036 INFO:teuthology.orchestra.run.smithi154.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-04-22T07:03:05.046 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:03:05.303 INFO:teuthology.orchestra.run.smithi154.stdout:(149/150): librados2-19.0.0-3156.g86f7587a.el9. 7.3 MB/s | 3.4 MB 00:00 2024-04-22T07:03:05.370 INFO:teuthology.orchestra.run.smithi154.stdout:(150/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 8.4 MB/s | 3.1 MB 00:00 2024-04-22T07:03:05.374 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:03:05.374 INFO:teuthology.orchestra.run.smithi154.stdout:Total 15 MB/s | 218 MB 00:14 2024-04-22T07:03:05.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:03:05.550 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:03:05.621 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:03:05.651 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:03:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-04-22T07:03:06.571 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-04-22T07:03:06.571 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-04-22T07:03:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:08.290 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-04-22T07:03:08.291 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-04-22T07:03:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:03:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:03:08.608 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:03:08.608 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:03:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:03:10.738 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-22T07:03:10.818 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:03:10.843 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:03:10.885 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:03:11.215 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:03:11.335 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:03:11.457 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:03:11.495 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:11.570 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:11.611 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:03:11.640 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:03:11.666 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:03:11.822 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:03:11.879 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:03:11.908 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:12.011 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:12.034 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:03:12.097 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:03:12.118 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:03:12.142 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:03:12.318 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:03:12.351 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:03:12.468 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:03:12.716 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:03:12.752 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:03:12.846 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:03:12.884 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:03:12.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:03:12.960 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:03:12.988 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:03:13.058 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:03:13.111 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:03:13.140 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:03:13.175 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:03:13.261 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:03:13.295 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:03:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:03:13.367 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:03:13.456 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:03:13.498 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:03:13.575 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:03:13.712 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:03:13.769 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:03:14.049 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-04-22T07:03:14.122 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:03:14.156 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:03:14.199 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:03:14.313 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:03:14.398 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:03:14.433 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:03:14.476 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:03:14.514 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:03:14.548 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:03:14.581 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:03:14.604 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:03:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:03:14.678 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:03:14.715 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:03:14.751 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:03:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:03:14.799 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:03:14.808 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:14.834 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:03:14.878 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:03:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:14.920 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:03:14.942 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:03:14.971 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:03:14.997 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:03:15.059 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:03:15.108 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:03:15.138 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:03:15.156 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:03:15.180 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:03:15.219 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:03:15.247 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:15.276 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:03:15.322 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:03:15.351 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:15.362 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:03:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:03:15.387 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:03:15.412 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:03:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:03:15.441 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:03:15.466 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:03:15.488 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:03:15.616 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:03:15.661 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:03:15.666 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:03:15.697 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:03:15.698 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:03:15.752 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:03:15.801 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:03:15.809 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:03:15.839 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:03:15.878 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:03:15.905 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:03:15.908 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:03:15.939 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:03:15.942 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:03:15.975 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:03:16.049 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:03:16.059 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:03:16.099 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:03:16.142 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:03:16.168 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:03:16.175 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:03:16.202 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:03:16.203 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:03:16.273 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:03:16.283 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:03:16.326 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:03:16.332 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:03:16.355 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:03:16.363 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:03:16.391 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:03:16.411 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:03:16.476 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:03:16.510 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:03:16.511 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:03:16.553 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:03:16.566 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:03:16.582 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:03:16.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:03:16.697 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:03:16.773 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:03:16.910 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:03:16.967 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:03:17.310 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:03:17.354 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:03:17.387 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:03:17.490 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:03:17.516 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:03:17.546 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:03:17.605 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:03:17.635 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:03:17.641 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:03:17.680 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:03:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:03:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:03:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:03:17.811 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:03:17.856 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:03:17.884 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:03:17.886 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:03:17.914 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:03:17.951 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:03:17.963 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:03:17.998 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:03:18.033 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:03:18.068 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:03:18.111 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:03:18.251 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:03:18.308 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:03:18.338 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:03:18.380 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:03:18.476 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:03:18.504 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:03:18.544 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:03:18.578 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:03:18.603 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:03:18.626 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:03:18.680 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:03:18.752 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:03:18.810 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:03:18.844 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:03:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:03:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:03:18.951 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:03:18.989 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:03:19.028 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:03:19.057 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:03:19.084 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:03:19.125 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:03:19.167 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:03:19.210 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:03:19.300 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:03:19.350 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:03:19.380 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:03:19.551 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:03:19.798 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-04-22T07:03:19.946 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:20.008 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:03:20.016 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:03:20.081 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:03:20.109 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:03:20.140 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:03:20.142 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:03:20.155 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:03:20.156 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:20.156 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:03:20.156 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:20.454 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:03:20.497 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:03:20.588 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:03:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:03:20.654 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:03:20.771 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:03:20.812 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:03:20.846 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:03:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:03:21.004 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:03:21.095 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:03:21.132 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:03:21.139 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:03:21.200 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:03:21.270 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:03:21.293 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:03:21.318 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:03:21.346 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:03:21.424 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:03:21.644 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:03:21.669 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:03:21.669 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:21.670 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:03:21.670 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:03:21.670 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:03:21.670 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:21.894 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:03:21.916 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:03:21.916 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:21.916 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:03:21.917 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:03:21.917 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:03:21.917 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:22.092 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:03:22.205 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:03:22.361 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:03:22.383 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:03:22.383 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:22.383 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:03:22.384 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:03:22.384 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:03:22.384 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:22.483 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:03:22.531 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:03:22.587 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:03:22.772 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:03:22.821 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:03:22.898 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:03:22.985 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:03:23.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:03:23.162 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:03:23.195 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:03:23.222 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:03:23.484 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:03:23.567 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:03:23.595 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:03:23.628 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:03:23.666 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:23.666 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:03:23.696 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:03:23.707 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:23.728 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:03:23.790 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:03:23.819 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:03:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:03:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:03:24.073 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:03:24.104 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:03:24.140 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:03:24.249 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:24.294 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:03:24.370 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:03:24.370 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:03:24.371 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:03:24.371 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:24.395 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:03:24.619 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:03:24.644 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:03:24.714 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:03:24.737 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:03:24.858 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:03:24.895 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:03:25.854 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:03:25.879 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:03:25.980 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:03:26.134 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:03:26.156 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:03:26.179 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:03:26.179 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:26.179 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:03:26.179 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:03:26.179 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:03:26.180 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:26.224 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:03:26.235 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:03:26.333 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:03:26.380 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:03:27.028 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:27.067 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:27.307 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:03:27.334 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:03:27.357 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:03:27.357 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:27.357 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:03:27.357 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:03:27.357 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:03:27.357 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:27.402 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:03:27.423 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:03:27.423 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:27.424 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:03:27.424 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:27.469 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:03:27.492 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:03:27.492 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:27.492 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:03:27.492 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:03:27.492 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:03:27.492 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:27.609 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:03:27.654 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:03:27.753 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:03:27.753 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:03:27.753 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:03:27.753 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:27.780 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:03:27.795 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:03:27.817 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:03:27.817 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:27.817 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:03:27.817 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:03:27.818 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:03:27.818 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:32.423 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:03:32.464 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:03:32.505 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:03:32.546 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:03:32.573 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:03:32.754 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:03:32.802 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:03:32.832 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:03:33.421 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:33.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:33.487 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:34.268 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:03:34.363 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:03:34.403 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:03:34.485 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:03:34.525 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:03:34.525 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:03:34.634 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:03:34.634 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:35.601 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:03:35.601 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-04-22T07:03:35.601 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-04-22T07:03:35.602 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-04-22T07:03:35.602 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-04-22T07:03:35.602 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-04-22T07:03:35.602 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-04-22T07:03:35.602 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-04-22T07:03:35.602 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:35.772 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:03:35.787 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:03:35.788 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:35.788 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:03:35.788 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:37.275 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:03:37.300 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:03:37.300 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:37.300 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:03:37.300 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:03:37.300 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:03:37.301 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:37.534 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:03:37.557 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:03:37.557 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:37.557 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:03:37.557 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:03:37.557 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:03:37.557 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:38.014 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:03:38.036 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:03:38.036 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:38.036 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:03:38.036 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:03:38.036 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:03:38.036 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:38.140 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:03:38.197 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:03:38.252 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:03:38.439 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:03:38.494 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:03:38.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:03:38.661 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:03:38.802 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:03:38.852 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:03:38.884 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:03:38.912 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:03:39.232 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-04-22T07:03:39.233 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:39.330 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:03:39.360 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:03:39.392 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:03:39.435 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:03:39.451 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:03:39.451 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:39.451 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:03:39.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:03:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:03:40.523 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:03:40.560 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:03:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:03:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:03:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:03:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:03:40.816 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:03:40.817 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:03:40.818 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:03:40.819 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:03:40.819 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:03:40.831 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:03:40.831 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:03:40.831 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:03:40.831 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:03:40.832 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:03:40.833 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:03:40.834 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:03:40.834 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:03:40.834 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:03:40.834 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:03:40.834 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:03:40.838 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:03:40.839 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:03:40.840 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:03:40.841 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:03:40.842 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:03:40.843 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:03:40.844 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:03:40.845 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:03:40.930 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:03:40.956 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:03:40.956 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:40.956 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:03:40.956 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:03:40.956 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:03:40.956 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:41.189 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:03:41.212 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:03:41.212 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:41.212 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:03:41.212 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:03:41.212 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:03:41.212 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:41.524 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:03:41.552 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:03:41.654 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:03:41.656 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:03:41.679 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:03:41.679 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:41.679 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:03:41.679 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:03:41.679 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:03:41.679 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:41.795 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:03:41.824 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:03:41.852 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:03:41.854 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:03:41.877 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:03:41.877 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:41.877 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:03:41.877 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:03:41.877 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:03:41.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:41.908 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:03:41.930 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:03:41.941 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:03:42.047 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:03:42.097 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:03:42.104 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:03:42.157 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:03:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:03:42.326 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:03:42.467 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:03:42.516 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:03:42.548 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:03:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-04-22T07:03:42.905 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.906 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.907 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:03:42.908 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:03:42.909 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:03:42.909 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:03:42.909 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:03:42.909 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.909 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.910 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:03:42.911 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.912 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:03:42.913 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:03:42.914 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:03:42.915 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.916 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:03:42.917 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:03:42.918 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:03:42.919 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:03:42.919 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:03:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-04-22T07:03:43.003 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:03:43.026 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:03:43.033 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:03:43.056 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:03:43.057 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:03:43.079 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:03:43.079 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:03:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:03:43.080 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:03:43.080 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:43.118 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:03:43.133 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:03:43.147 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:03:43.154 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:03:43.154 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:43.155 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:03:43.155 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:43.200 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:03:43.222 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:03:43.223 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:43.223 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:03:43.223 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:03:43.223 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:03:43.223 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:43.268 DEBUG:teuthology.parallel:result is None 2024-04-22T07:03:43.526 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:03:43.548 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:03:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:03:43.548 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:03:43.549 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:03:43.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:44.196 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:03:44.241 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:03:45.206 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:03:45.241 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:03:45.344 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:03:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:03:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:03:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:03:45.549 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:45.549 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:03:45.549 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:03:45.550 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:03:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:45.594 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:03:45.605 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:03:45.711 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:03:45.767 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:03:46.688 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:03:46.721 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:03:46.744 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:03:46.744 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:46.744 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:03:46.744 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:03:46.744 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:03:46.744 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:46.789 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:03:46.811 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:03:46.811 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:46.811 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:03:46.811 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:46.857 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:03:46.878 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:03:46.878 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:46.878 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:03:46.878 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:03:46.878 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:03:46.878 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:47.182 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:03:47.204 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:03:47.205 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:03:47.205 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:03:47.205 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:03:47.205 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:03:47.205 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:03:48.171 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:03:48.220 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:03:48.261 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:03:48.293 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:03:48.320 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:03:48.506 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:03:48.541 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:03:48.571 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:03:49.196 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:49.236 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:49.256 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:50.030 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:03:50.123 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:03:50.164 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:03:50.248 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:03:50.294 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:03:50.294 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:03:50.361 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:03:50.361 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:51.802 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:03:51.851 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:03:51.892 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:03:51.924 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:03:51.951 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:03:52.143 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:03:52.189 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:03:52.219 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:03:52.846 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:52.884 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:52.904 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:03:53.702 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:03:53.797 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:03:53.838 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:03:53.921 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:03:53.951 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:03:53.951 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:03:54.043 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:03:54.043 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:03:56.410 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:03:56.411 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:03:56.412 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:03:56.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:03:56.424 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:03:56.425 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:03:56.426 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:03:56.427 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:03:56.428 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:03:56.429 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:03:56.430 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:03:56.431 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:03:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-22T07:03:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:03:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:03:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:03:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:03:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:03:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:03:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:03:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:03:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:03:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:03:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-22T07:03:58.512 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-22T07:03:58.876 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:04:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:04:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:04:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:04:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:04:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:04:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:04:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:04:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:04:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:04:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:04:00.056 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:04:00.057 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:04:00.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:04:00.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:04:00.060 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:04:00.061 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:04:00.062 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:04:00.063 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:04:00.064 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:04:00.065 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:04:00.066 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:04:00.067 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:04:00.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:04:00.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:04:00.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:04:00.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:04:00.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:04:00.068 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:04:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:04:02.178 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:04:02.179 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.180 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.181 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:04:02.182 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:04:02.183 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:04:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:04:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:04:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:04:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:04:02.184 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:04:02.185 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:04:02.186 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:04:02.187 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:04:02.188 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:04:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:04:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:04:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:04:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:04:02.189 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:04:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:04:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:04:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:04:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:04:02.190 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:04:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:04:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:04:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.191 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:04:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:04:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:04:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:04:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:04:02.192 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:04:02.193 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:04:02.194 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:04:02.195 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:04:02.195 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:04:02.195 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:04:02.195 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-04-22T07:04:02.535 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:02.536 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:02.727 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:04:02.754 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:04:02.754 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:04:02.755 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:04:02.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:02.922 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:04:02.947 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:04:02.947 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:04:02.947 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:04:02.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:03.193 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:04:03.215 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:04:03.215 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:04:03.215 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:04:03.217 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T07:04:03.217 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:04:03.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:04:03.247 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:04:03.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:04:03.279 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:04:03.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:04:03.304 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T07:04:03.305 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:04:03.305 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:04:03.334 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:04:03.400 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:04:03.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:04:03.425 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:04:03.489 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:04:03.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:04:03.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:04:03.577 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T07:04:03.577 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:04:03.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:04:03.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:04:03.672 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:04:03.672 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:04:03.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:04:03.761 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:04:03.761 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:04:03.785 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:04:03.848 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T07:04:03.849 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:04:03.849 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:04:03.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:04:03.941 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:04:03.941 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:04:03.965 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:04:04.030 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:04:04.030 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:04:04.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:04:04.117 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-22T07:04:04.238 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-22T07:04:04.239 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:04.239 INFO:tasks.cephadm:Cluster fsid is 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:04:04.239 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-22T07:04:04.239 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.113', 'mon.b': '172.21.15.133', 'mon.c': '172.21.15.154'} 2024-04-22T07:04:04.240 INFO:tasks.cephadm:First mon is mon.a on smithi113 2024-04-22T07:04:04.240 INFO:tasks.cephadm:First mgr is x 2024-04-22T07:04:04.240 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-22T07:04:04.240 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname $(hostname -s) 2024-04-22T07:04:04.267 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-04-22T07:04:04.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname $(hostname -s) 2024-04-22T07:04:04.316 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-22T07:04:04.316 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:04.479 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78786/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-g86f7587a', 'node_name': '172.21.4.69+confusa07', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.g86f7587a'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-21 22:42:52.445656', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78786/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-g86f7587a', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.g86f7587a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-21 20:57:03.749940', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-22T07:04:04.722 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 86f7587a5a09af35f0895e1a2d08527638fae697 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:04.724 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/x86_64/flavors/default/cephadm 2024-04-22T07:04:04.724 DEBUG:teuthology.orchestra.run.smithi113:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:05.176 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r--. 1 ubuntu ubuntu 781637 Apr 22 07:04 /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:05.177 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:05.600 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 781637 Apr 22 07:04 /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:05.600 DEBUG:teuthology.orchestra.run.smithi154:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:06.194 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 781637 Apr 22 07:04 /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:06.195 DEBUG:teuthology.orchestra.run.smithi113:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:06.214 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:06.232 DEBUG:teuthology.orchestra.run.smithi154:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T07:04:06.269 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 on all hosts... 2024-04-22T07:04:06.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 pull 2024-04-22T07:04:06.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 pull 2024-04-22T07:04:06.275 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 pull 2024-04-22T07:04:07.749 INFO:teuthology.orchestra.run.smithi154.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697... 2024-04-22T07:04:07.853 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697... 2024-04-22T07:04:07.919 INFO:teuthology.orchestra.run.smithi113.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697... 2024-04-22T07:04:37.125 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-22T07:04:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T07:04:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61", 2024-04-22T07:04:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-04-22T07:04:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2" 2024-04-22T07:04:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-22T07:04:37.126 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-22T07:04:49.430 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:04:49.430 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T07:04:49.430 INFO:teuthology.orchestra.run.smithi113.stdout: "image_id": "1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61", 2024-04-22T07:04:49.431 INFO:teuthology.orchestra.run.smithi113.stdout: "repo_digests": [ 2024-04-22T07:04:49.431 INFO:teuthology.orchestra.run.smithi113.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2" 2024-04-22T07:04:49.431 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:04:49.431 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:04:49.688 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-22T07:04:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T07:04:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "image_id": "1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61", 2024-04-22T07:04:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "repo_digests": [ 2024-04-22T07:04:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2" 2024-04-22T07:04:49.689 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-04-22T07:04:49.689 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-22T07:04:49.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph 2024-04-22T07:04:49.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-04-22T07:04:49.771 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph 2024-04-22T07:04:49.798 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /etc/ceph 2024-04-22T07:04:49.821 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-04-22T07:04:49.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /etc/ceph 2024-04-22T07:04:49.871 INFO:tasks.cephadm:Writing seed config... 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-22T07:04:49.873 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-22T07:04:49.874 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-22T07:04:49.875 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-22T07:04:49.876 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-22T07:04:49.877 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:04:49.877 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-22T07:04:49.891 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 81035712-0076-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-22T07:04:49.892 DEBUG:teuthology.orchestra.run.smithi113:mon.a> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a.service 2024-04-22T07:04:49.934 DEBUG:teuthology.orchestra.run.smithi113:mgr.x> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x.service 2024-04-22T07:04:49.976 INFO:tasks.cephadm:Bootstrapping... 2024-04-22T07:04:49.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 -v bootstrap --fsid 81035712-0076-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.113 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:04:50.167 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:04:50.167 INFO:teuthology.orchestra.run.smithi113.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697', '-v', 'bootstrap', '--fsid', '81035712-0076-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.113', '--skip-admin-label'] 2024-04-22T07:04:50.167 INFO:teuthology.orchestra.run.smithi113.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-22T07:04:50.167 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying podman|docker is present... 2024-04-22T07:04:50.204 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T07:04:50.204 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying lvm2 is present... 2024-04-22T07:04:50.204 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying time synchronization is in place... 2024-04-22T07:04:50.211 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T07:04:50.212 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T07:04:50.217 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T07:04:50.218 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout inactive 2024-04-22T07:04:50.224 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout enabled 2024-04-22T07:04:50.230 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout active 2024-04-22T07:04:50.230 INFO:teuthology.orchestra.run.smithi113.stdout:Unit chronyd.service is enabled and running 2024-04-22T07:04:50.230 INFO:teuthology.orchestra.run.smithi113.stdout:Repeating the final host check... 2024-04-22T07:04:50.253 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T07:04:50.253 INFO:teuthology.orchestra.run.smithi113.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-22T07:04:50.254 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl is present 2024-04-22T07:04:50.254 INFO:teuthology.orchestra.run.smithi113.stdout:lvcreate is present 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout inactive 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout enabled 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout active 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:Unit chronyd.service is enabled and running 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:Host looks OK 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:Cluster fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:Acquiring lock 140418501640144 on /run/cephadm/81035712-0076-11ef-bc93-c7b262605968.lock 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:Lock 140418501640144 acquired on /run/cephadm/81035712-0076-11ef-bc93-c7b262605968.lock 2024-04-22T07:04:50.278 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying IP 172.21.15.113 port 3300 ... 2024-04-22T07:04:50.279 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying IP 172.21.15.113 port 6789 ... 2024-04-22T07:04:50.279 INFO:teuthology.orchestra.run.smithi113.stdout:Base mon IP(s) is [172.21.15.113:3300, 172.21.15.113:6789], mon addrv is [v2:172.21.15.113:3300,v1:172.21.15.113:6789] 2024-04-22T07:04:50.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.113 metric 100 2024-04-22T07:04:50.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.113 metric 100 2024-04-22T07:04:50.285 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-22T07:04:50.285 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-22T07:04:50.285 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-22T07:04:50.287 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-22T07:04:50.287 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-22T07:04:50.287 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T07:04:50.287 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-22T07:04:50.287 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout inet6 fe80::6e08:c44e:94c5:8ba4/64 scope link noprefixroute 2024-04-22T07:04:50.287 INFO:teuthology.orchestra.run.smithi113.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi113.stdout:Mon IP `172.21.15.113` is in CIDR network `172.21.0.0/20` 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi113.stdout:Mon IP `172.21.15.113` is in CIDR network `172.21.0.0/20` 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi113.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi113.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi113.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697... 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stdout 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697... 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stderr Getting image source signatures 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stderr Copying blob sha256:3f93b098358ed33bdcfea942070678da0220cbea06d5b0225ae8923eb855cc02 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stderr Copying config sha256:1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 2024-04-22T07:04:50.783 INFO:teuthology.orchestra.run.smithi113.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-22T07:04:51.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph: stdout ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:04:51.645 INFO:teuthology.orchestra.run.smithi113.stdout:Ceph version: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:04:51.646 INFO:teuthology.orchestra.run.smithi113.stdout:Extracting ceph user uid/gid from container image... 2024-04-22T07:04:52.526 INFO:teuthology.orchestra.run.smithi113.stdout:stat: stdout 167 167 2024-04-22T07:04:52.526 INFO:teuthology.orchestra.run.smithi113.stdout:Creating initial keys... 2024-04-22T07:04:53.416 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph-authtool: stdout AQAUDCZm8JNdMhAAGX2IEZnoCYOiVEUkn/EbXQ== 2024-04-22T07:04:54.358 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph-authtool: stdout AQAVDCZmMAiXLhAAWhpMyDWaPkiUGHiZr+MHiw== 2024-04-22T07:04:55.233 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph-authtool: stdout AQAWDCZm2oxuJxAAZ067KJ2N50Pjz1uB0nxh5A== 2024-04-22T07:04:55.233 INFO:teuthology.orchestra.run.smithi113.stdout:Creating initial monmap... 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool for a [v2:172.21.15.113:3300,v1:172.21.15.113:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:setting min_mon_release = quincy 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/monmaptool: set fsid to 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:04:56.078 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T07:04:56.079 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:04:56.079 INFO:teuthology.orchestra.run.smithi113.stdout:Creating mon... 2024-04-22T07:04:56.984 INFO:teuthology.orchestra.run.smithi113.stdout:create mon.a on 2024-04-22T07:04:57.182 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-22T07:04:57.330 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-22T07:04:57.502 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-81035712-0076-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-81035712-0076-11ef-bc93-c7b262605968.target. 2024-04-22T07:04:57.502 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-81035712-0076-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-81035712-0076-11ef-bc93-c7b262605968.target. 2024-04-22T07:04:57.731 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a 2024-04-22T07:04:57.731 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Failed to reset failed state of unit ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-22T07:04:57.890 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-81035712-0076-11ef-bc93-c7b262605968.target.wants/ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-81035712-0076-11ef-bc93-c7b262605968@.service. 2024-04-22T07:04:58.435 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T07:04:58.435 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout disabled 2024-04-22T07:04:58.442 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T07:04:58.442 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout inactive 2024-04-22T07:04:58.442 INFO:teuthology.orchestra.run.smithi113.stdout:firewalld.service is not enabled 2024-04-22T07:04:58.442 INFO:teuthology.orchestra.run.smithi113.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T07:04:58.442 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for mon to start... 2024-04-22T07:04:58.442 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for mon... 2024-04-22T07:04:59.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout cluster: 2024-04-22T07:04:59.572 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout id: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:04:59.572 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout services: 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.576759s) 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout data: 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout pgs: 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.883+0000 7fdb6225e700 1 Processor -- start 2024-04-22T07:04:59.573 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.883+0000 7fdb6225e700 1 -- start start 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.883+0000 7fdb6225e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c108b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.883+0000 7fdb6225e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c109110 con 0x7fdb5c108720 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.884+0000 7fdb5b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c108b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.884+0000 7fdb5b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c108b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54426/0 (socket says 172.21.15.113:54426) 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.884+0000 7fdb5b7fe700 1 -- 172.21.15.113:0/397893132 learned_addr learned my addr 172.21.15.113:0/397893132 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.884+0000 7fdb5b7fe700 1 -- 172.21.15.113:0/397893132 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c109920 con 0x7fdb5c108720 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.884+0000 7fdb5b7fe700 1 --2- 172.21.15.113:0/397893132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c108b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009e80 tx=0x7fdb4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e0c7a1b922a775 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.574 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.884+0000 7fdb5a7fc700 1 -- 172.21.15.113:0/397893132 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb5c108720 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb5a7fc700 1 -- 172.21.15.113:0/397893132 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c003a20 con 0x7fdb5c108720 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb5a7fc700 1 -- 172.21.15.113:0/397893132 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb4c003d20 con 0x7fdb5c108720 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 -- 172.21.15.113:0/397893132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 msgr2=0x7fdb5c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 --2- 172.21.15.113:0/397893132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c108b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009e80 tx=0x7fdb4c00b330 comp rx=0 tx=0).stop 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 -- 172.21.15.113:0/397893132 shutdown_connections 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 --2- 172.21.15.113:0/397893132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c108b40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 -- 172.21.15.113:0/397893132 >> 172.21.15.113:0/397893132 conn(0x7fdb5c103d20 msgr2=0x7fdb5c106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 -- 172.21.15.113:0/397893132 shutdown_connections 2024-04-22T07:04:59.577 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.885+0000 7fdb6225e700 1 -- 172.21.15.113:0/397893132 wait complete. 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb6225e700 1 Processor -- start 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb6225e700 1 -- start start 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb6225e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c07eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb6225e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c109e50 con 0x7fdb5c108720 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb5b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c07eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb5b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c07eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54442/0 (socket says 172.21.15.113:54442) 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.886+0000 7fdb5b7fe700 1 -- 172.21.15.113:0/2314428723 learned_addr learned my addr 172.21.15.113:0/2314428723 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb5b7fe700 1 -- 172.21.15.113:0/2314428723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c07f090 con 0x7fdb5c108720 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb5b7fe700 1 --2- 172.21.15.113:0/2314428723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c07eb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c004000 tx=0x7fdb4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb4c022020 con 0x7fdb5c108720 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c018040 con 0x7fdb5c108720 2024-04-22T07:04:59.578 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c07d280 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c07d7c0 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb4c025020 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.887+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdb4c00f050 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.888+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c0681b0 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.890+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdb4c02c030 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.959+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdb5c195700 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb58ff9700 1 -- 172.21.15.113:0/2314428723 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdb4c01e480 con 0x7fdb5c108720 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 msgr2=0x7fdb5c07eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb6225e700 1 --2- 172.21.15.113:0/2314428723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c07eb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c004000 tx=0x7fdb4c004ad0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 shutdown_connections 2024-04-22T07:04:59.579 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb6225e700 1 --2- 172.21.15.113:0/2314428723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5c108720 0x7fdb5c07eb50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.580 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 >> 172.21.15.113:0/2314428723 conn(0x7fdb5c103d20 msgr2=0x7fdb5c192b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.580 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.960+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 shutdown_connections 2024-04-22T07:04:59.580 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:58.961+0000 7fdb6225e700 1 -- 172.21.15.113:0/2314428723 wait complete. 2024-04-22T07:04:59.580 INFO:teuthology.orchestra.run.smithi113.stdout:mon is available 2024-04-22T07:04:59.580 INFO:teuthology.orchestra.run.smithi113.stdout:Assimilating anything we can from ceph.conf... 2024-04-22T07:05:00.581 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:00 smithi113 ceph-mon[27555]: from='client.? 172.21.15.113:0/2874617203' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-22T07:05:00.589 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:00.589 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout fsid = 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.113:3300,v1:172.21.15.113:6789] 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T07:05:00.590 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5e5213700 1 Processor -- start 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5e5213700 1 -- start start 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5e5213700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e0108bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5e5213700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0109180 con 0x7fb5e0108790 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5ded9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e0108bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5ded9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e0108bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54456/0 (socket says 172.21.15.113:54456) 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5ded9d700 1 -- 172.21.15.113:0/720373786 learned_addr learned my addr 172.21.15.113:0/720373786 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5ded9d700 1 -- 172.21.15.113:0/720373786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e0109940 con 0x7fb5e0108790 2024-04-22T07:05:00.591 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:04:59.999+0000 7fb5ded9d700 1 --2- 172.21.15.113:0/720373786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e0108bb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00098a0 tx=0x7fb5d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4009bd637ee4346 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5ddd9b700 1 -- 172.21.15.113:0/720373786 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d0004160 con 0x7fb5e0108790 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5ddd9b700 1 -- 172.21.15.113:0/720373786 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d0004300 con 0x7fb5e0108790 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5ddd9b700 1 -- 172.21.15.113:0/720373786 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d00036a0 con 0x7fb5e0108790 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 -- 172.21.15.113:0/720373786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 msgr2=0x7fb5e0108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 --2- 172.21.15.113:0/720373786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e0108bb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00098a0 tx=0x7fb5d0009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 -- 172.21.15.113:0/720373786 shutdown_connections 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 --2- 172.21.15.113:0/720373786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e0108bb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 -- 172.21.15.113:0/720373786 >> 172.21.15.113:0/720373786 conn(0x7fb5e0103d30 msgr2=0x7fb5e0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 -- 172.21.15.113:0/720373786 shutdown_connections 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.000+0000 7fb5e5213700 1 -- 172.21.15.113:0/720373786 wait complete. 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5e5213700 1 Processor -- start 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5e5213700 1 -- start start 2024-04-22T07:05:00.592 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5e5213700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5e5213700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0109e70 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5ded9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5ded9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54466/0 (socket says 172.21.15.113:54466) 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.001+0000 7fb5ded9d700 1 -- 172.21.15.113:0/2874617203 learned_addr learned my addr 172.21.15.113:0/2874617203 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5ded9d700 1 -- 172.21.15.113:0/2874617203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e019c3e0 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5ded9d700 1 --2- 172.21.15.113:0/2874617203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e019bea0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0000c00 tx=0x7fb5d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d0003db0 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d0011630 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e019c670 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d0011950 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e019f360 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb5d001a460 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.002+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb5d001baf0 con 0x7fb5e0108790 2024-04-22T07:05:00.593 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.003+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e00681b0 con 0x7fb5e0108790 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.005+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb5d0031020 con 0x7fb5e0108790 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.073+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb5e019cb50 con 0x7fb5e0108790 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.076+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7fb5e019cb50 con 0x7fb5e0108790 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.076+0000 7fb5c7fff700 1 -- 172.21.15.113:0/2874617203 <== mon.0 v2:172.21.15.113:3300/0 8 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fb5d0004260 con 0x7fb5e0108790 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 msgr2=0x7fb5e019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 --2- 172.21.15.113:0/2874617203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e019bea0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0000c00 tx=0x7fb5d0004750 comp rx=0 tx=0).stop 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 shutdown_connections 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 --2- 172.21.15.113:0/2874617203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e0108790 0x7fb5e019bea0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 >> 172.21.15.113:0/2874617203 conn(0x7fb5e0103d30 msgr2=0x7fb5e0192b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 shutdown_connections 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:00.077+0000 7fb5e5213700 1 -- 172.21.15.113:0/2874617203 wait complete. 2024-04-22T07:05:00.594 INFO:teuthology.orchestra.run.smithi113.stdout:Generating new minimal ceph.conf... 2024-04-22T07:05:01.628 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f1705e75700 1 Processor -- start 2024-04-22T07:05:01.630 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f1705e75700 1 -- start start 2024-04-22T07:05:01.630 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f1705e75700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170007d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f1705e75700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f170007d7e0 con 0x7f170007edf0 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170007d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170007d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54474/0 (socket says 172.21.15.113:54474) 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f16ff7fe700 1 -- 172.21.15.113:0/3614797267 learned_addr learned my addr 172.21.15.113:0/3614797267 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.012+0000 7f16ff7fe700 1 -- 172.21.15.113:0/3614797267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170007d960 con 0x7f170007edf0 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f16ff7fe700 1 --2- 172.21.15.113:0/3614797267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170007d2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f16f00098a0 tx=0x7f16f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d69f9f4bbb2bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.631 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f16fe7fc700 1 -- 172.21.15.113:0/3614797267 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0004160 con 0x7f170007edf0 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f16fe7fc700 1 -- 172.21.15.113:0/3614797267 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f16f0004300 con 0x7f170007edf0 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f16fe7fc700 1 -- 172.21.15.113:0/3614797267 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0011b20 con 0x7f170007edf0 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 -- 172.21.15.113:0/3614797267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 msgr2=0x7f170007d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 --2- 172.21.15.113:0/3614797267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170007d2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f16f00098a0 tx=0x7f16f0009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 -- 172.21.15.113:0/3614797267 shutdown_connections 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 --2- 172.21.15.113:0/3614797267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170007d2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 -- 172.21.15.113:0/3614797267 >> 172.21.15.113:0/3614797267 conn(0x7f1700103df0 msgr2=0x7f1700106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.635 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 -- 172.21.15.113:0/3614797267 shutdown_connections 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.013+0000 7f1705e75700 1 -- 172.21.15.113:0/3614797267 wait complete. 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f1705e75700 1 Processor -- start 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f1705e75700 1 -- start start 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f1705e75700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f1705e75700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f170010ed60 con 0x7f170007edf0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54476/0 (socket says 172.21.15.113:54476) 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.014+0000 7f16ff7fe700 1 -- 172.21.15.113:0/621915583 learned_addr learned my addr 172.21.15.113:0/621915583 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16ff7fe700 1 -- 172.21.15.113:0/621915583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170019ee10 con 0x7f170007edf0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16ff7fe700 1 --2- 172.21.15.113:0/621915583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170019e8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16f00116a0 tx=0x7f16f0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0003ab0 con 0x7f170007edf0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f16f0003c50 con 0x7f170007edf0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170019f0a0 con 0x7f170007edf0 2024-04-22T07:05:01.636 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0019870 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170019f500 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f16f002d430 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.015+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f16f0024820 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.016+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17000681b0 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.018+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f16f0011e20 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.085+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f17001a22b0 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.085+0000 7f16fcff9700 1 -- 172.21.15.113:0/621915583 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f16f001ac00 con 0x7f170007edf0 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 msgr2=0x7f170019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 --2- 172.21.15.113:0/621915583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170019e8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16f00116a0 tx=0x7f16f0011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 shutdown_connections 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 --2- 172.21.15.113:0/621915583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f170007edf0 0x7f170019e8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 >> 172.21.15.113:0/621915583 conn(0x7f1700103df0 msgr2=0x7f1700104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:01.637 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 shutdown_connections 2024-04-22T07:05:01.638 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:01.086+0000 7f1705e75700 1 -- 172.21.15.113:0/621915583 wait complete. 2024-04-22T07:05:01.638 INFO:teuthology.orchestra.run.smithi113.stdout:Restarting the monitor... 2024-04-22T07:05:01.646 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:01 smithi113 ceph-mon[27555]: from='client.? 172.21.15.113:0/621915583' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:05:02.080 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:01 smithi113 systemd[1]: Stopping Ceph mon.a for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:05:02.080 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a[27551]: 2024-04-22T07:05:01.887+0000 7fa1e05b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:05:02.080 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a[27551]: 2024-04-22T07:05:01.887+0000 7fa1e05b6700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T07:05:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 podman[27801]: 2024-04-22 07:05:02.082249365 +0000 UTC m=+0.291890751 container died 69cb5028a476734185001e69b119ed0e47fab0011b0b90dd6b3507016a38fe80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-22T07:05:02.678 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 podman[27801]: 2024-04-22 07:05:02.429544738 +0000 UTC m=+0.639186125 container cleanup 69cb5028a476734185001e69b119ed0e47fab0011b0b90dd6b3507016a38fe80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T07:05:02.678 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 bash[27801]: ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a 2024-04-22T07:05:03.011 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 podman[27815]: 2024-04-22 07:05:02.680525946 +0000 UTC m=+0.602164286 container remove 69cb5028a476734185001e69b119ed0e47fab0011b0b90dd6b3507016a38fe80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T07:05:03.012 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-22T07:05:03.012 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 systemd[1]: Stopped Ceph mon.a for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:05:03.012 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:02 smithi113 systemd[1]: Starting Ceph mon.a for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:05:03.281 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 podman[27914]: 2024-04-22 07:05:03.014540502 +0000 UTC m=+0.114677314 container create b65780c4ca1b883f1ebc5aa14aededfbe33e3627d91b330268067768f794ed97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:05:03.281 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 podman[27914]: 2024-04-22 07:05:02.917906903 +0000 UTC m=+0.018043733 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 podman[27914]: 2024-04-22 07:05:03.208817161 +0000 UTC m=+0.308953974 container init b65780c4ca1b883f1ebc5aa14aededfbe33e3627d91b330268067768f794ed97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416) 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 podman[27914]: 2024-04-22 07:05:03.211676332 +0000 UTC m=+0.311813145 container start b65780c4ca1b883f1ebc5aa14aededfbe33e3627d91b330268067768f794ed97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev), process ceph-mon, pid 2 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: pidfile_write: ignore empty --pid-file 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: load: jerasure load: lrc 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: RocksDB version: 7.9.2 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Git sha 0 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Compile date 2024-04-21 19:54:05 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: DB SUMMARY 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: DB Session ID: CG53T6QESC24W5SDIUR5 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: CURRENT file: CURRENT 2024-04-22T07:05:03.282 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T07:05:03.283 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-22T07:05:03.283 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-22T07:05:03.283 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106566 ; 2024-04-22T07:05:03.283 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.error_if_exists: 0 2024-04-22T07:05:03.283 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.create_if_missing: 0 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.paranoid_checks: 1 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.env: 0x55e99b006cc0 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.info_log: 0x55e99c5f3be0 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.statistics: (nil) 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.use_fsync: 0 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_log_file_size: 0 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T07:05:03.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.allow_fallocate: 1 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.use_direct_reads: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.db_log_dir: 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.wal_dir: 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T07:05:03.285 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.write_buffer_manager: 0x55e99c5870e0 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T07:05:03.286 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.unordered_write: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.row_cache: None 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.wal_filter: None 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.two_write_queues: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.wal_compression: 0 2024-04-22T07:05:03.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.atomic_flush: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.log_readahead_size: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_background_jobs: 2 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_background_compactions: -1 2024-04-22T07:05:03.288 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_subcompactions: 1 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_open_files: -1 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T07:05:03.289 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_background_flushes: -1 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Compression algorithms supported: 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kZSTD supported: 0 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kXpressCompression supported: 0 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kZlibCompression supported: 1 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kSnappyCompression supported: 1 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kLZ4Compression supported: 1 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: kBZip2Compression supported: 0 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-22T07:05:03.290 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.merge_operator: 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_filter: None 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e99c5f3dc0) 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: cache_index_and_filter_blocks: 1 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: pin_top_level_index_and_filter: 1 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: index_type: 0 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: data_block_index_type: 0 2024-04-22T07:05:03.291 INFO:journalctl@ceph.mon.a.smithi113.stdout: index_shortening: 1 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: checksum: 4 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: no_block_cache: 0 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_cache: 0x55e99c5b91f0 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_cache_name: BinnedLRUCache 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_cache_options: 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: capacity : 536870912 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: num_shard_bits : 4 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: strict_capacity_limit : 0 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: high_pri_pool_ratio: 0.000 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_cache_compressed: (nil) 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: persistent_cache: (nil) 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_size: 4096 2024-04-22T07:05:03.292 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_size_deviation: 10 2024-04-22T07:05:03.293 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_restart_interval: 16 2024-04-22T07:05:03.293 INFO:journalctl@ceph.mon.a.smithi113.stdout: index_block_restart_interval: 1 2024-04-22T07:05:03.293 INFO:journalctl@ceph.mon.a.smithi113.stdout: metadata_block_size: 4096 2024-04-22T07:05:03.293 INFO:journalctl@ceph.mon.a.smithi113.stdout: partition_filters: 0 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: use_delta_encoding: 1 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: filter_policy: bloomfilter 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: whole_key_filtering: 1 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: verify_compression: 0 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: read_amp_bytes_per_bit: 0 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: format_version: 5 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: enable_index_compression: 1 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: block_align: 0 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: max_auto_readahead_size: 262144 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: prepopulate_block_cache: 0 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: initial_auto_readahead_size: 8192 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T07:05:03.294 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression: NoCompression 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.num_levels: 7 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T07:05:03.295 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T07:05:03.296 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T07:05:03.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T07:05:03.298 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.inplace_update_support: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.bloom_locality: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.max_successive_merges: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T07:05:03.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.ttl: 2592000 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.enable_blob_files: false 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.min_blob_size: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T07:05:03.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-22T07:05:03.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-22T07:05:03.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cb285864-6bc6-411c-9b9e-dde78ccd956f 2024-04-22T07:05:03.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713769503239851, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-22T07:05:03.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-22T07:05:03.301 INFO:teuthology.orchestra.run.smithi113.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-22T07:05:03.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713769503240818, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99494, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14532, "raw_average_key_size": 49, "raw_value_size": 91569, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713769503, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cb285864-6bc6-411c-9b9e-dde78ccd956f", "db_session_id": "CG53T6QESC24W5SDIUR5", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713769503240908, "job": 1, "event": "recovery_finished"} 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e99c6ce000 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: DB pointer 0x55e99c6b8000 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout: ** DB Stats ** 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T07:05:03.302 INFO:journalctl@ceph.mon.a.smithi113.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: ** Compaction Stats [default] ** 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: L0 2/0 101.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: Sum 2/0 101.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 163.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: ** Compaction Stats [default] ** 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T07:05:03.303 INFO:journalctl@ceph.mon.a.smithi113.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 163.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Cumulative compaction: 0.00 GB write, 33.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Interval compaction: 0.00 GB write, 33.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Block cache BinnedLRUCache@0x55e99c5b91f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-22T07:05:03.304 INFO:journalctl@ceph.mon.a.smithi113.stdout: 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: starting mon.a rank 0 at public addrs [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] at bind addrs [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???) e1 preinit fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).mds e1 new map 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).mds e1 print_map 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: e1 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: legacy client fscid: -1 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout: No filesystems configured 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T07:05:03.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T07:05:03.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-22T07:05:03.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 bash[27914]: b65780c4ca1b883f1ebc5aa14aededfbe33e3627d91b330268067768f794ed97 2024-04-22T07:05:03.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 systemd[1]: Started Ceph mon.a for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:05:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: monmap epoch 1 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: last_changed 2024-04-22T07:04:55.557885+0000 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: min_mon_release 19 (squid) 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: election_strategy: 1 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: fsmap 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T07:05:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:03 smithi113 ceph-mon[27933]: mgrmap e1: no daemons active 2024-04-22T07:05:04.369 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c41531700 1 Processor -- start 2024-04-22T07:05:04.369 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c41531700 1 -- start start 2024-04-22T07:05:04.369 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c41531700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:04.369 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c41531700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c107400 con 0x7f5c3c106a10 2024-04-22T07:05:04.369 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:04.369 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54478/0 (socket says 172.21.15.113:54478) 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.696+0000 7f5c3affd700 1 -- 172.21.15.113:0/212044899 learned_addr learned my addr 172.21.15.113:0/212044899 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c3affd700 1 -- 172.21.15.113:0/212044899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c1099a0 con 0x7f5c3c106a10 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c3affd700 1 --2- 172.21.15.113:0/212044899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c106e30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0098a0 tx=0x7f5c2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92327a1a97153caa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c39ffb700 1 -- 172.21.15.113:0/212044899 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c004160 con 0x7f5c3c106a10 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c39ffb700 1 -- 172.21.15.113:0/212044899 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f5c2c004300 con 0x7f5c3c106a10 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c39ffb700 1 -- 172.21.15.113:0/212044899 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c011b20 con 0x7f5c3c106a10 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c41531700 1 -- 172.21.15.113:0/212044899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 msgr2=0x7f5c3c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.697+0000 7f5c41531700 1 --2- 172.21.15.113:0/212044899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c106e30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0098a0 tx=0x7f5c2c009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.698+0000 7f5c41531700 1 -- 172.21.15.113:0/212044899 shutdown_connections 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.698+0000 7f5c41531700 1 --2- 172.21.15.113:0/212044899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c106e30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.698+0000 7f5c41531700 1 -- 172.21.15.113:0/212044899 >> 172.21.15.113:0/212044899 conn(0x7f5c3c102360 msgr2=0x7f5c3c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.698+0000 7f5c41531700 1 -- 172.21.15.113:0/212044899 shutdown_connections 2024-04-22T07:05:04.370 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.698+0000 7f5c41531700 1 -- 172.21.15.113:0/212044899 wait complete. 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.698+0000 7f5c41531700 1 Processor -- start 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c41531700 1 -- start start 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c41531700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c41531700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c10cd50 con 0x7f5c3c106a10 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54482/0 (socket says 172.21.15.113:54482) 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c3affd700 1 -- 172.21.15.113:0/1396366390 learned_addr learned my addr 172.21.15.113:0/1396366390 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c3affd700 1 -- 172.21.15.113:0/1396366390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c19cc00 con 0x7f5c3c106a10 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c3affd700 1 --2- 172.21.15.113:0/1396366390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c19c6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c006af0 tx=0x7f5c2c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c003b90 con 0x7f5c3c106a10 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f5c2c003d30 con 0x7f5c3c106a10 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c19ce90 con 0x7f5c3c106a10 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.699+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c0197c0 con 0x7f5c3c106a10 2024-04-22T07:05:04.371 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.700+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c19d2f0 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.700+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c2c022020 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.700+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c2c02c3a0 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.700+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c3c0681b0 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.703+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5c2c0042e0 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.772+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f5c3c19fd10 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.774+0000 7f5c23fff700 1 -- 172.21.15.113:0/1396366390 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5c2c01ac50 con 0x7f5c3c106a10 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 msgr2=0x7f5c3c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 --2- 172.21.15.113:0/1396366390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c19c6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c006af0 tx=0x7f5c2c011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 shutdown_connections 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 --2- 172.21.15.113:0/1396366390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3c106a10 0x7f5c3c19c6c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 >> 172.21.15.113:0/1396366390 conn(0x7f5c3c102360 msgr2=0x7f5c3c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 shutdown_connections 2024-04-22T07:05:04.372 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:03.775+0000 7f5c41531700 1 -- 172.21.15.113:0/1396366390 wait complete. 2024-04-22T07:05:04.373 INFO:teuthology.orchestra.run.smithi113.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-22T07:05:04.387 INFO:teuthology.orchestra.run.smithi113.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:05:04.387 INFO:teuthology.orchestra.run.smithi113.stdout:Creating mgr... 2024-04-22T07:05:04.388 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-22T07:05:04.388 INFO:teuthology.orchestra.run.smithi113.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-22T07:05:04.636 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x 2024-04-22T07:05:04.637 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Failed to reset failed state of unit ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-22T07:05:04.784 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-81035712-0076-11ef-bc93-c7b262605968.target.wants/ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-81035712-0076-11ef-bc93-c7b262605968@.service. 2024-04-22T07:05:05.015 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1396366390' entity='client.admin' 2024-04-22T07:05:05.016 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:04 smithi113 systemd[1]: Starting Ceph mgr.x for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:05:05.292 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 podman[28155]: 2024-04-22 07:05:05.017352986 +0000 UTC m=+0.112108988 container create 8768c6742f4334d7a36c4b3cf5f7a5317dde72ccd1ee8cde30b8eea46ee717c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:05:05.292 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 podman[28155]: 2024-04-22 07:05:04.922887293 +0000 UTC m=+0.017643258 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:05:05.292 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 podman[28155]: 2024-04-22 07:05:05.227792361 +0000 UTC m=+0.322548304 container init 8768c6742f4334d7a36c4b3cf5f7a5317dde72ccd1ee8cde30b8eea46ee717c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:05:05.292 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 podman[28155]: 2024-04-22 07:05:05.230435885 +0000 UTC m=+0.325191829 container start 8768c6742f4334d7a36c4b3cf5f7a5317dde72ccd1ee8cde30b8eea46ee717c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-22T07:05:05.292 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 bash[28155]: 8768c6742f4334d7a36c4b3cf5f7a5317dde72ccd1ee8cde30b8eea46ee717c4 2024-04-22T07:05:05.305 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T07:05:05.305 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout disabled 2024-04-22T07:05:05.313 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T07:05:05.314 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout inactive 2024-04-22T07:05:05.314 INFO:teuthology.orchestra.run.smithi113.stdout:firewalld.service is not enabled 2024-04-22T07:05:05.314 INFO:teuthology.orchestra.run.smithi113.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T07:05:05.320 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T07:05:05.320 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout disabled 2024-04-22T07:05:05.327 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T07:05:05.328 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout inactive 2024-04-22T07:05:05.328 INFO:teuthology.orchestra.run.smithi113.stdout:firewalld.service is not enabled 2024-04-22T07:05:05.328 INFO:teuthology.orchestra.run.smithi113.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-22T07:05:05.328 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for mgr to start... 2024-04-22T07:05:05.328 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for mgr... 2024-04-22T07:05:05.566 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 systemd[1]: Started Ceph mgr.x for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:05:05.566 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:05.388+0000 7f3ae1a5a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:05:05.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:05.578+0000 7f3ae1a5a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:05:05.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:05.682+0000 7f3ae1a5a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:05:06.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:05 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2811773616' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T07:05:06.334 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:06.022+0000 7f3ae1a5a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:05:06.334 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:06.106+0000 7f3ae1a5a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 0 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:06.339 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T07:05:06.339 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T07:05:06.339 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T07:05:06.346 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T07:05:06.347 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:06.348 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T07:04:58.385182+0000", 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.763+0000 7f9683a4d700 1 Processor -- start 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f9683a4d700 1 -- start start 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f9683a4d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f9683a4d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f967c07d840 con 0x7f967c07ee00 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f96817e9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f96817e9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36576/0 (socket says 172.21.15.113:36576) 2024-04-22T07:05:06.349 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f96817e9700 1 -- 172.21.15.113:0/1301928340 learned_addr learned my addr 172.21.15.113:0/1301928340 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f96817e9700 1 -- 172.21.15.113:0/1301928340 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c07d9c0 con 0x7f967c07ee00 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.764+0000 7f96817e9700 1 --2- 172.21.15.113:0/1301928340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c07d300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f96780098a0 tx=0x7f9678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d715a0b83d824bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9673fff700 1 -- 172.21.15.113:0/1301928340 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9678004160 con 0x7f967c07ee00 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9673fff700 1 -- 172.21.15.113:0/1301928340 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f9678004300 con 0x7f967c07ee00 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9673fff700 1 -- 172.21.15.113:0/1301928340 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9678011b20 con 0x7f967c07ee00 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9683a4d700 1 -- 172.21.15.113:0/1301928340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 msgr2=0x7f967c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9683a4d700 1 --2- 172.21.15.113:0/1301928340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c07d300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f96780098a0 tx=0x7f9678009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9683a4d700 1 -- 172.21.15.113:0/1301928340 shutdown_connections 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9683a4d700 1 --2- 172.21.15.113:0/1301928340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c07d300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.765+0000 7f9683a4d700 1 -- 172.21.15.113:0/1301928340 >> 172.21.15.113:0/1301928340 conn(0x7f967c103d30 msgr2=0x7f967c1061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f9683a4d700 1 -- 172.21.15.113:0/1301928340 shutdown_connections 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f9683a4d700 1 -- 172.21.15.113:0/1301928340 wait complete. 2024-04-22T07:05:06.350 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f9683a4d700 1 Processor -- start 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f9683a4d700 1 -- start start 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f9683a4d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f9683a4d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f967c10ed60 con 0x7f967c07ee00 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f96817e9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.766+0000 7f96817e9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36582/0 (socket says 172.21.15.113:36582) 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96817e9700 1 -- 172.21.15.113:0/2811773616 learned_addr learned my addr 172.21.15.113:0/2811773616 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96817e9700 1 -- 172.21.15.113:0/2811773616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c19ee10 con 0x7f967c07ee00 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96817e9700 1 --2- 172.21.15.113:0/2811773616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c19e8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9678006af0 tx=0x7f9678011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9678003b90 con 0x7f967c07ee00 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967c19f0a0 con 0x7f967c07ee00 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f9678003d30 con 0x7f967c07ee00 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96780197c0 con 0x7f967c07ee00 2024-04-22T07:05:06.351 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f967c19f500 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9678022020 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.767+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f967802c600 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.768+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f967c0681b0 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.771+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f96780042e0 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.865+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f967c1a1e00 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.865+0000 7f96727fc700 1 -- 172.21.15.113:0/2811773616 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f967801ac50 con 0x7f967c07ee00 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 msgr2=0x7f967c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 --2- 172.21.15.113:0/2811773616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c19e8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9678006af0 tx=0x7f9678011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 shutdown_connections 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 --2- 172.21.15.113:0/2811773616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f967c07ee00 0x7f967c19e8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 >> 172.21.15.113:0/2811773616 conn(0x7f967c103d30 msgr2=0x7f967c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 shutdown_connections 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:05.866+0000 7f9683a4d700 1 -- 172.21.15.113:0/2811773616 wait complete. 2024-04-22T07:05:06.352 INFO:teuthology.orchestra.run.smithi113.stdout:mgr not available, waiting (1/15)... 2024-04-22T07:05:07.477 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:07.311+0000 7f3ae1a5a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:05:07.477 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:07.479+0000 7f3ae1a5a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:05:07.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:07.647+0000 7f3ae1a5a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:05:08.239 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:07.933+0000 7f3ae1a5a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:05:08.239 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:08.025+0000 7f3ae1a5a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:05:08.239 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:08.241+0000 7f3ae1a5a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:05:08.493 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:08.437+0000 7f3ae1a5a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:05:09.049 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:08 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/108850166' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T07:05:09.334 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:09.051+0000 7f3ae1a5a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:05:09.777 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:09.481+0000 7f3ae1a5a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:05:09.777 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:09.575+0000 7f3ae1a5a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:05:09.777 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:09.779+0000 7f3ae1a5a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T07:05:10.314 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T07:05:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T07:05:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T07:05:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 0 2024-04-22T07:05:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T07:05:10.316 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:10.317 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T07:05:10.317 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T07:05:10.317 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T07:05:10.330 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T07:05:10.331 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T07:04:58.385182+0000", 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.786+0000 7f591a52f700 1 Processor -- start 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.786+0000 7f591a52f700 1 -- start start 2024-04-22T07:05:10.332 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.786+0000 7f591a52f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f59141079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.786+0000 7f591a52f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5914107fc0 con 0x7f59141075d0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5913fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f59141079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5913fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f59141079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36598/0 (socket says 172.21.15.113:36598) 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5913fff700 1 -- 172.21.15.113:0/115485789 learned_addr learned my addr 172.21.15.113:0/115485789 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5913fff700 1 -- 172.21.15.113:0/115485789 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59141087d0 con 0x7f59141075d0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5913fff700 1 --2- 172.21.15.113:0/115485789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f59141079f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59080098a0 tx=0x7f5908009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544f625ffe4aecd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5912ffd700 1 -- 172.21.15.113:0/115485789 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5908004160 con 0x7f59141075d0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5912ffd700 1 -- 172.21.15.113:0/115485789 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f5908004300 con 0x7f59141075d0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.787+0000 7f5912ffd700 1 -- 172.21.15.113:0/115485789 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5908011b20 con 0x7f59141075d0 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 -- 172.21.15.113:0/115485789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 msgr2=0x7f59141079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 --2- 172.21.15.113:0/115485789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f59141079f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59080098a0 tx=0x7f5908009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 -- 172.21.15.113:0/115485789 shutdown_connections 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 --2- 172.21.15.113:0/115485789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f59141079f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:10.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 -- 172.21.15.113:0/115485789 >> 172.21.15.113:0/115485789 conn(0x7f5914102c10 msgr2=0x7f5914105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 -- 172.21.15.113:0/115485789 shutdown_connections 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.788+0000 7f591a52f700 1 -- 172.21.15.113:0/115485789 wait complete. 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f591a52f700 1 Processor -- start 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f591a52f700 1 -- start start 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f591a52f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f5914063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f591a52f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f591410bb80 con 0x7f59141075d0 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f5913fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f5914063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f5913fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f5914063900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36604/0 (socket says 172.21.15.113:36604) 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f5913fff700 1 -- 172.21.15.113:0/108850166 learned_addr learned my addr 172.21.15.113:0/108850166 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f5913fff700 1 -- 172.21.15.113:0/108850166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591419f410 con 0x7f59141075d0 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.789+0000 7f5913fff700 1 --2- 172.21.15.113:0/108850166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f5914063900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5908006af0 tx=0x7f5908011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:10.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5908003930 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591419f5e0 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f5908003ad0 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5908019610 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591419f980 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5908019c80 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.790+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5908024730 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.791+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59140681b0 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.793+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5908011e20 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.865+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f59141a0100 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.866+0000 7f59117fa700 1 -- 172.21.15.113:0/108850166 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5908033030 con 0x7f59141075d0 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.866+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 msgr2=0x7f5914063900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.866+0000 7f591a52f700 1 --2- 172.21.15.113:0/108850166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f5914063900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5908006af0 tx=0x7f5908011c10 comp rx=0 tx=0).stop 2024-04-22T07:05:10.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.867+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 shutdown_connections 2024-04-22T07:05:10.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.867+0000 7f591a52f700 1 --2- 172.21.15.113:0/108850166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59141075d0 0x7f5914063900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:10.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.867+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 >> 172.21.15.113:0/108850166 conn(0x7f5914102c10 msgr2=0x7f5914103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:10.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.867+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 shutdown_connections 2024-04-22T07:05:10.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:08.867+0000 7f591a52f700 1 -- 172.21.15.113:0/108850166 wait complete. 2024-04-22T07:05:10.336 INFO:teuthology.orchestra.run.smithi113.stdout:mgr not available, waiting (2/15)... 2024-04-22T07:05:10.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:10.301+0000 7f3ae1a5a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:05:10.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:10.393+0000 7f3ae1a5a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:05:10.965 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:10.631+0000 7f3ae1a5a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:05:10.965 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:10.762+0000 7f3ae1a5a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:05:10.965 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:10.854+0000 7f3ae1a5a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:05:10.966 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:10.967+0000 7f3ae1a5a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:05:11.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:11 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:11.263+0000 7f3ae1a5a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:05:12.142 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:11 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:11.872+0000 7f3ae1a5a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:05:12.142 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:12.091+0000 7f3ae1a5a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: Activating manager daemon x 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: mgrmap e2: x(active, starting, since 0.00303278s) 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:05:12.440 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T07:05:12.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: Manager daemon x is now available 2024-04-22T07:05:12.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T07:05:12.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T07:05:12.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' 2024-04-22T07:05:12.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' 2024-04-22T07:05:12.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:12 smithi113 ceph-mon[27933]: from='mgr.14100 172.21.15.113:0/1752214298' entity='mgr.x' 2024-04-22T07:05:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:14 smithi113 ceph-mon[27933]: mgrmap e3: x(active, since 1.00718s) 2024-04-22T07:05:15.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:15 smithi113 ceph-mon[27933]: mgrmap e4: x(active, since 2s) 2024-04-22T07:05:16.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:16 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3040031566' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T07:05:19.674 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 0 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T07:05:19.675 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T07:05:19.676 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T07:05:19.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T07:05:19.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T07:05:19.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T07:05:19.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T07:05:19.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ], 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T07:05:19.688 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T07:04:58.385182+0000", 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout }, 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.931+0000 7fca47c4f700 1 Processor -- start 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.931+0000 7fca47c4f700 1 -- start start 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.931+0000 7fca47c4f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca4010b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.931+0000 7fca47c4f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca40063d60 con 0x7fca40109140 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca459eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca4010b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca459eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca4010b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48172/0 (socket says 172.21.15.113:48172) 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca459eb700 1 -- 172.21.15.113:0/1431071434 learned_addr learned my addr 172.21.15.113:0/1431071434 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca459eb700 1 -- 172.21.15.113:0/1431071434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4010ba70 con 0x7fca40109140 2024-04-22T07:05:19.689 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca459eb700 1 --2- 172.21.15.113:0/1431071434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca4010b530 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fca380098a0 tx=0x7fca38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852b1dc80eb647f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca449e9700 1 -- 172.21.15.113:0/1431071434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca38004160 con 0x7fca40109140 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca449e9700 1 -- 172.21.15.113:0/1431071434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fca38004300 con 0x7fca40109140 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.932+0000 7fca449e9700 1 -- 172.21.15.113:0/1431071434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca38011b20 con 0x7fca40109140 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 -- 172.21.15.113:0/1431071434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 msgr2=0x7fca4010b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 --2- 172.21.15.113:0/1431071434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca4010b530 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fca380098a0 tx=0x7fca38009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 -- 172.21.15.113:0/1431071434 shutdown_connections 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 --2- 172.21.15.113:0/1431071434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca4010b530 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 -- 172.21.15.113:0/1431071434 >> 172.21.15.113:0/1431071434 conn(0x7fca40102c10 msgr2=0x7fca40105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 -- 172.21.15.113:0/1431071434 shutdown_connections 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 -- 172.21.15.113:0/1431071434 wait complete. 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 Processor -- start 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.933+0000 7fca47c4f700 1 -- start start 2024-04-22T07:05:19.690 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca47c4f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca401089c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca47c4f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca4010caa0 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca459eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca401089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca459eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca401089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48178/0 (socket says 172.21.15.113:48178) 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca459eb700 1 -- 172.21.15.113:0/3040031566 learned_addr learned my addr 172.21.15.113:0/3040031566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca459eb700 1 -- 172.21.15.113:0/3040031566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca40108f00 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca459eb700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca401089c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fca38006af0 tx=0x7fca38011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca38003b90 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fca38003d30 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca401070f0 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca380197c0 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.934+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca401075b0 con 0x7fca40109140 2024-04-22T07:05:19.691 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.935+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fca38022020 con 0x7fca40109140 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.935+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca400681b0 con 0x7fca40109140 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.935+0000 7fca3effd700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7fca28039560 0x7fca2803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.935+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca3804d620 con 0x7fca40109140 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.936+0000 7fca451ea700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7fca28039560 0x7fca2803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.936+0000 7fca451ea700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7fca28039560 0x7fca2803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fca34006fd0 tx=0x7fca34006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:15.941+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca380042a0 con 0x7fca40109140 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.223+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fca40107830 con 0x7fca40109140 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.223+0000 7fca3effd700 1 -- 172.21.15.113:0/3040031566 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fca3804d380 con 0x7fca40109140 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7fca28039560 msgr2=0x7fca2803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7fca28039560 0x7fca2803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fca34006fd0 tx=0x7fca34006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 msgr2=0x7fca401089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca401089c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fca38006af0 tx=0x7fca38011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 shutdown_connections 2024-04-22T07:05:19.692 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7fca28039560 0x7fca2803ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:19.693 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 --2- 172.21.15.113:0/3040031566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca40109140 0x7fca401089c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:19.693 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 >> 172.21.15.113:0/3040031566 conn(0x7fca40102c10 msgr2=0x7fca40103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:19.693 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 shutdown_connections 2024-04-22T07:05:19.693 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:16.226+0000 7fca47c4f700 1 -- 172.21.15.113:0/3040031566 wait complete. 2024-04-22T07:05:19.693 INFO:teuthology.orchestra.run.smithi113.stdout:mgr is available 2024-04-22T07:05:20.810 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:20 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/603690701' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T07:05:20.892 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout fsid = 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 2024-04-22T07:05:20.893 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.133+0000 7ff898c2c700 1 Processor -- start 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.133+0000 7ff898c2c700 1 -- start start 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.133+0000 7ff898c2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89410b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.133+0000 7ff898c2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff89407ce90 con 0x7ff894109180 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.134+0000 7ff89259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89410b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.134+0000 7ff89259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89410b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48182/0 (socket says 172.21.15.113:48182) 2024-04-22T07:05:20.894 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.134+0000 7ff89259c700 1 -- 172.21.15.113:0/3667676781 learned_addr learned my addr 172.21.15.113:0/3667676781 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.134+0000 7ff89259c700 1 -- 172.21.15.113:0/3667676781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89410bab0 con 0x7ff894109180 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.134+0000 7ff89259c700 1 --2- 172.21.15.113:0/3667676781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89410b570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8880098a0 tx=0x7ff888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396f65c59b487b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.134+0000 7ff89159a700 1 -- 172.21.15.113:0/3667676781 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff888004160 con 0x7ff894109180 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff89159a700 1 -- 172.21.15.113:0/3667676781 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7ff888004300 con 0x7ff894109180 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff89159a700 1 -- 172.21.15.113:0/3667676781 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff888011b20 con 0x7ff894109180 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 -- 172.21.15.113:0/3667676781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 msgr2=0x7ff89410b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 --2- 172.21.15.113:0/3667676781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89410b570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8880098a0 tx=0x7ff888009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 -- 172.21.15.113:0/3667676781 shutdown_connections 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 --2- 172.21.15.113:0/3667676781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89410b570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 -- 172.21.15.113:0/3667676781 >> 172.21.15.113:0/3667676781 conn(0x7ff894102c10 msgr2=0x7ff894105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 -- 172.21.15.113:0/3667676781 shutdown_connections 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.135+0000 7ff898c2c700 1 -- 172.21.15.113:0/3667676781 wait complete. 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.136+0000 7ff898c2c700 1 Processor -- start 2024-04-22T07:05:20.895 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff898c2c700 1 -- start start 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff898c2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff898c2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410cae0 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff89259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff89259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48188/0 (socket says 172.21.15.113:48188) 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff89259c700 1 -- 172.21.15.113:0/603690701 learned_addr learned my addr 172.21.15.113:0/603690701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff89259c700 1 -- 172.21.15.113:0/603690701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89419cb10 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff89259c700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89419c5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff888006af0 tx=0x7ff888011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.137+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8880039a0 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.138+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7ff888003b40 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.138+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89419cda0 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.138+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff888019670 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.138+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89419d200 con 0x7ff894109180 2024-04-22T07:05:20.896 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.138+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff888019970 con 0x7ff894109180 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.139+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8940681b0 con 0x7ff894109180 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.139+0000 7ff8837fe700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7ff87c039510 0x7ff87c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.139+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff88804dbf0 con 0x7ff894109180 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.139+0000 7ff891d9b700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7ff87c039510 0x7ff87c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.139+0000 7ff891d9b700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7ff87c039510 0x7ff87c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff884006fd0 tx=0x7ff884006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.144+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff888011e20 con 0x7ff894109180 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.346+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff89419fc20 con 0x7ff894109180 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.347+0000 7ff8837fe700 1 -- 172.21.15.113:0/603690701 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7ff89419fc20 con 0x7ff894109180 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.349+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7ff87c039510 msgr2=0x7ff87c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.349+0000 7ff898c2c700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7ff87c039510 0x7ff87c03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff884006fd0 tx=0x7ff884006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.349+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 msgr2=0x7ff89419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.349+0000 7ff898c2c700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89419c5d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff888006af0 tx=0x7ff888011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.350+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 shutdown_connections 2024-04-22T07:05:20.897 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.350+0000 7ff898c2c700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7ff87c039510 0x7ff87c03b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:20.898 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.350+0000 7ff898c2c700 1 --2- 172.21.15.113:0/603690701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff894109180 0x7ff89419c5d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:20.898 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.350+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 >> 172.21.15.113:0/603690701 conn(0x7ff894102c10 msgr2=0x7ff894103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:20.898 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.350+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 shutdown_connections 2024-04-22T07:05:20.898 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:20.350+0000 7ff898c2c700 1 -- 172.21.15.113:0/603690701 wait complete. 2024-04-22T07:05:20.898 INFO:teuthology.orchestra.run.smithi113.stdout:Enabling cephadm module... 2024-04-22T07:05:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:21 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3075629725' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-22T07:05:21.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:21 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: ignoring --setuser ceph since I am not root 2024-04-22T07:05:21.835 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:21 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: ignoring --setgroup ceph since I am not root 2024-04-22T07:05:21.835 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:21 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:21.762+0000 7fd6032d5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:05:22.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1532f4f700 1 Processor -- start 2024-04-22T07:05:22.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1532f4f700 1 -- start start 2024-04-22T07:05:22.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1532f4f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c07d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1532f4f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f152c07d8b0 con 0x7f152c07ee30 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1530ceb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c07d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1530ceb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c07d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48196/0 (socket says 172.21.15.113:48196) 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.279+0000 7f1530ceb700 1 -- 172.21.15.113:0/29703728 learned_addr learned my addr 172.21.15.113:0/29703728 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.280+0000 7f1530ceb700 1 -- 172.21.15.113:0/29703728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c07da30 con 0x7f152c07ee30 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.280+0000 7f1530ceb700 1 --2- 172.21.15.113:0/29703728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c07d2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15200098a0 tx=0x7f1520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a4002bf97fdcb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.280+0000 7f152b7fe700 1 -- 172.21.15.113:0/29703728 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1520004160 con 0x7f152c07ee30 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.280+0000 7f152b7fe700 1 -- 172.21.15.113:0/29703728 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f1520004300 con 0x7f152c07ee30 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.280+0000 7f152b7fe700 1 -- 172.21.15.113:0/29703728 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1520011b20 con 0x7f152c07ee30 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- 172.21.15.113:0/29703728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 msgr2=0x7f152c07d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 --2- 172.21.15.113:0/29703728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c07d2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15200098a0 tx=0x7f1520009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- 172.21.15.113:0/29703728 shutdown_connections 2024-04-22T07:05:22.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 --2- 172.21.15.113:0/29703728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c07d2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- 172.21.15.113:0/29703728 >> 172.21.15.113:0/29703728 conn(0x7f152c103df0 msgr2=0x7f152c106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- 172.21.15.113:0/29703728 shutdown_connections 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- 172.21.15.113:0/29703728 wait complete. 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 Processor -- start 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- start start 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1532f4f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f152c10ed80 con 0x7f152c07ee30 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1530ceb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1530ceb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48212/0 (socket says 172.21.15.113:48212) 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.281+0000 7f1530ceb700 1 -- 172.21.15.113:0/3075629725 learned_addr learned my addr 172.21.15.113:0/3075629725 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1530ceb700 1 -- 172.21.15.113:0/3075629725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c19edf0 con 0x7f152c07ee30 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1530ceb700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c19e8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15200116a0 tx=0x7f1520011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:22.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1520003ab0 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c19f080 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f1520003c50 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1520019870 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.282+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c19f4e0 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.283+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f152001ac00 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.283+0000 7f1529ffb700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f151c039510 0x7f151c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.283+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152c0681b0 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.283+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f152004da90 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.283+0000 7f152bfff700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f151c039510 0x7f151c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.284+0000 7f152bfff700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f151c039510 0x7f151c03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f1518006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.288+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1520011e20 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.543+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f152c1a1f70 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.594+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f152001aa00 con 0x7f152c07ee30 2024-04-22T07:05:22.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.595+0000 7f1529ffb700 1 -- 172.21.15.113:0/3075629725 <== mon.0 v2:172.21.15.113:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f152004dd10 con 0x7f152c07ee30 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f151c039510 msgr2=0x7f151c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f151c039510 0x7f151c03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f1518006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 msgr2=0x7f152c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c19e8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15200116a0 tx=0x7f1520011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 shutdown_connections 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f151c039510 0x7f151c03b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 --2- 172.21.15.113:0/3075629725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152c07ee30 0x7f152c19e8b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 >> 172.21.15.113:0/3075629725 conn(0x7f152c103df0 msgr2=0x7f152c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 shutdown_connections 2024-04-22T07:05:22.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:21.597+0000 7f1532f4f700 1 -- 172.21.15.113:0/3075629725 wait complete. 2024-04-22T07:05:22.201 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:21 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:21.948+0000 7fd6032d5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:05:22.202 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:22 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:22.051+0000 7fd6032d5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:05:22.452 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:22 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:22.368+0000 7fd6032d5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:05:22.453 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:22 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:22.454+0000 7fd6032d5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:05:22.748 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:22 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3075629725' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-22T07:05:22.748 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:22 smithi113 ceph-mon[27933]: mgrmap e5: x(active, since 9s) 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.544+0000 7f4e41a2c700 1 Processor -- start 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.544+0000 7f4e41a2c700 1 -- start start 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e41a2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c106e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:23.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e41a2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1073f0 con 0x7f4e3c106a00 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48238/0 (socket says 172.21.15.113:48238) 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e3affd700 1 -- 172.21.15.113:0/3762860273 learned_addr learned my addr 172.21.15.113:0/3762860273 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e3affd700 1 -- 172.21.15.113:0/3762860273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c1099e0 con 0x7f4e3c106a00 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e3affd700 1 --2- 172.21.15.113:0/3762860273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c106e20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0098a0 tx=0x7f4e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f469a93eb843594 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e39ffb700 1 -- 172.21.15.113:0/3762860273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e2c004160 con 0x7f4e3c106a00 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e39ffb700 1 -- 172.21.15.113:0/3762860273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f4e2c004300 con 0x7f4e3c106a00 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.545+0000 7f4e39ffb700 1 -- 172.21.15.113:0/3762860273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e2c011b20 con 0x7f4e3c106a00 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 -- 172.21.15.113:0/3762860273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 msgr2=0x7f4e3c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 --2- 172.21.15.113:0/3762860273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c106e20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0098a0 tx=0x7f4e2c009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 -- 172.21.15.113:0/3762860273 shutdown_connections 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 --2- 172.21.15.113:0/3762860273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c106e20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 -- 172.21.15.113:0/3762860273 >> 172.21.15.113:0/3762860273 conn(0x7f4e3c102340 msgr2=0x7f4e3c1047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 -- 172.21.15.113:0/3762860273 shutdown_connections 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.546+0000 7f4e41a2c700 1 -- 172.21.15.113:0/3762860273 wait complete. 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.547+0000 7f4e41a2c700 1 Processor -- start 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.547+0000 7f4e41a2c700 1 -- start start 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.547+0000 7f4e41a2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.547+0000 7f4e41a2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c10cd90 con 0x7f4e3c106a00 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.547+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48246/0 (socket says 172.21.15.113:48246) 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e3affd700 1 -- 172.21.15.113:0/1381684848 learned_addr learned my addr 172.21.15.113:0/1381684848 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:23.441 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e3affd700 1 -- 172.21.15.113:0/1381684848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c19cc10 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e3affd700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c19c6d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c006af0 tx=0x7f4e2c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e2c003930 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c19cea0 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f4e2c003ad0 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e2c019610 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.548+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c19d300 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.549+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f4e2c0198d0 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.549+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c0681b0 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.549+0000 7f4e40a2a700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 0x7f4e2403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.549+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e2c04dc50 con 0x7f4e3c106a00 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.549+0000 7f4e3a7fc700 1 -- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 msgr2=0x7f4e2403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.549+0000 7f4e3a7fc700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 0x7f4e2403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:05:23.442 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.558+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e2c011e20 con 0x7f4e3c106a00 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.750+0000 7f4e3a7fc700 1 -- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 msgr2=0x7f4e2403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.750+0000 7f4e3a7fc700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 0x7f4e2403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.819+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4e3c19fd20 con 0x7f4e3c106a00 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.819+0000 7f4e40a2a700 1 -- 172.21.15.113:0/1381684848 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f4e3c19fd20 con 0x7f4e3c106a00 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 msgr2=0x7f4e2403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 0x7f4e2403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 msgr2=0x7f4e3c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c19c6d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c006af0 tx=0x7f4e2c011c10 comp rx=0 tx=0).stop 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 shutdown_connections 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f4e24039560 0x7f4e2403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 --2- 172.21.15.113:0/1381684848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e3c106a00 0x7f4e3c19c6d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.821+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 >> 172.21.15.113:0/1381684848 conn(0x7f4e3c102340 msgr2=0x7f4e3c102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.822+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 shutdown_connections 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:22.822+0000 7f4e41a2c700 1 -- 172.21.15.113:0/1381684848 wait complete. 2024-04-22T07:05:23.443 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for the mgr to restart... 2024-04-22T07:05:23.444 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for mgr epoch 5... 2024-04-22T07:05:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:23 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1381684848' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T07:05:23.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:23 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:23.671+0000 7fd6032d5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:05:24.301 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:23 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:23.847+0000 7fd6032d5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:05:24.301 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:24.017+0000 7fd6032d5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:05:24.301 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:24.303+0000 7fd6032d5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:05:24.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:24.398+0000 7fd6032d5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:05:25.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:24.600+0000 7fd6032d5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:05:25.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:24.793+0000 7fd6032d5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:05:25.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:25 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:25.391+0000 7fd6032d5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:05:26.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:25 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:25.813+0000 7fd6032d5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:05:26.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:25 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:25.910+0000 7fd6032d5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:05:26.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:26 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:26.111+0000 7fd6032d5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:05:26.965 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:26 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:26.631+0000 7fd6032d5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:05:26.965 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:26 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:26.723+0000 7fd6032d5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:05:27.310 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:26 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:26.967+0000 7fd6032d5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:05:27.311 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:27.099+0000 7fd6032d5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:05:27.311 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:27.196+0000 7fd6032d5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:05:27.311 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:27.312+0000 7fd6032d5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:05:28.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:27.637+0000 7fd6032d5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:05:28.508 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:28.238+0000 7fd6032d5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:05:28.508 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:28.457+0000 7fd6032d5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:05:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: Active manager daemon x restarted 2024-04-22T07:05:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: Activating manager daemon x 2024-04-22T07:05:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: osdmap e2: 0 total, 0 up, 0 in 2024-04-22T07:05:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: mgrmap e6: x(active, starting, since 0.00349139s) 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: Manager daemon x is now available 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:05:28.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:28 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T07:05:29.816 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:29 smithi113 ceph-mon[27933]: Found migration_current of "None". Setting to last migration. 2024-04-22T07:05:29.816 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:29 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T07:05:29.816 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:29 smithi113 ceph-mon[27933]: mgrmap e7: x(active, since 1.00559s) 2024-04-22T07:05:30.062 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:30.062 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-22T07:05:30.062 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T07:05:30.062 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.868+0000 7f0a9a8df700 1 Processor -- start 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.868+0000 7f0a9a8df700 1 -- start start 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.868+0000 7f0a9a8df700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a9410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.868+0000 7f0a9a8df700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94063cc0 con 0x7f0a941091f0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a9410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a9410b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48262/0 (socket says 172.21.15.113:48262) 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a93fff700 1 -- 172.21.15.113:0/865411872 learned_addr learned my addr 172.21.15.113:0/865411872 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a93fff700 1 -- 172.21.15.113:0/865411872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9410bb20 con 0x7f0a941091f0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a93fff700 1 --2- 172.21.15.113:0/865411872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a9410b5e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a840098a0 tx=0x7f0a84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d77a4fa1e02f453 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a92ffd700 1 -- 172.21.15.113:0/865411872 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a84004160 con 0x7f0a941091f0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a92ffd700 1 -- 172.21.15.113:0/865411872 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f0a84004300 con 0x7f0a941091f0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.869+0000 7f0a92ffd700 1 -- 172.21.15.113:0/865411872 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a84011b20 con 0x7f0a941091f0 2024-04-22T07:05:30.063 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 -- 172.21.15.113:0/865411872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 msgr2=0x7f0a9410b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/865411872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a9410b5e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a840098a0 tx=0x7f0a84009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 -- 172.21.15.113:0/865411872 shutdown_connections 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/865411872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a9410b5e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 -- 172.21.15.113:0/865411872 >> 172.21.15.113:0/865411872 conn(0x7f0a94102c60 msgr2=0x7f0a94105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 -- 172.21.15.113:0/865411872 shutdown_connections 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.870+0000 7f0a9a8df700 1 -- 172.21.15.113:0/865411872 wait complete. 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a9a8df700 1 Processor -- start 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a9a8df700 1 -- start start 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a9a8df700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a94108a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a9a8df700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9410cae0 con 0x7f0a941091f0 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a94108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a94108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48272/0 (socket says 172.21.15.113:48272) 2024-04-22T07:05:30.064 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a93fff700 1 -- 172.21.15.113:0/3330956876 learned_addr learned my addr 172.21.15.113:0/3330956876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.871+0000 7f0a93fff700 1 -- 172.21.15.113:0/3330956876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a94108fb0 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.872+0000 7f0a93fff700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a94108a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a840116a0 tx=0x7f0a84011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.872+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a84003ab0 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.872+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a941071a0 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.872+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f0a84003c50 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.872+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a84019870 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.872+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a94107660 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.873+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0a84019b30 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.873+0000 7f0a917fa700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.873+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 --> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a7c03c130 con 0x7f0a7c039560 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.873+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a8404dca0 con 0x7f0a941091f0 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.873+0000 7f0a937fe700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 msgr2=0x7f0a7c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:30.065 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:23.873+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:24.074+0000 7f0a937fe700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 msgr2=0x7f0a7c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:24.074+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:24.475+0000 7f0a937fe700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 msgr2=0x7f0a7c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:24.475+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:25.276+0000 7f0a937fe700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 msgr2=0x7f0a7c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:25.276+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:26.877+0000 7f0a937fe700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 msgr2=0x7f0a7c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1303444723 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:26.877+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:28.461+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f0a8402dcd0 con 0x7f0a941091f0 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:28.461+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 msgr2=0x7f0a7c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:28.461+0000 7f0a917fa700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.462+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f0a8404f620 con 0x7f0a941091f0 2024-04-22T07:05:30.066 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.462+0000 7f0a917fa700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f0a7c03ce50 0x7f0a7c03f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.462+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a8401fec0 con 0x7f0a7c03ce50 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.463+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f0a7c03ce50 0x7f0a7c03f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.464+0000 7f0a937fe700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f0a7c03ce50 0x7f0a7c03f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0a88003a10 tx=0x7f0a88009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.464+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0a8401fec0 con 0x7f0a7c03ce50 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0a940681b0 con 0x7f0a7c03ce50 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a917fa700 1 -- 172.21.15.113:0/3330956876 <== mgr.14118 v2:172.21.15.113:6800/1807162111 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0a940681b0 con 0x7f0a7c03ce50 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f0a7c03ce50 msgr2=0x7f0a7c03f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f0a7c03ce50 0x7f0a7c03f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0a88003a10 tx=0x7f0a88009290 comp rx=0 tx=0).stop 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 msgr2=0x7f0a94108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a94108a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a840116a0 tx=0x7f0a84011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 shutdown_connections 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f0a7c03ce50 0x7f0a7c03f240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a941091f0 0x7f0a94108a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 --2- 172.21.15.113:0/3330956876 >> [v2:172.21.15.113:6800/1303444723,v1:172.21.15.113:6801/1303444723] conn(0x7f0a7c039560 0x7f0a7c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:30.068 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 >> 172.21.15.113:0/3330956876 conn(0x7f0a94102c60 msgr2=0x7f0a94103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:30.068 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 shutdown_connections 2024-04-22T07:05:30.068 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:29.471+0000 7f0a9a8df700 1 -- 172.21.15.113:0/3330956876 wait complete. 2024-04-22T07:05:30.068 INFO:teuthology.orchestra.run.smithi113.stdout:mgr epoch 5 is available 2024-04-22T07:05:30.068 INFO:teuthology.orchestra.run.smithi113.stdout:Setting orchestrator backend to cephadm... 2024-04-22T07:05:30.719 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:30 smithi113 ceph-mon[27933]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T07:05:30.719 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:30 smithi113 ceph-mon[27933]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T07:05:30.719 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:30 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:30.719 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:30 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:30.719 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:30 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:05:31.213 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa9a177f700 1 Processor -- start 2024-04-22T07:05:31.213 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa9a177f700 1 -- start start 2024-04-22T07:05:31.213 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa9a177f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:31.213 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa9a177f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c07d8d0 con 0x7fa99c07ee50 2024-04-22T07:05:31.213 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa99affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa99affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40096/0 (socket says 172.21.15.113:40096) 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa99affd700 1 -- 172.21.15.113:0/3298896623 learned_addr learned my addr 172.21.15.113:0/3298896623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa99affd700 1 -- 172.21.15.113:0/3298896623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c07da50 con 0x7fa99c07ee50 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.488+0000 7fa99affd700 1 --2- 172.21.15.113:0/3298896623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c07d300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098a0 tx=0x7fa98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7839f9de6f888 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa999ffb700 1 -- 172.21.15.113:0/3298896623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa98c004160 con 0x7fa99c07ee50 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa999ffb700 1 -- 172.21.15.113:0/3298896623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa98c004300 con 0x7fa99c07ee50 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa999ffb700 1 -- 172.21.15.113:0/3298896623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa98c011b20 con 0x7fa99c07ee50 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 -- 172.21.15.113:0/3298896623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 msgr2=0x7fa99c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 --2- 172.21.15.113:0/3298896623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c07d300 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098a0 tx=0x7fa98c009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 -- 172.21.15.113:0/3298896623 shutdown_connections 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 --2- 172.21.15.113:0/3298896623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c07d300 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 -- 172.21.15.113:0/3298896623 >> 172.21.15.113:0/3298896623 conn(0x7fa99c103df0 msgr2=0x7fa99c106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:31.214 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 -- 172.21.15.113:0/3298896623 shutdown_connections 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.489+0000 7fa9a177f700 1 -- 172.21.15.113:0/3298896623 wait complete. 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa9a177f700 1 Processor -- start 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa9a177f700 1 -- start start 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa9a177f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa9a177f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c10ed60 con 0x7fa99c07ee50 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa99affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa99affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40108/0 (socket says 172.21.15.113:40108) 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.490+0000 7fa99affd700 1 -- 172.21.15.113:0/1428462076 learned_addr learned my addr 172.21.15.113:0/1428462076 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa99affd700 1 -- 172.21.15.113:0/1428462076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c19edf0 con 0x7fa99c07ee50 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa99affd700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c19e8b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0116a0 tx=0x7fa98c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa98c003ab0 con 0x7fa99c07ee50 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c19f080 con 0x7fa99c07ee50 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa98c003c50 con 0x7fa99c07ee50 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa98c019870 con 0x7fa99c07ee50 2024-04-22T07:05:31.215 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.491+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c19f4e0 con 0x7fa99c07ee50 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.492+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fa98c019b30 con 0x7fa99c07ee50 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.492+0000 7fa983fff700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fa984039450 0x7fa98403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.492+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa98c04d960 con 0x7fa99c07ee50 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.492+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c0681b0 con 0x7fa99c07ee50 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.492+0000 7fa99a7fc700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fa984039450 0x7fa98403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.493+0000 7fa99a7fc700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fa984039450 0x7fa98403b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa990006fd0 tx=0x7fa990006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.498+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa98c011e20 con 0x7fa99c07ee50 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.610+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa98c02dcd0 con 0x7fa99c07ee50 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.714+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa99c104f70 con 0x7fa984039450 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.718+0000 7fa983fff700 1 -- 172.21.15.113:0/1428462076 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa99c104f70 con 0x7fa984039450 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fa984039450 msgr2=0x7fa98403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fa984039450 0x7fa98403b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa990006fd0 tx=0x7fa990006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:31.216 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 msgr2=0x7fa99c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c19e8b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0116a0 tx=0x7fa98c011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 shutdown_connections 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fa984039450 0x7fa98403b910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 --2- 172.21.15.113:0/1428462076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa99c07ee50 0x7fa99c19e8b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 >> 172.21.15.113:0/1428462076 conn(0x7fa99c103df0 msgr2=0x7fa99c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 shutdown_connections 2024-04-22T07:05:31.217 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:30.721+0000 7fa9a177f700 1 -- 172.21.15.113:0/1428462076 wait complete. 2024-04-22T07:05:31.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:30] ENGINE Bus STARTING 2024-04-22T07:05:31.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:30] ENGINE Serving on http://172.21.15.113:8765 2024-04-22T07:05:31.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:30] ENGINE Serving on https://172.21.15.113:7150 2024-04-22T07:05:31.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:30] ENGINE Bus STARTED 2024-04-22T07:05:31.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: mgrmap e8: x(active, since 2s) 2024-04-22T07:05:31.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:31.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:31 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:05:32.431 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.616+0000 7f2cd3b42700 1 Processor -- start 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd3b42700 1 -- start start 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd3b42700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc107470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd3b42700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc07cdc0 con 0x7f2ccc107050 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd18de700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd18de700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc107470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40110/0 (socket says 172.21.15.113:40110) 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd18de700 1 -- 172.21.15.113:0/2309062257 learned_addr learned my addr 172.21.15.113:0/2309062257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd18de700 1 -- 172.21.15.113:0/2309062257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc1079b0 con 0x7f2ccc107050 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd18de700 1 --2- 172.21.15.113:0/2309062257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc107470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0098a0 tx=0x7f2cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7438367c5d89b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.617+0000 7f2cd08dc700 1 -- 172.21.15.113:0/2309062257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc004160 con 0x7f2ccc107050 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd08dc700 1 -- 172.21.15.113:0/2309062257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f2cbc004300 con 0x7f2ccc107050 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd08dc700 1 -- 172.21.15.113:0/2309062257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc011b20 con 0x7f2ccc107050 2024-04-22T07:05:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 -- 172.21.15.113:0/2309062257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 msgr2=0x7f2ccc107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 --2- 172.21.15.113:0/2309062257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc107470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0098a0 tx=0x7f2cbc009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 -- 172.21.15.113:0/2309062257 shutdown_connections 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 --2- 172.21.15.113:0/2309062257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc107470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 -- 172.21.15.113:0/2309062257 >> 172.21.15.113:0/2309062257 conn(0x7f2ccc102c50 msgr2=0x7f2ccc105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 -- 172.21.15.113:0/2309062257 shutdown_connections 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.618+0000 7f2cd3b42700 1 -- 172.21.15.113:0/2309062257 wait complete. 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd3b42700 1 Processor -- start 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd3b42700 1 -- start start 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd3b42700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd3b42700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1088c0 con 0x7f2ccc107050 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd18de700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd18de700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40124/0 (socket says 172.21.15.113:40124) 2024-04-22T07:05:32.438 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd18de700 1 -- 172.21.15.113:0/863416539 learned_addr learned my addr 172.21.15.113:0/863416539 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.619+0000 7f2cd18de700 1 -- 172.21.15.113:0/863416539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19edc0 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.620+0000 7f2cd18de700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc19e880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0116a0 tx=0x7f2cbc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.620+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc003ab0 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.620+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f2cbc003c50 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.620+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19f050 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.620+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cbc019870 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.620+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc19f4b0 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.621+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc0681b0 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.621+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2cbc01ac00 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.621+0000 7f2cc2ffd700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f2cb8039560 0x7f2cb803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.621+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2cbc04d840 con 0x7f2ccc107050 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.621+0000 7f2cd10dd700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f2cb8039560 0x7f2cb803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.622+0000 7f2cd10dd700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f2cb8039560 0x7f2cb803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8006fd0 tx=0x7f2cc8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:32.439 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.626+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cbc011e20 con 0x7f2ccc107050 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.826+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2ccc0754f0 con 0x7f2cb8039560 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.827+0000 7f2cc2ffd700 1 -- 172.21.15.113:0/863416539 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2ccc0754f0 con 0x7f2cb8039560 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f2cb8039560 msgr2=0x7f2cb803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f2cb8039560 0x7f2cb803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8006fd0 tx=0x7f2cc8006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 msgr2=0x7f2ccc19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc19e880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0116a0 tx=0x7f2cbc011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 shutdown_connections 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f2cb8039560 0x7f2cb803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 --2- 172.21.15.113:0/863416539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ccc107050 0x7f2ccc19e880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 >> 172.21.15.113:0/863416539 conn(0x7f2ccc102c50 msgr2=0x7f2ccc195460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 shutdown_connections 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:31.829+0000 7f2cd3b42700 1 -- 172.21.15.113:0/863416539 wait complete. 2024-04-22T07:05:32.440 INFO:teuthology.orchestra.run.smithi113.stdout:Generating ssh key... 2024-04-22T07:05:32.695 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:32 smithi113 ceph-mon[27933]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:33.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: Generating public/private rsa key pair. 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: Your identification has been saved in /tmp/tmp71ymppx0/key. 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: Your public key has been saved in /tmp/tmp71ymppx0/key.pub. 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: The key fingerprint is: 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: SHA256:0CaVuu8gVydW0Ce4i8FVrrrwYVQ6Et7svSBupeO5fp8 ceph-81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: The key's randomart image is: 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: +---[RSA 3072]----+ 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | o=. | 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | o+oo . | 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | ..oo+.oo | 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | . ++*.o | 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | o *+S.. | 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | ++*.o | 2024-04-22T07:05:33.585 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | +oO.. | 2024-04-22T07:05:33.586 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | .+O.=.o | 2024-04-22T07:05:33.586 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: | +*+ooE | 2024-04-22T07:05:33.586 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: +----[SHA256]-----+ 2024-04-22T07:05:33.877 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-mon[27933]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:33.877 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:33.877 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:33 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f8834943700 1 Processor -- start 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f8834943700 1 -- start start 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f8834943700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883010b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f8834943700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f883007cde0 con 0x7f8830109240 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f882e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883010b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f882e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883010b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40140/0 (socket says 172.21.15.113:40140) 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.894+0000 7f882e59c700 1 -- 172.21.15.113:0/13587635 learned_addr learned my addr 172.21.15.113:0/13587635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f882e59c700 1 -- 172.21.15.113:0/13587635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883010bb70 con 0x7f8830109240 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f882e59c700 1 --2- 172.21.15.113:0/13587635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883010b630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88200098a0 tx=0x7f8820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b65d536078ba7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:34.144 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f882d59a700 1 -- 172.21.15.113:0/13587635 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820004160 con 0x7f8830109240 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f882d59a700 1 -- 172.21.15.113:0/13587635 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f8820004300 con 0x7f8830109240 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f882d59a700 1 -- 172.21.15.113:0/13587635 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820011b20 con 0x7f8830109240 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f8834943700 1 -- 172.21.15.113:0/13587635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 msgr2=0x7f883010b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.895+0000 7f8834943700 1 --2- 172.21.15.113:0/13587635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883010b630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88200098a0 tx=0x7f8820009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 -- 172.21.15.113:0/13587635 shutdown_connections 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 --2- 172.21.15.113:0/13587635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883010b630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 -- 172.21.15.113:0/13587635 >> 172.21.15.113:0/13587635 conn(0x7f8830102c10 msgr2=0x7f8830105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 -- 172.21.15.113:0/13587635 shutdown_connections 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 -- 172.21.15.113:0/13587635 wait complete. 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 Processor -- start 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.896+0000 7f8834943700 1 -- start start 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f8834943700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f8834943700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f883010cb30 con 0x7f8830109240 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f882e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:34.145 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f882e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40144/0 (socket says 172.21.15.113:40144) 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f882e59c700 1 -- 172.21.15.113:0/1120118053 learned_addr learned my addr 172.21.15.113:0/1120118053 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f882e59c700 1 -- 172.21.15.113:0/1120118053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883019cbe0 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f882e59c700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883019c6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88200116a0 tx=0x7f8820011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820003ab0 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f8820003c50 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883019ce70 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820019870 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.897+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883019d2d0 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.898+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8820019ae0 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.898+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88300681b0 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.898+0000 7f881f7fe700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f88180395b0 0x7f881803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.898+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f882004d890 con 0x7f8830109240 2024-04-22T07:05:34.146 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.899+0000 7f882dd9b700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f88180395b0 0x7f881803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.899+0000 7f882dd9b700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f88180395b0 0x7f881803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8824006fd0 tx=0x7f8824006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:32.903+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8820011e20 con 0x7f8830109240 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.102+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8830103e60 con 0x7f88180395b0 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.278+0000 7f881f7fe700 1 -- 172.21.15.113:0/1120118053 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8830103e60 con 0x7f88180395b0 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f88180395b0 msgr2=0x7f881803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f88180395b0 0x7f881803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8824006fd0 tx=0x7f8824006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 msgr2=0x7f883019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883019c6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88200116a0 tx=0x7f8820011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 shutdown_connections 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f88180395b0 0x7f881803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 --2- 172.21.15.113:0/1120118053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8830109240 0x7f883019c6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 >> 172.21.15.113:0/1120118053 conn(0x7f8830102c10 msgr2=0x7f8830103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 shutdown_connections 2024-04-22T07:05:34.147 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:33.280+0000 7f8834943700 1 -- 172.21.15.113:0/1120118053 wait complete. 2024-04-22T07:05:34.771 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:34 smithi113 ceph-mon[27933]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:34.771 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:34 smithi113 ceph-mon[27933]: Generating ssh key... 2024-04-22T07:05:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:05:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd749a5b700 1 Processor -- start 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd749a5b700 1 -- start start 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd749a5b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74410a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd749a5b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd744108350 con 0x7fd744109e10 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd742ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd742ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74410a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40150/0 (socket says 172.21.15.113:40150) 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd742ffd700 1 -- 172.21.15.113:0/4053082105 learned_addr learned my addr 172.21.15.113:0/4053082105 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd742ffd700 1 -- 172.21.15.113:0/4053082105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441084d0 con 0x7fd744109e10 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd742ffd700 1 --2- 172.21.15.113:0/4053082105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74410a230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e80 tx=0x7fd73c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1defa9a24b18216 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.561+0000 7fd741ffb700 1 -- 172.21.15.113:0/4053082105 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd73c017070 con 0x7fd744109e10 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd741ffb700 1 -- 172.21.15.113:0/4053082105 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fd73c00b8c0 con 0x7fd744109e10 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd741ffb700 1 -- 172.21.15.113:0/4053082105 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd73c0045c0 con 0x7fd744109e10 2024-04-22T07:05:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 -- 172.21.15.113:0/4053082105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 msgr2=0x7fd74410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 --2- 172.21.15.113:0/4053082105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74410a230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e80 tx=0x7fd73c00b330 comp rx=0 tx=0).stop 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 -- 172.21.15.113:0/4053082105 shutdown_connections 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 --2- 172.21.15.113:0/4053082105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74410a230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 -- 172.21.15.113:0/4053082105 >> 172.21.15.113:0/4053082105 conn(0x7fd744103d50 msgr2=0x7fd744106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 -- 172.21.15.113:0/4053082105 shutdown_connections 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.562+0000 7fd749a5b700 1 -- 172.21.15.113:0/4053082105 wait complete. 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd749a5b700 1 Processor -- start 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd749a5b700 1 -- start start 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd749a5b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74407da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd749a5b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd74410cce0 con 0x7fd744109e10 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd742ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74407da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd742ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74407da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40164/0 (socket says 172.21.15.113:40164) 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.563+0000 7fd742ffd700 1 -- 172.21.15.113:0/2835189205 learned_addr learned my addr 172.21.15.113:0/2835189205 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd742ffd700 1 -- 172.21.15.113:0/2835189205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74407df80 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd742ffd700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74407da40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e50 tx=0x7fd73c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd73c017070 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74407c170 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fd73c022070 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd73c01d690 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.564+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74407c6b0 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.565+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd73c01d950 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.565+0000 7fd748a59700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fd72c039560 0x7fd72c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.565+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd73c00f050 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.565+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440681b0 con 0x7fd744109e10 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.565+0000 7fd7427fc700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fd72c039560 0x7fd72c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.565+0000 7fd7427fc700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fd72c039560 0x7fd72c03ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd738006fd0 tx=0x7fd738006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.570+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd73c02c200 con 0x7fd744109e10 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.769+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd74407ce50 con 0x7fd72c039560 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.769+0000 7fd748a59700 1 -- 172.21.15.113:0/2835189205 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd74407ce50 con 0x7fd72c039560 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.771+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fd72c039560 msgr2=0x7fd72c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.771+0000 7fd749a5b700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fd72c039560 0x7fd72c03ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd738006fd0 tx=0x7fd738006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.771+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 msgr2=0x7fd74407da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74407da40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e50 tx=0x7fd73c00bd50 comp rx=0 tx=0).stop 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 shutdown_connections 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fd72c039560 0x7fd72c03ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 --2- 172.21.15.113:0/2835189205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd744109e10 0x7fd74407da40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 >> 172.21.15.113:0/2835189205 conn(0x7fd744103d50 msgr2=0x7fd744104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 shutdown_connections 2024-04-22T07:05:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:34.772+0000 7fd749a5b700 1 -- 172.21.15.113:0/2835189205 wait complete. 2024-04-22T07:05:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-22T07:05:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:Adding key to root@localhost authorized_keys... 2024-04-22T07:05:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:Adding host smithi113... 2024-04-22T07:05:35.999 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:35 smithi113 ceph-mon[27933]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:36.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:36 smithi113 ceph-mon[27933]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi113", "addr": "172.21.15.113", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:36.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:36 smithi113 ceph-mon[27933]: Deploying cephadm binary to smithi113 2024-04-22T07:05:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout Added host 'smithi113' with addr '172.21.15.113' 2024-04-22T07:05:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.788+0000 7fb087d49700 1 Processor -- start 2024-04-22T07:05:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb087d49700 1 -- start start 2024-04-22T07:05:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb087d49700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb080106e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb087d49700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb080107400 con 0x7fb080106a10 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb085ae5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb080106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb085ae5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb080106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37160/0 (socket says 172.21.15.113:37160) 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb085ae5700 1 -- 172.21.15.113:0/343500365 learned_addr learned my addr 172.21.15.113:0/343500365 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb085ae5700 1 -- 172.21.15.113:0/343500365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0801099a0 con 0x7fb080106a10 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb085ae5700 1 --2- 172.21.15.113:0/343500365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb080106e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0700098a0 tx=0x7fb070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf10c78f7135b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb084ae3700 1 -- 172.21.15.113:0/343500365 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070004160 con 0x7fb080106a10 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb084ae3700 1 -- 172.21.15.113:0/343500365 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fb070004300 con 0x7fb080106a10 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.789+0000 7fb084ae3700 1 -- 172.21.15.113:0/343500365 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070011b20 con 0x7fb080106a10 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 -- 172.21.15.113:0/343500365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 msgr2=0x7fb080106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 --2- 172.21.15.113:0/343500365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb080106e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0700098a0 tx=0x7fb070009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 -- 172.21.15.113:0/343500365 shutdown_connections 2024-04-22T07:05:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 --2- 172.21.15.113:0/343500365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb080106e30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 -- 172.21.15.113:0/343500365 >> 172.21.15.113:0/343500365 conn(0x7fb080102360 msgr2=0x7fb080104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 -- 172.21.15.113:0/343500365 shutdown_connections 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.790+0000 7fb087d49700 1 -- 172.21.15.113:0/343500365 wait complete. 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb087d49700 1 Processor -- start 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb087d49700 1 -- start start 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb087d49700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb08019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb087d49700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb08010cd50 con 0x7fb080106a10 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb085ae5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb08019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb085ae5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb08019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37170/0 (socket says 172.21.15.113:37170) 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb085ae5700 1 -- 172.21.15.113:0/3404078771 learned_addr learned my addr 172.21.15.113:0/3404078771 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb085ae5700 1 -- 172.21.15.113:0/3404078771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019cc00 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb085ae5700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb08019c6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb070006af0 tx=0x7fb070011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070003b90 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019ce90 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fb070003d30 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.791+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0700197c0 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.792+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08019d2f0 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.792+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb070022020 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.792+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0800681b0 con 0x7fb080106a10 2024-04-22T07:05:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.792+0000 7fb076ffd700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fb06c0395b0 0x7fb06c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.793+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb07004d920 con 0x7fb080106a10 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.793+0000 7fb0852e4700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fb06c0395b0 0x7fb06c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.793+0000 7fb0852e4700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fb06c0395b0 0x7fb06c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006fd0 tx=0x7fb07c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:35.798+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb07000bc30 con 0x7fb080106a10 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:36.000+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi113", "addr": "172.21.15.113", "target": ["mon-mgr", ""]}) -- 0x7fb080103510 con 0x7fb06c0395b0 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.017+0000 7fb076ffd700 1 -- 172.21.15.113:0/3404078771 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb080103510 con 0x7fb06c0395b0 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.019+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fb06c0395b0 msgr2=0x7fb06c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.019+0000 7fb087d49700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fb06c0395b0 0x7fb06c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006fd0 tx=0x7fb07c006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.019+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 msgr2=0x7fb08019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.019+0000 7fb087d49700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb08019c6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb070006af0 tx=0x7fb070011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.020+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 shutdown_connections 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.020+0000 7fb087d49700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fb06c0395b0 0x7fb06c03ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.020+0000 7fb087d49700 1 --2- 172.21.15.113:0/3404078771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb080106a10 0x7fb08019c6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.020+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 >> 172.21.15.113:0/3404078771 conn(0x7fb080102360 msgr2=0x7fb080102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.020+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 shutdown_connections 2024-04-22T07:05:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:40.020+0000 7fb087d49700 1 -- 172.21.15.113:0/3404078771 wait complete. 2024-04-22T07:05:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:Deploying unmanaged mon service... 2024-04-22T07:05:41.120 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:41 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:41.120 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:41 smithi113 ceph-mon[27933]: Added host smithi113 2024-04-22T07:05:41.120 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:41 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:05:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-22T07:05:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.081+0000 7f8ee8215700 1 Processor -- start 2024-04-22T07:05:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.081+0000 7f8ee8215700 1 -- start start 2024-04-22T07:05:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.081+0000 7f8ee8215700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee0106e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee8215700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01073d0 con 0x7f8ee01069e0 2024-04-22T07:05:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee5fb1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee0106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee5fb1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee0106e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37184/0 (socket says 172.21.15.113:37184) 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee5fb1700 1 -- 172.21.15.113:0/2182847952 learned_addr learned my addr 172.21.15.113:0/2182847952 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee5fb1700 1 -- 172.21.15.113:0/2182847952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee0107550 con 0x7f8ee01069e0 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee5fb1700 1 --2- 172.21.15.113:0/2182847952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee0106e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00098a0 tx=0x7f8ed0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ce199770aca4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee4faf700 1 -- 172.21.15.113:0/2182847952 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ed0004160 con 0x7f8ee01069e0 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee4faf700 1 -- 172.21.15.113:0/2182847952 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f8ed0004300 con 0x7f8ee01069e0 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.082+0000 7f8ee4faf700 1 -- 172.21.15.113:0/2182847952 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ed0011b20 con 0x7f8ee01069e0 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 -- 172.21.15.113:0/2182847952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 msgr2=0x7f8ee0106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 --2- 172.21.15.113:0/2182847952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee0106e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00098a0 tx=0x7f8ed0009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 -- 172.21.15.113:0/2182847952 shutdown_connections 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 --2- 172.21.15.113:0/2182847952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee0106e00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 -- 172.21.15.113:0/2182847952 >> 172.21.15.113:0/2182847952 conn(0x7f8ee0102360 msgr2=0x7f8ee0104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 -- 172.21.15.113:0/2182847952 shutdown_connections 2024-04-22T07:05:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 -- 172.21.15.113:0/2182847952 wait complete. 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.083+0000 7f8ee8215700 1 Processor -- start 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee8215700 1 -- start start 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee8215700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee8215700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee010a8a0 con 0x7f8ee01069e0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee5fb1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee5fb1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37188/0 (socket says 172.21.15.113:37188) 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee5fb1700 1 -- 172.21.15.113:0/466888190 learned_addr learned my addr 172.21.15.113:0/466888190 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee5fb1700 1 -- 172.21.15.113:0/466888190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee019cbb0 con 0x7f8ee01069e0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee5fb1700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee019c670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006af0 tx=0x7f8ed0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ed0003930 con 0x7f8ee01069e0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f8ed0003ad0 con 0x7f8ee01069e0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee019cea0 con 0x7f8ee01069e0 2024-04-22T07:05:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ed0019610 con 0x7f8ee01069e0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.084+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee019fb30 con 0x7f8ee01069e0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.085+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8ed001a430 con 0x7f8ee01069e0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.085+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee00681b0 con 0x7f8ee01069e0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.085+0000 7f8ed6ffd700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f8ecc0395b0 0x7f8ecc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.085+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ed004da00 con 0x7f8ee01069e0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.086+0000 7f8ee57b0700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f8ecc0395b0 0x7f8ecc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.086+0000 7f8ee57b0700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f8ecc0395b0 0x7f8ecc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8edc006fd0 tx=0x7f8edc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.091+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ed0011e20 con 0x7f8ee01069e0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.306+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8ee01035a0 con 0x7f8ecc0395b0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.313+0000 7f8ed6ffd700 1 -- 172.21.15.113:0/466888190 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8ee01035a0 con 0x7f8ecc0395b0 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.315+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f8ecc0395b0 msgr2=0x7f8ecc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.315+0000 7f8ee8215700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f8ecc0395b0 0x7f8ecc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8edc006fd0 tx=0x7f8edc006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.315+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 msgr2=0x7f8ee019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.315+0000 7f8ee8215700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee019c670 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006af0 tx=0x7f8ed0011c10 comp rx=0 tx=0).stop 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.315+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 shutdown_connections 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.316+0000 7f8ee8215700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f8ecc0395b0 0x7f8ecc03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.316+0000 7f8ee8215700 1 --2- 172.21.15.113:0/466888190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ee01069e0 0x7f8ee019c670 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.316+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 >> 172.21.15.113:0/466888190 conn(0x7f8ee0102360 msgr2=0x7f8ee0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.316+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 shutdown_connections 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:41.316+0000 7f8ee8215700 1 -- 172.21.15.113:0/466888190 wait complete. 2024-04-22T07:05:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:Deploying unmanaged mgr service... 2024-04-22T07:05:42.501 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:42 smithi113 ceph-mon[27933]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:42.501 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:42 smithi113 ceph-mon[27933]: Saving service mon spec with placement count:5 2024-04-22T07:05:42.501 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:42 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.276+0000 7feee775b700 1 Processor -- start 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.276+0000 7feee775b700 1 -- start start 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.276+0000 7feee775b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee0107a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.276+0000 7feee775b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feee0108040 con 0x7feee0107650 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feee54f7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee0107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feee54f7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee0107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37202/0 (socket says 172.21.15.113:37202) 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feee54f7700 1 -- 172.21.15.113:0/3423106180 learned_addr learned my addr 172.21.15.113:0/3423106180 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feee54f7700 1 -- 172.21.15.113:0/3423106180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee0108800 con 0x7feee0107650 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feee54f7700 1 --2- 172.21.15.113:0/3423106180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee0107a70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feedc00bb40 tx=0x7feedc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e2d814dc274a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feed7fff700 1 -- 172.21.15.113:0/3423106180 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc00dbe0 con 0x7feee0107650 2024-04-22T07:05:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feed7fff700 1 -- 172.21.15.113:0/3423106180 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7feedc0044e0 con 0x7feee0107650 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.277+0000 7feed7fff700 1 -- 172.21.15.113:0/3423106180 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc010470 con 0x7feee0107650 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 -- 172.21.15.113:0/3423106180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 msgr2=0x7feee0107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 --2- 172.21.15.113:0/3423106180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee0107a70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feedc00bb40 tx=0x7feedc00be50 comp rx=0 tx=0).stop 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 -- 172.21.15.113:0/3423106180 shutdown_connections 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 --2- 172.21.15.113:0/3423106180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee0107a70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 -- 172.21.15.113:0/3423106180 >> 172.21.15.113:0/3423106180 conn(0x7feee0102c50 msgr2=0x7feee0105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 -- 172.21.15.113:0/3423106180 shutdown_connections 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 -- 172.21.15.113:0/3423106180 wait complete. 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 Processor -- start 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.278+0000 7feee775b700 1 -- start start 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee775b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee775b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feee007bfb0 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee54f7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee54f7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37204/0 (socket says 172.21.15.113:37204) 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee54f7700 1 -- 172.21.15.113:0/3822292061 learned_addr learned my addr 172.21.15.113:0/3822292061 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee54f7700 1 -- 172.21.15.113:0/3822292061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee019a9e0 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee54f7700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee019a4a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feedc00d650 tx=0x7feedc004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc027070 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7feedc010d50 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc022cf0 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee019ac70 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.279+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee019d960 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.280+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7feedc02a070 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.280+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee00681b0 con 0x7feee0107650 2024-04-22T07:05:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.281+0000 7feed67fc700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7feecc0395b0 0x7feecc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.281+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feedc01f070 con 0x7feee0107650 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.281+0000 7feee4cf6700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7feecc0395b0 0x7feecc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.281+0000 7feee4cf6700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7feecc0395b0 0x7feecc03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feed0006fd0 tx=0x7feed0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.286+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feedc015070 con 0x7feee0107650 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.503+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7feee00754f0 con 0x7feecc0395b0 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.510+0000 7feed67fc700 1 -- 172.21.15.113:0/3822292061 <== mgr.14118 v2:172.21.15.113:6800/1807162111 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feee00754f0 con 0x7feecc0395b0 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7feecc0395b0 msgr2=0x7feecc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7feecc0395b0 0x7feecc03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feed0006fd0 tx=0x7feed0006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 msgr2=0x7feee019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee019a4a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feedc00d650 tx=0x7feedc004080 comp rx=0 tx=0).stop 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 shutdown_connections 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7feecc0395b0 0x7feecc03ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 --2- 172.21.15.113:0/3822292061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feee0107650 0x7feee019a4a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.512+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 >> 172.21.15.113:0/3822292061 conn(0x7feee0102c50 msgr2=0x7feee0190ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.513+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 shutdown_connections 2024-04-22T07:05:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:42.513+0000 7feee775b700 1 -- 172.21.15.113:0/3822292061 wait complete. 2024-04-22T07:05:43.684 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:43 smithi113 ceph-mon[27933]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:43.684 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:43 smithi113 ceph-mon[27933]: Saving service mgr spec with placement count:2 2024-04-22T07:05:43.684 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:43 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:44.710 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c8218700 1 Processor -- start 2024-04-22T07:05:44.710 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c8218700 1 -- start start 2024-04-22T07:05:44.710 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c8218700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0106e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c8218700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50c0107420 con 0x7f50c0106a30 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c5fb4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c5fb4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37214/0 (socket says 172.21.15.113:37214) 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c5fb4700 1 -- 172.21.15.113:0/1848789612 learned_addr learned my addr 172.21.15.113:0/1848789612 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.475+0000 7f50c5fb4700 1 -- 172.21.15.113:0/1848789612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c01099c0 con 0x7f50c0106a30 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c5fb4700 1 --2- 172.21.15.113:0/1848789612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0106e50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0098a0 tx=0x7f50bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673726808ea694c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c4fb2700 1 -- 172.21.15.113:0/1848789612 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50bc004160 con 0x7f50c0106a30 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c4fb2700 1 -- 172.21.15.113:0/1848789612 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f50bc004300 con 0x7f50c0106a30 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c4fb2700 1 -- 172.21.15.113:0/1848789612 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50bc011b20 con 0x7f50c0106a30 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 -- 172.21.15.113:0/1848789612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 msgr2=0x7f50c0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 --2- 172.21.15.113:0/1848789612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0106e50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0098a0 tx=0x7f50bc009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:44.711 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 -- 172.21.15.113:0/1848789612 shutdown_connections 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 --2- 172.21.15.113:0/1848789612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0106e50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 -- 172.21.15.113:0/1848789612 >> 172.21.15.113:0/1848789612 conn(0x7f50c0102340 msgr2=0x7f50c01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 -- 172.21.15.113:0/1848789612 shutdown_connections 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.476+0000 7f50c8218700 1 -- 172.21.15.113:0/1848789612 wait complete. 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c8218700 1 Processor -- start 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c8218700 1 -- start start 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c8218700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c8218700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50c010cd70 con 0x7f50c0106a30 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c5fb4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c5fb4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37220/0 (socket says 172.21.15.113:37220) 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c5fb4700 1 -- 172.21.15.113:0/336839712 learned_addr learned my addr 172.21.15.113:0/336839712 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.477+0000 7f50c5fb4700 1 -- 172.21.15.113:0/336839712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c0109780 con 0x7f50c0106a30 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.478+0000 7f50c5fb4700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0109240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50bc006af0 tx=0x7f50bc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.478+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50bc003b90 con 0x7f50c0106a30 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.478+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f50bc003d30 con 0x7f50c0106a30 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.478+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50bc0197c0 con 0x7f50c0106a30 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.478+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c0107970 con 0x7f50c0106a30 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.478+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c0107e30 con 0x7f50c0106a30 2024-04-22T07:05:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.479+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f50bc022020 con 0x7f50c0106a30 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.479+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c00681b0 con 0x7f50c0106a30 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.479+0000 7f50b6ffd700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f50ac039600 0x7f50ac03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.479+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f50bc04d920 con 0x7f50c0106a30 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.479+0000 7f50c57b3700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f50ac039600 0x7f50ac03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.479+0000 7f50c57b3700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f50ac039600 0x7f50ac03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50b0006fd0 tx=0x7f50b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.484+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50bc0042c0 con 0x7f50c0106a30 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.685+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f50c0108650 con 0x7f50c0106a30 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.690+0000 7f50b6ffd700 1 -- 172.21.15.113:0/336839712 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f50bc00be10 con 0x7f50c0106a30 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f50ac039600 msgr2=0x7f50ac03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f50ac039600 0x7f50ac03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50b0006fd0 tx=0x7f50b0006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 msgr2=0x7f50c0109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0109240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50bc006af0 tx=0x7f50bc011ce0 comp rx=0 tx=0).stop 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 shutdown_connections 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f50ac039600 0x7f50ac03bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 --2- 172.21.15.113:0/336839712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50c0106a30 0x7f50c0109240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 >> 172.21.15.113:0/336839712 conn(0x7f50c0102340 msgr2=0x7f50c0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 shutdown_connections 2024-04-22T07:05:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:43.692+0000 7f50c8218700 1 -- 172.21.15.113:0/336839712 wait complete. 2024-04-22T07:05:44.971 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:44 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/336839712' entity='client.admin' 2024-04-22T07:05:44.972 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:44 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:45 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3467004442' entity='client.admin' 2024-04-22T07:05:46.562 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.167+0000 7f54dbdc7700 1 Processor -- start 2024-04-22T07:05:46.562 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54dbdc7700 1 -- start start 2024-04-22T07:05:46.562 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54dbdc7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d4107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54dbdc7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4108000 con 0x7f54d4107610 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54d9b63700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d4107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54d9b63700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d4107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37222/0 (socket says 172.21.15.113:37222) 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54d9b63700 1 -- 172.21.15.113:0/1019460431 learned_addr learned my addr 172.21.15.113:0/1019460431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54d9b63700 1 -- 172.21.15.113:0/1019460431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4108810 con 0x7f54d4107610 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54d9b63700 1 --2- 172.21.15.113:0/1019460431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d4107a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54d00098a0 tx=0x7f54d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1625c8b1bcdaaea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.168+0000 7f54d8b61700 1 -- 172.21.15.113:0/1019460431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54d0004160 con 0x7f54d4107610 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54d8b61700 1 -- 172.21.15.113:0/1019460431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f54d0004300 con 0x7f54d4107610 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54d8b61700 1 -- 172.21.15.113:0/1019460431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54d0011af0 con 0x7f54d4107610 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 -- 172.21.15.113:0/1019460431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 msgr2=0x7f54d4107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 --2- 172.21.15.113:0/1019460431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d4107a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54d00098a0 tx=0x7f54d0009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 -- 172.21.15.113:0/1019460431 shutdown_connections 2024-04-22T07:05:46.563 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 --2- 172.21.15.113:0/1019460431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d4107a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 -- 172.21.15.113:0/1019460431 >> 172.21.15.113:0/1019460431 conn(0x7f54d4102c10 msgr2=0x7f54d4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 -- 172.21.15.113:0/1019460431 shutdown_connections 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.169+0000 7f54dbdc7700 1 -- 172.21.15.113:0/1019460431 wait complete. 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54dbdc7700 1 Processor -- start 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54dbdc7700 1 -- start start 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54dbdc7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54dbdc7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54d410bbc0 con 0x7f54d4107610 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54d9b63700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54d9b63700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37226/0 (socket says 172.21.15.113:37226) 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54d9b63700 1 -- 172.21.15.113:0/3467004442 learned_addr learned my addr 172.21.15.113:0/3467004442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.170+0000 7f54d9b63700 1 -- 172.21.15.113:0/3467004442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d419cb80 con 0x7f54d4107610 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.171+0000 7f54d9b63700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d419c640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009520 tx=0x7f54d0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.171+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54d00036a0 con 0x7f54d4107610 2024-04-22T07:05:46.564 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.171+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f54d001a430 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.171+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d419ce10 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.171+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d419fb00 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.171+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54d00193c0 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.172+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f54d0019670 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.172+0000 7f54caffd700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f54c0039560 0x7f54c003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.172+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f54d004e890 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.172+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b8005360 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.172+0000 7f54d9362700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f54c0039560 0x7f54c003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.172+0000 7f54d9362700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f54c0039560 0x7f54c003ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006fd0 tx=0x7f54c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.177+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54d0020240 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.394+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f54b8005650 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.396+0000 7f54caffd700 1 -- 172.21.15.113:0/3467004442 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f54d0029d50 con 0x7f54d4107610 2024-04-22T07:05:46.565 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f54c0039560 msgr2=0x7f54c003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f54c0039560 0x7f54c003ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006fd0 tx=0x7f54c4006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 msgr2=0x7f54d419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d419c640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009520 tx=0x7f54d0003eb0 comp rx=0 tx=0).stop 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 shutdown_connections 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7f54c0039560 0x7f54c003ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 --2- 172.21.15.113:0/3467004442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f54d4107610 0x7f54d419c640 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 >> 172.21.15.113:0/3467004442 conn(0x7f54d4102c10 msgr2=0x7f54d4103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 shutdown_connections 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:45.399+0000 7f54dbdc7700 1 -- 172.21.15.113:0/3467004442 wait complete. 2024-04-22T07:05:46.566 INFO:teuthology.orchestra.run.smithi113.stdout:Enabling the dashboard module... 2024-04-22T07:05:48.437 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-mon[27933]: from='mgr.14118 172.21.15.113:0/4006368748' entity='mgr.x' 2024-04-22T07:05:48.437 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2786146379' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-22T07:05:48.437 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: ignoring --setuser ceph since I am not root 2024-04-22T07:05:48.437 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: ignoring --setgroup ceph since I am not root 2024-04-22T07:05:48.437 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:48.281+0000 7f1e3b8bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.964+0000 7fcbded01700 1 Processor -- start 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.964+0000 7fcbded01700 1 -- start start 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbded01700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd8106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbded01700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81073e0 con 0x7fcbd81069f0 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbdca9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd8106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbdca9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd8106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41580/0 (socket says 172.21.15.113:41580) 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbdca9d700 1 -- 172.21.15.113:0/845456316 learned_addr learned my addr 172.21.15.113:0/845456316 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbdca9d700 1 -- 172.21.15.113:0/845456316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd8109980 con 0x7fcbd81069f0 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.965+0000 7fcbdca9d700 1 --2- 172.21.15.113:0/845456316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd8106e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0098a0 tx=0x7fcbcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1526e34d03c71ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:48.683 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbd77fe700 1 -- 172.21.15.113:0/845456316 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbcc004160 con 0x7fcbd81069f0 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbd77fe700 1 -- 172.21.15.113:0/845456316 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcbcc004300 con 0x7fcbd81069f0 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbd77fe700 1 -- 172.21.15.113:0/845456316 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbcc011af0 con 0x7fcbd81069f0 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 -- 172.21.15.113:0/845456316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 msgr2=0x7fcbd8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 --2- 172.21.15.113:0/845456316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd8106e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0098a0 tx=0x7fcbcc009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 -- 172.21.15.113:0/845456316 shutdown_connections 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 --2- 172.21.15.113:0/845456316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd8106e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 -- 172.21.15.113:0/845456316 >> 172.21.15.113:0/845456316 conn(0x7fcbd81023c0 msgr2=0x7fcbd81047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 -- 172.21.15.113:0/845456316 shutdown_connections 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.966+0000 7fcbded01700 1 -- 172.21.15.113:0/845456316 wait complete. 2024-04-22T07:05:48.684 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbded01700 1 Processor -- start 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbded01700 1 -- start start 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbded01700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbded01700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd810cd30 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbdca9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbdca9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41590/0 (socket says 172.21.15.113:41590) 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbdca9d700 1 -- 172.21.15.113:0/2786146379 learned_addr learned my addr 172.21.15.113:0/2786146379 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.967+0000 7fcbdca9d700 1 -- 172.21.15.113:0/2786146379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd819cb90 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.968+0000 7fcbdca9d700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd819c650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009520 tx=0x7fcbcc003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.968+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbcc0036a0 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.968+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcbcc01a430 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.968+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd819ce20 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.968+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbcc0193c0 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.968+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd819fb10 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.969+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcbcc019560 con 0x7fcbd81069f0 2024-04-22T07:05:48.685 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.969+0000 7fcbd5ffb700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fcbc80395b0 0x7fcbc803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.969+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcbcc04d810 con 0x7fcbd81069f0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.969+0000 7fcbd7fff700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fcbc80395b0 0x7fcbc803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.969+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd80681b0 con 0x7fcbd81069f0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.969+0000 7fcbd7fff700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fcbc80395b0 0x7fcbc803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4006fd0 tx=0x7fcbc4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:46.974+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbcc00f2e0 con 0x7fcbd81069f0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:47.243+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcbd81a0060 con 0x7fcbd81069f0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.098+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcbcc00b7e0 con 0x7fcbd81069f0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.100+0000 7fcbd5ffb700 1 -- 172.21.15.113:0/2786146379 <== mon.0 v2:172.21.15.113:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcbcc01a5e0 con 0x7fcbd81069f0 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fcbc80395b0 msgr2=0x7fcbc803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fcbc80395b0 0x7fcbc803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4006fd0 tx=0x7fcbc4006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 msgr2=0x7fcbd819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:48.686 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd819c650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009520 tx=0x7fcbcc003eb0 comp rx=0 tx=0).stop 2024-04-22T07:05:48.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 shutdown_connections 2024-04-22T07:05:48.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fcbc80395b0 0x7fcbc803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:48.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 --2- 172.21.15.113:0/2786146379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd81069f0 0x7fcbd819c650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:48.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 >> 172.21.15.113:0/2786146379 conn(0x7fcbd81023c0 msgr2=0x7fcbd8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:48.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 shutdown_connections 2024-04-22T07:05:48.687 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:48.102+0000 7fcbded01700 1 -- 172.21.15.113:0/2786146379 wait complete. 2024-04-22T07:05:48.800 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:48.466+0000 7f1e3b8bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:05:48.801 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:48.569+0000 7f1e3b8bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:05:49.082 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:48.891+0000 7f1e3b8bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:05:49.082 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:48.980+0000 7f1e3b8bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:05:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:49 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2786146379' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-22T07:05:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:49 smithi113 ceph-mon[27933]: mgrmap e9: x(active, since 19s) 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.103+0000 7fdcfc39c700 1 Processor -- start 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.103+0000 7fdcfc39c700 1 -- start start 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfc39c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf410b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfc39c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf4063d40 con 0x7fdcf4109160 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfa138700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf410b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfa138700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf410b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41620/0 (socket says 172.21.15.113:41620) 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfa138700 1 -- 172.21.15.113:0/3057060006 learned_addr learned my addr 172.21.15.113:0/3057060006 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfa138700 1 -- 172.21.15.113:0/3057060006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf410ba90 con 0x7fdcf4109160 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcfa138700 1 --2- 172.21.15.113:0/3057060006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf410b550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00098a0 tx=0x7fdcf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826bc85d5ab1d334 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.104+0000 7fdcf9136700 1 -- 172.21.15.113:0/3057060006 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf0004160 con 0x7fdcf4109160 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcf9136700 1 -- 172.21.15.113:0/3057060006 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdcf0004300 con 0x7fdcf4109160 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcf9136700 1 -- 172.21.15.113:0/3057060006 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf0011af0 con 0x7fdcf4109160 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3057060006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 msgr2=0x7fdcf410b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 --2- 172.21.15.113:0/3057060006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf410b550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00098a0 tx=0x7fdcf0009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:50.021 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3057060006 shutdown_connections 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 --2- 172.21.15.113:0/3057060006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf410b550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3057060006 >> 172.21.15.113:0/3057060006 conn(0x7fdcf4102c10 msgr2=0x7fdcf4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3057060006 shutdown_connections 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.105+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3057060006 wait complete. 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfc39c700 1 Processor -- start 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfc39c700 1 -- start start 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfc39c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfc39c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf410ca90 con 0x7fdcf4109160 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfa138700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfa138700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41636/0 (socket says 172.21.15.113:41636) 2024-04-22T07:05:50.022 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfa138700 1 -- 172.21.15.113:0/3251167742 learned_addr learned my addr 172.21.15.113:0/3251167742 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.106+0000 7fdcfa138700 1 -- 172.21.15.113:0/3251167742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf419cb30 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.107+0000 7fdcfa138700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf419c5f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00117b0 tx=0x7fdcf0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.107+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf00039e0 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.107+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf419ce20 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.107+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdcf0003b80 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.107+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf0024780 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.107+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf419fab0 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.108+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fdcf0022020 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.108+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf40681b0 con 0x7fdcf4109160 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.108+0000 7fdceb7fe700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 0x7fdce003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:50.023 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.108+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdcf004d6a0 con 0x7fdcf4109160 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.108+0000 7fdcf9937700 1 -- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 msgr2=0x7fdce003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.108+0000 7fdcf9937700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 0x7fdce003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.117+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdcf0024920 con 0x7fdcf4109160 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.309+0000 7fdcf9937700 1 -- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 msgr2=0x7fdce003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.309+0000 7fdcf9937700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 0x7fdce003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.406+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdcf419fc80 con 0x7fdcf4109160 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.407+0000 7fdceb7fe700 1 -- 172.21.15.113:0/3251167742 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fdcf002c350 con 0x7fdcf4109160 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 msgr2=0x7fdce003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 0x7fdce003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 msgr2=0x7fdcf419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf419c5f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00117b0 tx=0x7fdcf0011b70 comp rx=0 tx=0).stop 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 shutdown_connections 2024-04-22T07:05:50.024 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7fdce0039660 0x7fdce003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:50.025 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 --2- 172.21.15.113:0/3251167742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcf4109160 0x7fdcf419c5f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:50.025 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 >> 172.21.15.113:0/3251167742 conn(0x7fdcf4102c10 msgr2=0x7fdcf4103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:50.025 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 shutdown_connections 2024-04-22T07:05:50.025 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:49.410+0000 7fdcfc39c700 1 -- 172.21.15.113:0/3251167742 wait complete. 2024-04-22T07:05:50.025 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for the mgr to restart... 2024-04-22T07:05:50.025 INFO:teuthology.orchestra.run.smithi113.stdout:Waiting for mgr epoch 9... 2024-04-22T07:05:50.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:50 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3251167742' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T07:05:50.306 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:50.193+0000 7f1e3b8bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:05:50.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:50.378+0000 7f1e3b8bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:05:50.584 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:50.557+0000 7f1e3b8bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:05:50.951 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:50.857+0000 7f1e3b8bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:05:50.951 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:50.953+0000 7f1e3b8bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:05:51.334 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:51.159+0000 7f1e3b8bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:05:51.819 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:51.354+0000 7f1e3b8bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:05:52.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:51.952+0000 7f1e3b8bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:05:52.669 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:52.374+0000 7f1e3b8bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:05:52.670 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:52.471+0000 7f1e3b8bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:05:52.670 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:52.671+0000 7f1e3b8bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:05:53.527 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:53.189+0000 7f1e3b8bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:05:53.528 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:53.283+0000 7f1e3b8bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:05:53.835 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:53.529+0000 7f1e3b8bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:05:53.835 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:53.663+0000 7f1e3b8bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:05:53.835 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:53.757+0000 7f1e3b8bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:05:54.170 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:53.874+0000 7f1e3b8bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:05:54.170 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:54.171+0000 7f1e3b8bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:05:55.040 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:54.771+0000 7f1e3b8bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:05:55.040 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:05:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:05:54.989+0000 7f1e3b8bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:05:55.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: Active manager daemon x restarted 2024-04-22T07:05:55.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: Activating manager daemon x 2024-04-22T07:05:55.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: osdmap e3: 0 total, 0 up, 0 in 2024-04-22T07:05:55.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: mgrmap e10: x(active, starting, since 0.0033154s) 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: Manager daemon x is now available 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:05:55.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T07:05:56.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T07:05:56.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:56 smithi113 ceph-mon[27933]: mgrmap e11: x(active, since 1.00581s) 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout { 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout } 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.412+0000 7febf8aa3700 1 Processor -- start 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.412+0000 7febf8aa3700 1 -- start start 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf8aa3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf4106e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf8aa3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febf41073d0 con 0x7febf41069e0 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf4106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf4106e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41650/0 (socket says 172.21.15.113:41650) 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf259c700 1 -- 172.21.15.113:0/3902060204 learned_addr learned my addr 172.21.15.113:0/3902060204 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf259c700 1 -- 172.21.15.113:0/3902060204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf4109a10 con 0x7febf41069e0 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf259c700 1 --2- 172.21.15.113:0/3902060204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf4106e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febe40098a0 tx=0x7febe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d3ae4cbf638d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.413+0000 7febf159a700 1 -- 172.21.15.113:0/3902060204 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4004160 con 0x7febf41069e0 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf159a700 1 -- 172.21.15.113:0/3902060204 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7febe4004300 con 0x7febf41069e0 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf159a700 1 -- 172.21.15.113:0/3902060204 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4011af0 con 0x7febf41069e0 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf8aa3700 1 -- 172.21.15.113:0/3902060204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 msgr2=0x7febf4106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf8aa3700 1 --2- 172.21.15.113:0/3902060204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf4106e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febe40098a0 tx=0x7febe4009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf8aa3700 1 -- 172.21.15.113:0/3902060204 shutdown_connections 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf8aa3700 1 --2- 172.21.15.113:0/3902060204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf4106e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.414+0000 7febf8aa3700 1 -- 172.21.15.113:0/3902060204 >> 172.21.15.113:0/3902060204 conn(0x7febf41023b0 msgr2=0x7febf41047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.415+0000 7febf8aa3700 1 -- 172.21.15.113:0/3902060204 shutdown_connections 2024-04-22T07:05:56.873 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.415+0000 7febf8aa3700 1 -- 172.21.15.113:0/3902060204 wait complete. 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.415+0000 7febf8aa3700 1 Processor -- start 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.415+0000 7febf8aa3700 1 -- start start 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf8aa3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf8aa3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7febf410cc40 con 0x7febf41069e0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf259c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41666/0 (socket says 172.21.15.113:41666) 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf259c700 1 -- 172.21.15.113:0/240054026 learned_addr learned my addr 172.21.15.113:0/240054026 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf259c700 1 -- 172.21.15.113:0/240054026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf419cc30 con 0x7febf41069e0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf259c700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf419c6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febe40117b0 tx=0x7febe4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe40039e0 con 0x7febf41069e0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7febe4003b80 con 0x7febf41069e0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.416+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf419cec0 con 0x7febf41069e0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.417+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4024780 con 0x7febf41069e0 2024-04-22T07:05:56.874 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.417+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf419fbb0 con 0x7febf41069e0 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.417+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7febe4022020 con 0x7febf41069e0 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.417+0000 7febe37fe700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.418+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7febe4054060 con 0x7febf41069e0 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.418+0000 7febf1d9b700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 msgr2=0x7febdc03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.418+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.418+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 --> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7febf40681b0 con 0x7febdc0396b0 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.619+0000 7febf1d9b700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 msgr2=0x7febdc03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:50.619+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:51.020+0000 7febf1d9b700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 msgr2=0x7febdc03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:51.020+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:51.821+0000 7febf1d9b700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 msgr2=0x7febdc03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:51.821+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:53.422+0000 7febf1d9b700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 msgr2=0x7febdc03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/1807162111 2024-04-22T07:05:56.875 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:53.422+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:54.992+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7febe40198e0 con 0x7febf41069e0 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:54.992+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 msgr2=0x7febdc03bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:54.992+0000 7febe37fe700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:55.993+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7febe4042f30 con 0x7febf41069e0 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:55.994+0000 7febe37fe700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7febdc03cf90 0x7febdc03f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:55.994+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7febe400af50 con 0x7febdc03cf90 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:55.995+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7febdc03cf90 0x7febdc03f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:55.995+0000 7febf1d9b700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7febdc03cf90 0x7febdc03f380 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febe8003a10 tx=0x7febe80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:55.996+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7febe400af50 con 0x7febdc03cf90 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.005+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7febf410cdc0 con 0x7febdc03cf90 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febe37fe700 1 -- 172.21.15.113:0/240054026 <== mgr.14150 v2:172.21.15.113:6800/610623186 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7febf410cdc0 con 0x7febdc03cf90 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7febdc03cf90 msgr2=0x7febdc03f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7febdc03cf90 0x7febdc03f380 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febe8003a10 tx=0x7febe80092b0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.876 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 msgr2=0x7febf419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf419c6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febe40117b0 tx=0x7febe4011b70 comp rx=0 tx=0).stop 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 shutdown_connections 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7febdc03cf90 0x7febdc03f380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf41069e0 0x7febf419c6f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 --2- 172.21.15.113:0/240054026 >> [v2:172.21.15.113:6800/1807162111,v1:172.21.15.113:6801/1807162111] conn(0x7febdc0396b0 0x7febdc03bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 >> 172.21.15.113:0/240054026 conn(0x7febf41023b0 msgr2=0x7febf4102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 shutdown_connections 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:56.006+0000 7febf8aa3700 1 -- 172.21.15.113:0/240054026 wait complete. 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:mgr epoch 9 is available 2024-04-22T07:05:56.877 INFO:teuthology.orchestra.run.smithi113.stdout:Generating a dashboard self-signed certificate... 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:55] ENGINE Bus STARTING 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:55] ENGINE Serving on https://172.21.15.113:7150 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:55] ENGINE Serving on http://172.21.15.113:8765 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: [22/Apr/2024:07:05:55] ENGINE Bus STARTED 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T07:05:57.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:57 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:05:58.195 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-22T07:05:58.195 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.303+0000 7fd440eda700 1 Processor -- start 2024-04-22T07:05:58.195 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.303+0000 7fd440eda700 1 -- start start 2024-04-22T07:05:58.195 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd440eda700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c108b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd440eda700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c109110 con 0x7fd43c108720 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd43a59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c108b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd43a59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c108b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58134/0 (socket says 172.21.15.113:58134) 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd43a59c700 1 -- 172.21.15.113:0/3892503048 learned_addr learned my addr 172.21.15.113:0/3892503048 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd43a59c700 1 -- 172.21.15.113:0/3892503048 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c109920 con 0x7fd43c108720 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.304+0000 7fd43a59c700 1 --2- 172.21.15.113:0/3892503048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c108b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0098a0 tx=0x7fd42c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff382ad184560bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd43959a700 1 -- 172.21.15.113:0/3892503048 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c004160 con 0x7fd43c108720 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd43959a700 1 -- 172.21.15.113:0/3892503048 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd42c004300 con 0x7fd43c108720 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd43959a700 1 -- 172.21.15.113:0/3892503048 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c011af0 con 0x7fd43c108720 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 -- 172.21.15.113:0/3892503048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 msgr2=0x7fd43c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 --2- 172.21.15.113:0/3892503048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c108b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0098a0 tx=0x7fd42c009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 -- 172.21.15.113:0/3892503048 shutdown_connections 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 --2- 172.21.15.113:0/3892503048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c108b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 -- 172.21.15.113:0/3892503048 >> 172.21.15.113:0/3892503048 conn(0x7fd43c103d20 msgr2=0x7fd43c106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 -- 172.21.15.113:0/3892503048 shutdown_connections 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.305+0000 7fd440eda700 1 -- 172.21.15.113:0/3892503048 wait complete. 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.306+0000 7fd440eda700 1 Processor -- start 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.306+0000 7fd440eda700 1 -- start start 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.306+0000 7fd440eda700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.306+0000 7fd440eda700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c10ccd0 con 0x7fd43c108720 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.306+0000 7fd43a59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd43a59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58140/0 (socket says 172.21.15.113:58140) 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd43a59c700 1 -- 172.21.15.113:0/1939406690 learned_addr learned my addr 172.21.15.113:0/1939406690 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:58.197 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd43a59c700 1 -- 172.21.15.113:0/1939406690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c19cc20 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd43a59c700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c19c6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009520 tx=0x7fd42c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c0036a0 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd42c01a430 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c19ceb0 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42c019550 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.307+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c19fba0 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.308+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fd42c019850 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.308+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c0681b0 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.308+0000 7fd42b7fe700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd4240394a0 0x7fd42403b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.308+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd42c01a900 con 0x7fd43c108720 2024-04-22T07:05:58.198 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.308+0000 7fd439d9b700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd4240394a0 0x7fd42403b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.309+0000 7fd439d9b700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd4240394a0 0x7fd42403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd430006fd0 tx=0x7fd430006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.313+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd42c04d2a0 con 0x7fd43c108720 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.523+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd43c104f60 con 0x7fd4240394a0 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.596+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd42c052070 con 0x7fd43c108720 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.639+0000 7fd42b7fe700 1 -- 172.21.15.113:0/1939406690 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd43c104f60 con 0x7fd4240394a0 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd4240394a0 msgr2=0x7fd42403b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd4240394a0 0x7fd42403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd430006fd0 tx=0x7fd430006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 msgr2=0x7fd43c19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c19c6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009520 tx=0x7fd42c0045f0 comp rx=0 tx=0).stop 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 shutdown_connections 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd4240394a0 0x7fd42403b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 --2- 172.21.15.113:0/1939406690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd43c108720 0x7fd43c19c6e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 >> 172.21.15.113:0/1939406690 conn(0x7fd43c103d20 msgr2=0x7fd43c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:58.199 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 shutdown_connections 2024-04-22T07:05:58.200 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:57.641+0000 7fd440eda700 1 -- 172.21.15.113:0/1939406690 wait complete. 2024-04-22T07:05:58.200 INFO:teuthology.orchestra.run.smithi113.stdout:Creating initial admin user... 2024-04-22T07:05:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:58 smithi113 ceph-mon[27933]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:58 smithi113 ceph-mon[27933]: mgrmap e12: x(active, since 2s) 2024-04-22T07:05:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:05:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$r.ToXzzOfeGig.VBgfKOb.3cAPg6XUNCNo1iB5Fy3C61rBUbKMQlS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713769559, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa844ab4700 1 Processor -- start 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa844ab4700 1 -- start start 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa844ab4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa8401090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa844ab4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401096c0 con 0x7fa840108cd0 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa83e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa8401090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:59.600 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa83e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa8401090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58156/0 (socket says 172.21.15.113:58156) 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.614+0000 7fa83e59c700 1 -- 172.21.15.113:0/3934031139 learned_addr learned my addr 172.21.15.113:0/3934031139 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa83e59c700 1 -- 172.21.15.113:0/3934031139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8401067d0 con 0x7fa840108cd0 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa83e59c700 1 --2- 172.21.15.113:0/3934031139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa8401090f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8300098a0 tx=0x7fa830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cb88c8e1d779b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa83d59a700 1 -- 172.21.15.113:0/3934031139 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa830004160 con 0x7fa840108cd0 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa83d59a700 1 -- 172.21.15.113:0/3934031139 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa830004300 con 0x7fa840108cd0 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa83d59a700 1 -- 172.21.15.113:0/3934031139 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa830011af0 con 0x7fa840108cd0 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa844ab4700 1 -- 172.21.15.113:0/3934031139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 msgr2=0x7fa8401090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.615+0000 7fa844ab4700 1 --2- 172.21.15.113:0/3934031139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa8401090f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8300098a0 tx=0x7fa830009bb0 comp rx=0 tx=0).stop 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 -- 172.21.15.113:0/3934031139 shutdown_connections 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 --2- 172.21.15.113:0/3934031139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa8401090f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 -- 172.21.15.113:0/3934031139 >> 172.21.15.113:0/3934031139 conn(0x7fa840102370 msgr2=0x7fa8401047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 -- 172.21.15.113:0/3934031139 shutdown_connections 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 -- 172.21.15.113:0/3934031139 wait complete. 2024-04-22T07:05:59.601 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 Processor -- start 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.616+0000 7fa844ab4700 1 -- start start 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa844ab4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa84019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa844ab4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa840107890 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa83e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa84019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa83e59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa84019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58160/0 (socket says 172.21.15.113:58160) 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa83e59c700 1 -- 172.21.15.113:0/2763741151 learned_addr learned my addr 172.21.15.113:0/2763741151 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa83e59c700 1 -- 172.21.15.113:0/2763741151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84019ee00 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa83e59c700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa84019e8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa8300117b0 tx=0x7fa8300045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8300036a0 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa83001a430 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84019f090 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa830019530 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.617+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401a1d80 con 0x7fa840108cd0 2024-04-22T07:05:59.602 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.618+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa8300197e0 con 0x7fa840108cd0 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.618+0000 7fa82f7fe700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa828039610 0x7fa82803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.618+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8400681b0 con 0x7fa840108cd0 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.618+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa83004d8e0 con 0x7fa840108cd0 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.618+0000 7fa83dd9b700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa828039610 0x7fa82803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.619+0000 7fa83dd9b700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa828039610 0x7fa82803bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa834006fd0 tx=0x7fa834006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.623+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa83000bbf0 con 0x7fa840108cd0 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:58.848+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa84019f570 con 0x7fa828039610 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.077+0000 7fa82f7fe700 1 -- 172.21.15.113:0/2763741151 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa84019f570 con 0x7fa828039610 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.079+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa828039610 msgr2=0x7fa82803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.079+0000 7fa844ab4700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa828039610 0x7fa82803bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa834006fd0 tx=0x7fa834006e40 comp rx=0 tx=0).stop 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.079+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 msgr2=0x7fa84019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.079+0000 7fa844ab4700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa84019e8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa8300117b0 tx=0x7fa8300045f0 comp rx=0 tx=0).stop 2024-04-22T07:05:59.603 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.080+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 shutdown_connections 2024-04-22T07:05:59.604 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.080+0000 7fa844ab4700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa828039610 0x7fa82803bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:59.604 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.080+0000 7fa844ab4700 1 --2- 172.21.15.113:0/2763741151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa840108cd0 0x7fa84019e8c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:59.604 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.080+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 >> 172.21.15.113:0/2763741151 conn(0x7fa840102370 msgr2=0x7fa840194f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:59.604 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.080+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 shutdown_connections 2024-04-22T07:05:59.604 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.080+0000 7fa844ab4700 1 -- 172.21.15.113:0/2763741151 wait complete. 2024-04-22T07:05:59.604 INFO:teuthology.orchestra.run.smithi113.stdout:Fetching dashboard port number... 2024-04-22T07:05:59.970 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:59 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:05:59.970 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:59 smithi113 ceph-mon[27933]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:05:59.970 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:05:59 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:00.859 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stdout 8443 2024-04-22T07:06:00.859 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f2601b7c700 1 Processor -- start 2024-04-22T07:06:00.859 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f2601b7c700 1 -- start start 2024-04-22T07:06:00.859 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f2601b7c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc10b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:00.859 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f2601b7c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc07cde0 con 0x7f25fc109240 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc10b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc10b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58168/0 (socket says 172.21.15.113:58168) 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.989+0000 7f25fb7fe700 1 -- 172.21.15.113:0/2417794625 learned_addr learned my addr 172.21.15.113:0/2417794625 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f25fb7fe700 1 -- 172.21.15.113:0/2417794625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc10bb70 con 0x7f25fc109240 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f25fb7fe700 1 --2- 172.21.15.113:0/2417794625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc10b630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098a0 tx=0x7f25ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b5a3fd8f6ed5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f25fa7fc700 1 -- 172.21.15.113:0/2417794625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25ec004160 con 0x7f25fc109240 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f25fa7fc700 1 -- 172.21.15.113:0/2417794625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f25ec004300 con 0x7f25fc109240 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f25fa7fc700 1 -- 172.21.15.113:0/2417794625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25ec011af0 con 0x7f25fc109240 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f2601b7c700 1 -- 172.21.15.113:0/2417794625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 msgr2=0x7f25fc10b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f2601b7c700 1 --2- 172.21.15.113:0/2417794625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc10b630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098a0 tx=0x7f25ec009bb0 comp rx=0 tx=0).stop 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f2601b7c700 1 -- 172.21.15.113:0/2417794625 shutdown_connections 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f2601b7c700 1 --2- 172.21.15.113:0/2417794625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc10b630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:00.860 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.990+0000 7f2601b7c700 1 -- 172.21.15.113:0/2417794625 >> 172.21.15.113:0/2417794625 conn(0x7f25fc102c10 msgr2=0x7f25fc105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.991+0000 7f2601b7c700 1 -- 172.21.15.113:0/2417794625 shutdown_connections 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.991+0000 7f2601b7c700 1 -- 172.21.15.113:0/2417794625 wait complete. 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.991+0000 7f2601b7c700 1 Processor -- start 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.991+0000 7f2601b7c700 1 -- start start 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.991+0000 7f2601b7c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f2601b7c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10cb70 con 0x7f25fc109240 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25fb7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58170/0 (socket says 172.21.15.113:58170) 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25fb7fe700 1 -- 172.21.15.113:0/2915699919 learned_addr learned my addr 172.21.15.113:0/2915699919 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25fb7fe700 1 -- 172.21.15.113:0/2915699919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc19cbf0 con 0x7f25fc109240 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25fb7fe700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc19c6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0117b0 tx=0x7f25ec0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25ec0036a0 con 0x7f25fc109240 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f25ec01a430 con 0x7f25fc109240 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc19ce80 con 0x7f25fc109240 2024-04-22T07:06:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25ec019530 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.992+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc19fb70 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.993+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f25ec0197e0 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.993+0000 7f25f8ff9700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f25e4039610 0x7f25e403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.993+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0681b0 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.993+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f25ec04db40 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.994+0000 7f25faffd700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f25e4039610 0x7f25e403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.994+0000 7f25faffd700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f25e4039610 0x7f25e403bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006fd0 tx=0x7f25f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:05:59.999+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25ec00fda0 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.275+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f25fc19fd40 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.276+0000 7f25f8ff9700 1 -- 172.21.15.113:0/2915699919 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f25ec033030 con 0x7f25fc109240 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f25e4039610 msgr2=0x7f25e403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f25e4039610 0x7f25e403bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006fd0 tx=0x7f25f0006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:00.862 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 msgr2=0x7f25fc19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc19c6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0117b0 tx=0x7f25ec0045f0 comp rx=0 tx=0).stop 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 shutdown_connections 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f25e4039610 0x7f25e403bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 --2- 172.21.15.113:0/2915699919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25fc109240 0x7f25fc19c6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 >> 172.21.15.113:0/2915699919 conn(0x7f25fc102c10 msgr2=0x7f25fc1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 shutdown_connections 2024-04-22T07:06:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:00.280+0000 7f2601b7c700 1 -- 172.21.15.113:0/2915699919 wait complete. 2024-04-22T07:06:00.865 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T07:06:00.865 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout disabled 2024-04-22T07:06:00.872 INFO:teuthology.orchestra.run.smithi113.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T07:06:00.872 INFO:teuthology.orchestra.run.smithi113.stdout:systemctl: stdout inactive 2024-04-22T07:06:00.873 INFO:teuthology.orchestra.run.smithi113.stdout:firewalld.service is not enabled 2024-04-22T07:06:00.873 INFO:teuthology.orchestra.run.smithi113.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-22T07:06:00.874 INFO:teuthology.orchestra.run.smithi113.stdout:Ceph Dashboard is now available at: 2024-04-22T07:06:00.874 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:00.874 INFO:teuthology.orchestra.run.smithi113.stdout: URL: https://smithi113:8443/ 2024-04-22T07:06:00.874 INFO:teuthology.orchestra.run.smithi113.stdout: User: admin 2024-04-22T07:06:00.874 INFO:teuthology.orchestra.run.smithi113.stdout: Password: 0oqje4zchj 2024-04-22T07:06:00.874 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:00.875 INFO:teuthology.orchestra.run.smithi113.stdout:Saving cluster configuration to /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config directory 2024-04-22T07:06:00.875 INFO:teuthology.orchestra.run.smithi113.stdout:Enabling autotune for osd_memory_target 2024-04-22T07:06:00.991 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2915699919' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-22T07:06:02.779 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f050102b700 1 Processor -- start 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f050102b700 1 -- start start 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f050102b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc108c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f050102b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc109220 con 0x7f04fc108830 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f04fad9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc108c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f04fad9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc108c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58178/0 (socket says 172.21.15.113:58178) 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.788+0000 7f04fad9d700 1 -- 172.21.15.113:0/985843428 learned_addr learned my addr 172.21.15.113:0/985843428 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f04fad9d700 1 -- 172.21.15.113:0/985843428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc1099e0 con 0x7f04fc108830 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f04fad9d700 1 --2- 172.21.15.113:0/985843428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc108c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0098a0 tx=0x7f04ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3431d79d17be6338 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f04fa59c700 1 -- 172.21.15.113:0/985843428 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04ec004160 con 0x7f04fc108830 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f04fa59c700 1 -- 172.21.15.113:0/985843428 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f04ec004300 con 0x7f04fc108830 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f04fa59c700 1 -- 172.21.15.113:0/985843428 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04ec011af0 con 0x7f04fc108830 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f050102b700 1 -- 172.21.15.113:0/985843428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 msgr2=0x7f04fc108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:02.780 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f050102b700 1 --2- 172.21.15.113:0/985843428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc108c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0098a0 tx=0x7f04ec009bb0 comp rx=0 tx=0).stop 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f050102b700 1 -- 172.21.15.113:0/985843428 shutdown_connections 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f050102b700 1 --2- 172.21.15.113:0/985843428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc108c50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.789+0000 7f050102b700 1 -- 172.21.15.113:0/985843428 >> 172.21.15.113:0/985843428 conn(0x7f04fc103df0 msgr2=0x7f04fc106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.790+0000 7f050102b700 1 -- 172.21.15.113:0/985843428 shutdown_connections 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.790+0000 7f050102b700 1 -- 172.21.15.113:0/985843428 wait complete. 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.790+0000 7f050102b700 1 Processor -- start 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.790+0000 7f050102b700 1 -- start start 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f050102b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc07eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f050102b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10cd90 con 0x7f04fc108830 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04fad9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc07eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04fad9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc07eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58192/0 (socket says 172.21.15.113:58192) 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04fad9d700 1 -- 172.21.15.113:0/3824625590 learned_addr learned my addr 172.21.15.113:0/3824625590 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04fad9d700 1 -- 172.21.15.113:0/3824625590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc07f090 con 0x7f04fc108830 2024-04-22T07:06:02.781 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04fad9d700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc07eb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009520 tx=0x7f04ec0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04ec0036a0 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f04ec01a430 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc07d280 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04ec019550 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.791+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc07d7c0 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.792+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f04ec019850 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.792+0000 7f04f8d99700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f04e8039650 0x7f04e803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.792+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc0681b0 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.792+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f04ec04d990 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.792+0000 7f04f259c700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f04e8039650 0x7f04e803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.793+0000 7f04f259c700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f04e8039650 0x7f04e803bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04e4006fd0 tx=0x7f04e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:01.797+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04ec00f2e0 con 0x7f04fc108830 2024-04-22T07:06:02.782 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.000+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f04fc07e0b0 con 0x7f04fc108830 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.001+0000 7f04f8d99700 1 -- 172.21.15.113:0/3824625590 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f04ec032030 con 0x7f04fc108830 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f04e8039650 msgr2=0x7f04e803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f04e8039650 0x7f04e803bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04e4006fd0 tx=0x7f04e4006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 msgr2=0x7f04fc07eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc07eb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009520 tx=0x7f04ec0045f0 comp rx=0 tx=0).stop 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 shutdown_connections 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f04e8039650 0x7f04e803bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 --2- 172.21.15.113:0/3824625590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04fc108830 0x7f04fc07eb50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 >> 172.21.15.113:0/3824625590 conn(0x7f04fc103df0 msgr2=0x7f04fc1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 shutdown_connections 2024-04-22T07:06:02.783 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:02.003+0000 7f050102b700 1 -- 172.21.15.113:0/3824625590 wait complete. 2024-04-22T07:06:03.001 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:02 smithi113 ceph-mon[27933]: mgrmap e13: x(active, since 6s) 2024-04-22T07:06:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b9173700 1 Processor -- start 2024-04-22T07:06:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b9173700 1 -- start start 2024-04-22T07:06:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b9173700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b4107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b9173700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4108000 con 0x7f27b4107610 2024-04-22T07:06:04.087 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b4107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b4107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58194/0 (socket says 172.21.15.113:58194) 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.232+0000 7f27b2d9d700 1 -- 172.21.15.113:0/2596234711 learned_addr learned my addr 172.21.15.113:0/2596234711 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.233+0000 7f27b2d9d700 1 -- 172.21.15.113:0/2596234711 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b4108810 con 0x7f27b4107610 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.233+0000 7f27b2d9d700 1 --2- 172.21.15.113:0/2596234711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b4107a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27a40098a0 tx=0x7f27a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e595beda01d59a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.233+0000 7f27b259c700 1 -- 172.21.15.113:0/2596234711 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a4004160 con 0x7f27b4107610 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.233+0000 7f27b259c700 1 -- 172.21.15.113:0/2596234711 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f27a4004300 con 0x7f27b4107610 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.233+0000 7f27b259c700 1 -- 172.21.15.113:0/2596234711 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a4011af0 con 0x7f27b4107610 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 -- 172.21.15.113:0/2596234711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 msgr2=0x7f27b4107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 --2- 172.21.15.113:0/2596234711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b4107a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27a40098a0 tx=0x7f27a4009bb0 comp rx=0 tx=0).stop 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 -- 172.21.15.113:0/2596234711 shutdown_connections 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 --2- 172.21.15.113:0/2596234711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b4107a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 -- 172.21.15.113:0/2596234711 >> 172.21.15.113:0/2596234711 conn(0x7f27b4102c10 msgr2=0x7f27b4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 -- 172.21.15.113:0/2596234711 shutdown_connections 2024-04-22T07:06:04.088 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 -- 172.21.15.113:0/2596234711 wait complete. 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.234+0000 7f27b9173700 1 Processor -- start 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b9173700 1 -- start start 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b9173700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b407ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b9173700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f27b410bbc0 con 0x7f27b4107610 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b407ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b407ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58208/0 (socket says 172.21.15.113:58208) 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b2d9d700 1 -- 172.21.15.113:0/21145674 learned_addr learned my addr 172.21.15.113:0/21145674 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b2d9d700 1 -- 172.21.15.113:0/21145674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b419f350 con 0x7f27b4107610 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b2d9d700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b407ca50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009520 tx=0x7f27a4003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a40036a0 con 0x7f27b4107610 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f27a401a430 con 0x7f27b4107610 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b419f520 con 0x7f27b4107610 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27a40193c0 con 0x7f27b4107610 2024-04-22T07:06:04.089 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.235+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b419f940 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.236+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f27a4019670 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.236+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b40681b0 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.236+0000 7f27b0d99700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f27a0039650 0x7f27a003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.237+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27a404da40 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.237+0000 7f27aa59c700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f27a0039650 0x7f27a003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.237+0000 7f27aa59c700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f27a0039650 0x7f27a003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f279c006fd0 tx=0x7f279c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.242+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27a4020240 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.552+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f27b419fd50 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.555+0000 7f27b0d99700 1 -- 172.21.15.113:0/21145674 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f27a404de20 con 0x7f27b4107610 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f27a0039650 msgr2=0x7f27a003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f27a0039650 0x7f27a003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f279c006fd0 tx=0x7f279c006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 msgr2=0x7f27b407ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:04.090 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b407ca50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009520 tx=0x7f27a4003eb0 comp rx=0 tx=0).stop 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 shutdown_connections 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f27a0039650 0x7f27a003bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 --2- 172.21.15.113:0/21145674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27b4107610 0x7f27b407ca50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 >> 172.21.15.113:0/21145674 conn(0x7f27b4102c10 msgr2=0x7f27b41036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 shutdown_connections 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr 2024-04-22T07:06:03.557+0000 7f27b9173700 1 -- 172.21.15.113:0/21145674 wait complete. 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout:Or, if you are only running a single cluster on this host: 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-22T07:06:04.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: ceph telemetry on 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:For more information see: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:04.092 INFO:teuthology.orchestra.run.smithi113.stdout:Bootstrap complete. 2024-04-22T07:06:04.126 INFO:tasks.cephadm:Fetching config... 2024-04-22T07:06:04.126 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:04.126 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-22T07:06:04.142 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-22T07:06:04.142 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:04.142 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-22T07:06:04.197 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-22T07:06:04.226 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:04.226 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-22T07:06:04.260 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-22T07:06:04.260 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:04.260 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-22T07:06:04.316 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-22T07:06:04.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T07:06:04.388 INFO:teuthology.orchestra.run.smithi113.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:04.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T07:06:04.438 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:04.450 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T07:06:04.488 INFO:teuthology.orchestra.run.smithi154.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGnGbmeNSdBIx2hte8nZEGYQyO1olrIY1/I/YOflOwtpR8bwNHPJb+PYkvCwokgI8oZJUhuXJlbaFBDEtbGMWwM1w2xDudZ2UFUUf0yUSCZLNnlQPk2MU1TrMn8Vb9Pxh+GS9RIrWYRdiFXqimhXp29pyMVo0SlHSsLXfyeAPlv88V02Vus3NcgTgq9oO6nu4O+MvSDxU/lEbDrQGsFr6g/MHTecd+DJf71PjMvGVFzTTvsY0dkQDhivQb1hGEFlP6v14MOKMRI0CPvHDXkqI3VpJQTV8LiIPB7MYaCouz1SToNlk6TLsuMnBsNalNDdwZw94qfbeCKiQOqjActE7wGvqlaWJnc0isN7LgsjcsDO1w7IoK6IzJoQqNH2eduY7xH4TuSw96u5MWlM4mkltLYbRvkQzE6hU/HqIaGaV/mgeWiN2+Y8kBMndb/V3R22GYu/sM9V2jZhkEWbOCWtKxrYAIH0zZXHO3Kcqb0qTV6OY5fsDGQGUGM36yujHfEuE= ceph-81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:04.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-22T07:06:04.640 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/21145674' entity='client.admin' 2024-04-22T07:06:04.740 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 -- 172.21.15.113:0/1832317233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30841024b0 msgr2=0x7f30841028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 --2- 172.21.15.113:0/1832317233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30841024b0 0x7f30841028d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3074009910 tx=0x7f3074009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 -- 172.21.15.113:0/1832317233 shutdown_connections 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 --2- 172.21.15.113:0/1832317233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30841024b0 0x7f30841028d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 -- 172.21.15.113:0/1832317233 >> 172.21.15.113:0/1832317233 conn(0x7f30840fda70 msgr2=0x7f30840ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 -- 172.21.15.113:0/1832317233 shutdown_connections 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.960+0000 7f3089c9e700 1 -- 172.21.15.113:0/1832317233 wait complete. 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f3089c9e700 1 Processor -- start 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f3089c9e700 1 -- start start 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f3089c9e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 0x7f3084074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:06.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f3089c9e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3084108530 con 0x7f30840766c0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f30837fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 0x7f3084074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f30837fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 0x7f3084074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50114/0 (socket says 172.21.15.113:50114) 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.961+0000 7f30837fe700 1 -- 172.21.15.113:0/3491813263 learned_addr learned my addr 172.21.15.113:0/3491813263 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f30837fe700 1 -- 172.21.15.113:0/3491813263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084076ae0 con 0x7f30840766c0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f30837fe700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 0x7f3084074e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3074000c00 tx=0x7f3074009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3074027070 con 0x7f30840766c0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3074022470 con 0x7f30840766c0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3084075370 con 0x7f30840766c0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f307400f5d0 con 0x7f30840766c0 2024-04-22T07:06:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.962+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3084075850 con 0x7f30840766c0 2024-04-22T07:06:06.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.963+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f307402a070 con 0x7f30840766c0 2024-04-22T07:06:06.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.963+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3084076cb0 con 0x7f30840766c0 2024-04-22T07:06:06.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.963+0000 7f30817fa700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3070039650 0x7f307003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:06.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.963+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f307401e030 con 0x7f30840766c0 2024-04-22T07:06:06.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.963+0000 7f307affd700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3070039650 0x7f307003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:06.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.963+0000 7f307affd700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3070039650 0x7f307003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:06.968+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3074013070 con 0x7f30840766c0 2024-04-22T07:06:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.163+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3084075b90 con 0x7f30840766c0 2024-04-22T07:06:07.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.166+0000 7f30817fa700 1 -- 172.21.15.113:0/3491813263 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f30740292c0 con 0x7f30840766c0 2024-04-22T07:06:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3070039650 msgr2=0x7f307003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3070039650 0x7f307003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 msgr2=0x7f3084074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 0x7f3084074e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3074000c00 tx=0x7f3074009e60 comp rx=0 tx=0).stop 2024-04-22T07:06:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 shutdown_connections 2024-04-22T07:06:07.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3070039650 0x7f307003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:07.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 --2- 172.21.15.113:0/3491813263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30840766c0 0x7f3084074e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:07.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 >> 172.21.15.113:0/3491813263 conn(0x7f30840fda70 msgr2=0x7f30840fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:07.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 shutdown_connections 2024-04-22T07:06:07.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:07.168+0000 7f3089c9e700 1 -- 172.21.15.113:0/3491813263 wait complete. 2024-04-22T07:06:07.754 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-22T07:06:07.754 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-22T07:06:07.987 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi113", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:06:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:08.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:08.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:08.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3491813263' entity='client.admin' 2024-04-22T07:06:08.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:08.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 -- 172.21.15.113:0/4226136430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 msgr2=0x7fe00c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 --2- 172.21.15.113:0/4226136430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c101800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009910 tx=0x7fdffc009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 -- 172.21.15.113:0/4226136430 shutdown_connections 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 --2- 172.21.15.113:0/4226136430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c101800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 -- 172.21.15.113:0/4226136430 >> 172.21.15.113:0/4226136430 conn(0x7fe00c0fc9a0 msgr2=0x7fe00c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 -- 172.21.15.113:0/4226136430 shutdown_connections 2024-04-22T07:06:09.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.425+0000 7fe0116c1700 1 -- 172.21.15.113:0/4226136430 wait complete. 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe0116c1700 1 Processor -- start 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe0116c1700 1 -- start start 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe0116c1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe0116c1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c107460 con 0x7fe00c1013e0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe00affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe00affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50130/0 (socket says 172.21.15.113:50130) 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe00affd700 1 -- 172.21.15.113:0/1638645182 learned_addr learned my addr 172.21.15.113:0/1638645182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe00affd700 1 -- 172.21.15.113:0/1638645182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c198090 con 0x7fe00c1013e0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.426+0000 7fe00affd700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c197b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009ec0 tx=0x7fdffc004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.427+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdffc027070 con 0x7fe00c1013e0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.427+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdffc022470 con 0x7fe00c1013e0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.427+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdffc00f5f0 con 0x7fe00c1013e0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.427+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c198320 con 0x7fe00c1013e0 2024-04-22T07:06:09.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.427+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c19b010 con 0x7fe00c1013e0 2024-04-22T07:06:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.428+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fdffc02a070 con 0x7fe00c1013e0 2024-04-22T07:06:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.428+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c107be0 con 0x7fe00c1013e0 2024-04-22T07:06:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.428+0000 7fdff3fff700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff4039650 0x7fdff403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.428+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdffc01e030 con 0x7fe00c1013e0 2024-04-22T07:06:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.428+0000 7fe00a7fc700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff4039650 0x7fdff403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:09.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.428+0000 7fe00a7fc700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff4039650 0x7fdff403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe000006fd0 tx=0x7fe000006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:09.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.433+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdffc013070 con 0x7fe00c1013e0 2024-04-22T07:06:09.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.634+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe00c0fe050 con 0x7fdff4039650 2024-04-22T07:06:09.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.638+0000 7fdff3fff700 1 -- 172.21.15.113:0/1638645182 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe00c0fe050 con 0x7fdff4039650 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff4039650 msgr2=0x7fdff403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff4039650 0x7fdff403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe000006fd0 tx=0x7fe000006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 msgr2=0x7fe00c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c197b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009ec0 tx=0x7fdffc004690 comp rx=0 tx=0).stop 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 shutdown_connections 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff4039650 0x7fdff403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 --2- 172.21.15.113:0/1638645182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c1013e0 0x7fe00c197b50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 >> 172.21.15.113:0/1638645182 conn(0x7fe00c0fc9a0 msgr2=0x7fe00c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 shutdown_connections 2024-04-22T07:06:09.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:09.640+0000 7fe0116c1700 1 -- 172.21.15.113:0/1638645182 wait complete. 2024-04-22T07:06:10.398 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2024-04-22T07:06:10.398 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:06:10.398 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2024-04-22T07:06:10.418 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:06:10.418 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:10.476 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2024-04-22T07:06:10.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch host add smithi133 2024-04-22T07:06:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:06:10.921 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:10.922 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:10.922 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:10.922 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:11.895 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:11 smithi113 ceph-mon[27933]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:06:12.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.172+0000 7f8d7ee70700 1 -- 172.21.15.113:0/2092798380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 msgr2=0x7f8d78101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.172+0000 7f8d7ee70700 1 --2- 172.21.15.113:0/2092798380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78101980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009910 tx=0x7f8d6c009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.172+0000 7f8d7ee70700 1 -- 172.21.15.113:0/2092798380 shutdown_connections 2024-04-22T07:06:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.172+0000 7f8d7ee70700 1 --2- 172.21.15.113:0/2092798380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78101980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.172+0000 7f8d7ee70700 1 -- 172.21.15.113:0/2092798380 >> 172.21.15.113:0/2092798380 conn(0x7f8d780fce90 msgr2=0x7f8d780ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.173+0000 7f8d7ee70700 1 -- 172.21.15.113:0/2092798380 shutdown_connections 2024-04-22T07:06:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.173+0000 7f8d7ee70700 1 -- 172.21.15.113:0/2092798380 wait complete. 2024-04-22T07:06:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7ee70700 1 Processor -- start 2024-04-22T07:06:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7ee70700 1 -- start start 2024-04-22T07:06:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7ee70700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7ee70700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810b960 con 0x7f8d781015a0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7cc0c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7cc0c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50140/0 (socket says 172.21.15.113:50140) 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7cc0c700 1 -- 172.21.15.113:0/4026668741 learned_addr learned my addr 172.21.15.113:0/4026668741 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7cc0c700 1 -- 172.21.15.113:0/4026668741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d78199260 con 0x7f8d781015a0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.174+0000 7f8d7cc0c700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78198d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005780 tx=0x7f8d6c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.175+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d6c026070 con 0x7f8d781015a0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.175+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8d6c004370 con 0x7f8d781015a0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.175+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d6c003a30 con 0x7f8d781015a0 2024-04-22T07:06:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.175+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d78199550 con 0x7f8d781015a0 2024-04-22T07:06:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.175+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d78199a10 con 0x7f8d781015a0 2024-04-22T07:06:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.176+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f8d6c029070 con 0x7f8d781015a0 2024-04-22T07:06:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.176+0000 7f8d75ffb700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8d68039650 0x7f8d6803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.176+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d78102a80 con 0x7f8d781015a0 2024-04-22T07:06:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.176+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8d6c01e070 con 0x7f8d781015a0 2024-04-22T07:06:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.176+0000 7f8d77fff700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8d68039650 0x7f8d6803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.176+0000 7f8d77fff700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8d68039650 0x7f8d6803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d64006fd0 tx=0x7f8d64006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.181+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d6c013070 con 0x7f8d781015a0 2024-04-22T07:06:12.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:12.381+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}) -- 0x7f8d7806aed0 con 0x7f8d68039650 2024-04-22T07:06:13.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:13 smithi113 ceph-mon[27933]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:06:13.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:13 smithi113 ceph-mon[27933]: Deploying cephadm binary to smithi133 2024-04-22T07:06:16.284 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:15 smithi113 ceph-mon[27933]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.562+0000 7f8d75ffb700 1 -- 172.21.15.113:0/4026668741 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8d7806aed0 con 0x7f8d68039650 2024-04-22T07:06:16.560 INFO:teuthology.orchestra.run.smithi113.stdout:Added host 'smithi133' with addr '172.21.15.133' 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.564+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8d68039650 msgr2=0x7f8d6803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.564+0000 7f8d7ee70700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8d68039650 0x7f8d6803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d64006fd0 tx=0x7f8d64006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.564+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 msgr2=0x7f8d78198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78198d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005780 tx=0x7f8d6c0057b0 comp rx=0 tx=0).stop 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 shutdown_connections 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8d68039650 0x7f8d6803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 --2- 172.21.15.113:0/4026668741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d781015a0 0x7f8d78198d20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 >> 172.21.15.113:0/4026668741 conn(0x7f8d780fce90 msgr2=0x7f8d78102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 shutdown_connections 2024-04-22T07:06:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:16.565+0000 7f8d7ee70700 1 -- 172.21.15.113:0/4026668741 wait complete. 2024-04-22T07:06:17.242 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T07:06:17.476 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:17.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:17.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:17 smithi113 ceph-mon[27933]: Added host smithi133 2024-04-22T07:06:17.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:17.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:17 smithi113 ceph-mon[27933]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:17.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:18.640 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 -- 172.21.15.113:0/272859704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 msgr2=0x7fd180101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 --2- 172.21.15.113:0/272859704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180101990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd174009910 tx=0x7fd174009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 -- 172.21.15.113:0/272859704 shutdown_connections 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 --2- 172.21.15.113:0/272859704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180101990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 -- 172.21.15.113:0/272859704 >> 172.21.15.113:0/272859704 conn(0x7fd1800fce80 msgr2=0x7fd1800ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 -- 172.21.15.113:0/272859704 shutdown_connections 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.850+0000 7fd18671d700 1 -- 172.21.15.113:0/272859704 wait complete. 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd18671d700 1 Processor -- start 2024-04-22T07:06:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd18671d700 1 -- start start 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd18671d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd18671d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010b970 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33594/0 (socket says 172.21.15.113:33594) 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.851+0000 7fd17ffff700 1 -- 172.21.15.113:0/2096989621 learned_addr learned my addr 172.21.15.113:0/2096989621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd17ffff700 1 -- 172.21.15.113:0/2096989621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd180199300 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd17ffff700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180198dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd174006010 tx=0x7fd174009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd174027070 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801995f0 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd17400be30 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17400f460 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.852+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd180199ab0 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.853+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd17402a070 con 0x7fd1801015b0 2024-04-22T07:06:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.853+0000 7fd17d7fa700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd168039650 0x7fd16803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.853+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd180102a90 con 0x7fd1801015b0 2024-04-22T07:06:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.853+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd17401e030 con 0x7fd1801015b0 2024-04-22T07:06:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.853+0000 7fd17f7fe700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd168039650 0x7fd16803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.853+0000 7fd17f7fe700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd168039650 0x7fd16803bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:18.858+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd174013070 con 0x7fd1801015b0 2024-04-22T07:06:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.060+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1800640a0 con 0x7fd168039650 2024-04-22T07:06:19.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.062+0000 7fd17d7fa700 1 -- 172.21.15.113:0/2096989621 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fd1800640a0 con 0x7fd168039650 2024-04-22T07:06:19.061 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:19.061 INFO:teuthology.orchestra.run.smithi113.stdout:[{"addr": "172.21.15.113", "hostname": "smithi113", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2024-04-22T07:06:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd168039650 msgr2=0x7fd16803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd168039650 0x7fd16803bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 msgr2=0x7fd180198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180198dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd174006010 tx=0x7fd174009ec0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 shutdown_connections 2024-04-22T07:06:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd168039650 0x7fd16803bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 --2- 172.21.15.113:0/2096989621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1801015b0 0x7fd180198dc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 >> 172.21.15.113:0/2096989621 conn(0x7fd1800fce80 msgr2=0x7fd180102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 shutdown_connections 2024-04-22T07:06:19.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:19.065+0000 7fd18671d700 1 -- 172.21.15.113:0/2096989621 wait complete. 2024-04-22T07:06:19.638 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi154 2024-04-22T07:06:19.639 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:06:19.639 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.conf 2024-04-22T07:06:19.658 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:06:19.658 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:19.716 INFO:tasks.cephadm:Adding host smithi154 to orchestrator... 2024-04-22T07:06:19.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch host add smithi154 2024-04-22T07:06:19.953 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:20.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:20 smithi113 ceph-mon[27933]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:20.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:20 smithi113 ceph-mon[27933]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:06:20.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.499+0000 7ff80f88a700 1 -- 172.21.15.113:0/1924226635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 msgr2=0x7ff808108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.499+0000 7ff80f88a700 1 --2- 172.21.15.113:0/1924226635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808108890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff804009910 tx=0x7ff804009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:21.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.499+0000 7ff80f88a700 1 -- 172.21.15.113:0/1924226635 shutdown_connections 2024-04-22T07:06:21.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.499+0000 7ff80f88a700 1 --2- 172.21.15.113:0/1924226635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808108890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:21.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.499+0000 7ff80f88a700 1 -- 172.21.15.113:0/1924226635 >> 172.21.15.113:0/1924226635 conn(0x7ff8080fdf50 msgr2=0x7ff808100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:21.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80f88a700 1 -- 172.21.15.113:0/1924226635 shutdown_connections 2024-04-22T07:06:21.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80f88a700 1 -- 172.21.15.113:0/1924226635 wait complete. 2024-04-22T07:06:21.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80f88a700 1 Processor -- start 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80f88a700 1 -- start start 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80f88a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80f88a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff80810ca50 con 0x7ff8081084b0 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80d626700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80d626700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33604/0 (socket says 172.21.15.113:33604) 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.500+0000 7ff80d626700 1 -- 172.21.15.113:0/1338794727 learned_addr learned my addr 172.21.15.113:0/1338794727 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff80d626700 1 -- 172.21.15.113:0/1338794727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8081984c0 con 0x7ff8081084b0 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff80d626700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808197f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff804005780 tx=0x7ff8040057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff804026070 con 0x7ff8081084b0 2024-04-22T07:06:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8081987b0 con 0x7ff8081084b0 2024-04-22T07:06:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff804004370 con 0x7ff8081084b0 2024-04-22T07:06:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff804003a30 con 0x7ff8081084b0 2024-04-22T07:06:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.501+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80819c850 con 0x7ff8081084b0 2024-04-22T07:06:21.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.502+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff804029070 con 0x7ff8081084b0 2024-04-22T07:06:21.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.502+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff808103b60 con 0x7ff8081084b0 2024-04-22T07:06:21.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.502+0000 7ff7fa7fc700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7f4039650 0x7ff7f403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:21.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.502+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff80401e070 con 0x7ff8081084b0 2024-04-22T07:06:21.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.502+0000 7ff80ce25700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7f4039650 0x7ff7f403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:21.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.502+0000 7ff80ce25700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7f4039650 0x7ff7f403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:21.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.507+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff804013070 con 0x7ff8081084b0 2024-04-22T07:06:21.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:21.711+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}) -- 0x7ff8081992e0 con 0x7ff7f4039650 2024-04-22T07:06:22.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:22 smithi113 ceph-mon[27933]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:06:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:06:23.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:23.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:24.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:24 smithi113 ceph-mon[27933]: Deploying cephadm binary to smithi154 2024-04-22T07:06:24.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:24 smithi113 ceph-mon[27933]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-22T07:06:24.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:24 smithi113 ceph-mon[27933]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:06:24.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:24 smithi113 ceph-mon[27933]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:25.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.759+0000 7ff7fa7fc700 1 -- 172.21.15.113:0/1338794727 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff8081992e0 con 0x7ff7f4039650 2024-04-22T07:06:25.758 INFO:teuthology.orchestra.run.smithi113.stdout:Added host 'smithi154' with addr '172.21.15.154' 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7f4039650 msgr2=0x7ff7f403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7f4039650 0x7ff7f403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc006fd0 tx=0x7ff7fc006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 msgr2=0x7ff808197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808197f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff804005780 tx=0x7ff8040057b0 comp rx=0 tx=0).stop 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 shutdown_connections 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7f4039650 0x7ff7f403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 --2- 172.21.15.113:0/1338794727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8081084b0 0x7ff808197f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 >> 172.21.15.113:0/1338794727 conn(0x7ff8080fdf50 msgr2=0x7ff808109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 shutdown_connections 2024-04-22T07:06:25.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:25.761+0000 7ff80f88a700 1 -- 172.21.15.113:0/1338794727 wait complete. 2024-04-22T07:06:26.343 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T07:06:26.574 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:27.263 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:27 smithi113 ceph-mon[27933]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:06:27.263 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:27 smithi113 ceph-mon[27933]: Added host smithi154 2024-04-22T07:06:27.263 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:27.263 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:28.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 -- 172.21.15.113:0/3309642711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 msgr2=0x7fb8701039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:28.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 --2- 172.21.15.113:0/3309642711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb8701039f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009910 tx=0x7fb86c009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 -- 172.21.15.113:0/3309642711 shutdown_connections 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 --2- 172.21.15.113:0/3309642711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb8701039f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 -- 172.21.15.113:0/3309642711 >> 172.21.15.113:0/3309642711 conn(0x7fb8700fce80 msgr2=0x7fb8700ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 -- 172.21.15.113:0/3309642711 shutdown_connections 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.018+0000 7fb876ac6700 1 -- 172.21.15.113:0/3309642711 wait complete. 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb876ac6700 1 Processor -- start 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb876ac6700 1 -- start start 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb876ac6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb870197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:28.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb876ac6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb87010caf0 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb874862700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb870197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb874862700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb870197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37534/0 (socket says 172.21.15.113:37534) 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb874862700 1 -- 172.21.15.113:0/1468749316 learned_addr learned my addr 172.21.15.113:0/1468749316 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb874862700 1 -- 172.21.15.113:0/1468749316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8701984c0 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb874862700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb870197f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0057a0 tx=0x7fb86c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.019+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb86c026070 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.020+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb86c004370 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.020+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb870198750 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.020+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb86c003b90 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.020+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb870198bb0 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.020+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fb86c029070 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.021+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87006afe0 con 0x7fb870103610 2024-04-22T07:06:28.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.021+0000 7fb865ffb700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb8580396b0 0x7fb85803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:28.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.021+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb86c01e070 con 0x7fb870103610 2024-04-22T07:06:28.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.021+0000 7fb867fff700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb8580396b0 0x7fb85803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:28.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.021+0000 7fb867fff700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb8580396b0 0x7fb85803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb860006fd0 tx=0x7fb860006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:28.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.026+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb86c013070 con 0x7fb870103610 2024-04-22T07:06:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.229+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb870199350 con 0x7fb8580396b0 2024-04-22T07:06:28.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.230+0000 7fb865ffb700 1 -- 172.21.15.113:0/1468749316 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fb870199350 con 0x7fb8580396b0 2024-04-22T07:06:28.228 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:06:28.228 INFO:teuthology.orchestra.run.smithi113.stdout:[{"addr": "172.21.15.113", "hostname": "smithi113", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}] 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb8580396b0 msgr2=0x7fb85803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb8580396b0 0x7fb85803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb860006fd0 tx=0x7fb860006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 msgr2=0x7fb870197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb870197f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0057a0 tx=0x7fb86c005dc0 comp rx=0 tx=0).stop 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 shutdown_connections 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb8580396b0 0x7fb85803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 --2- 172.21.15.113:0/1468749316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb870103610 0x7fb870197f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 >> 172.21.15.113:0/1468749316 conn(0x7fb8700fce80 msgr2=0x7fb8701043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 shutdown_connections 2024-04-22T07:06:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:28.232+0000 7fb876ac6700 1 -- 172.21.15.113:0/1468749316 wait complete. 2024-04-22T07:06:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:28 smithi113 ceph-mon[27933]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:28.847 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-22T07:06:28.848 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-22T07:06:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:29.474 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:29 smithi113 ceph-mon[27933]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:06:29.474 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:29 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:29.474 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:29 smithi113 ceph-mon[27933]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:30.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.541+0000 7f283cc68700 1 -- 172.21.15.113:0/2000961460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 msgr2=0x7f2838101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.541+0000 7f283cc68700 1 --2- 172.21.15.113:0/2000961460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838101980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2828009910 tx=0x7f2828009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.541+0000 7f283cc68700 1 -- 172.21.15.113:0/2000961460 shutdown_connections 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.541+0000 7f283cc68700 1 --2- 172.21.15.113:0/2000961460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838101980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.542+0000 7f283cc68700 1 -- 172.21.15.113:0/2000961460 >> 172.21.15.113:0/2000961460 conn(0x7f28380fce90 msgr2=0x7f28380ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.542+0000 7f283cc68700 1 -- 172.21.15.113:0/2000961460 shutdown_connections 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.542+0000 7f283cc68700 1 -- 172.21.15.113:0/2000961460 wait complete. 2024-04-22T07:06:30.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.542+0000 7f283cc68700 1 Processor -- start 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.542+0000 7f283cc68700 1 -- start start 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.542+0000 7f283cc68700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283cc68700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f283810b960 con 0x7f28381015a0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283659c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283659c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37554/0 (socket says 172.21.15.113:37554) 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283659c700 1 -- 172.21.15.113:0/1618514243 learned_addr learned my addr 172.21.15.113:0/1618514243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283659c700 1 -- 172.21.15.113:0/1618514243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2838198410 con 0x7f28381015a0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283659c700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838197ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f2828009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2828027070 con 0x7f28381015a0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f282800be30 con 0x7f28381015a0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2838198700 con 0x7f28381015a0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282800f460 con 0x7f28381015a0 2024-04-22T07:06:30.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.543+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283819c7a0 con 0x7f28381015a0 2024-04-22T07:06:30.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.544+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f282802a070 con 0x7f28381015a0 2024-04-22T07:06:30.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.544+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2838102a80 con 0x7f28381015a0 2024-04-22T07:06:30.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.544+0000 7f282f7fe700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f28240396b0 0x7f282403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:30.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.544+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f282801e030 con 0x7f28381015a0 2024-04-22T07:06:30.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.544+0000 7f282ffff700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f28240396b0 0x7f282403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:30.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.545+0000 7f282ffff700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f28240396b0 0x7f282403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2820006fd0 tx=0x7f2820006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:30.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.549+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2828013070 con 0x7f28381015a0 2024-04-22T07:06:30.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.751+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2838199230 con 0x7f28381015a0 2024-04-22T07:06:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.802+0000 7f282f7fe700 1 -- 172.21.15.113:0/1618514243 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2828031070 con 0x7f28381015a0 2024-04-22T07:06:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f28240396b0 msgr2=0x7f282403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f28240396b0 0x7f282403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2820006fd0 tx=0x7f2820006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 msgr2=0x7f2838197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838197ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f2828009ec0 comp rx=0 tx=0).stop 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 shutdown_connections 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f28240396b0 0x7f282403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 --2- 172.21.15.113:0/1618514243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28381015a0 0x7f2838197ed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 >> 172.21.15.113:0/1618514243 conn(0x7f28380fce90 msgr2=0x7f283806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 shutdown_connections 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:30.804+0000 7f283cc68700 1 -- 172.21.15.113:0/1618514243 wait complete. 2024-04-22T07:06:30.803 INFO:teuthology.orchestra.run.smithi113.stderr:adjusted tunables profile to default 2024-04-22T07:06:31.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:30 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1618514243' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T07:06:31.420 INFO:tasks.cephadm:Adding mon.a on smithi113 2024-04-22T07:06:31.420 INFO:tasks.cephadm:Adding mon.b on smithi133 2024-04-22T07:06:31.420 INFO:tasks.cephadm:Adding mon.c on smithi154 2024-04-22T07:06:31.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c' 2024-04-22T07:06:31.708 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T07:06:31.756 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T07:06:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1618514243' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T07:06:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:32 smithi113 ceph-mon[27933]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-22T07:06:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:32 smithi113 ceph-mon[27933]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:06:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:32 smithi113 ceph-mon[27933]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:33.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:33.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:33.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 -- 172.21.15.154:0/3340958639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 msgr2=0x7fb96c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 --2- 172.21.15.154:0/3340958639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c108870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009910 tx=0x7fb95c009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 -- 172.21.15.154:0/3340958639 shutdown_connections 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 --2- 172.21.15.154:0/3340958639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c108870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 -- 172.21.15.154:0/3340958639 >> 172.21.15.154:0/3340958639 conn(0x7fb96c077a30 msgr2=0x7fb96c077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 -- 172.21.15.154:0/3340958639 shutdown_connections 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.285+0000 7fb9735aa700 1 -- 172.21.15.154:0/3340958639 wait complete. 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb9735aa700 1 Processor -- start 2024-04-22T07:06:33.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb9735aa700 1 -- start start 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb9735aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb9735aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10c9c0 con 0x7fb96c108490 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb971346700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb971346700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.154:49730/0 (socket says 172.21.15.154:49730) 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb971346700 1 -- 172.21.15.154:0/608597284 learned_addr learned my addr 172.21.15.154:0/608597284 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.286+0000 7fb971346700 1 -- 172.21.15.154:0/608597284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c198360 con 0x7fb96c108490 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.287+0000 7fb971346700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c197e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006010 tx=0x7fb95c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.287+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c027070 con 0x7fb96c108490 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.287+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c198650 con 0x7fb96c108490 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.287+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb95c00be30 con 0x7fb96c108490 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.287+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c19c6f0 con 0x7fb96c108490 2024-04-22T07:06:33.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.287+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c00f460 con 0x7fb96c108490 2024-04-22T07:06:33.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.288+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fb95c02a070 con 0x7fb96c108490 2024-04-22T07:06:33.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.288+0000 7fb9627fc700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb958039650 0x7fb95803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:33.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.288+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c103d50 con 0x7fb96c108490 2024-04-22T07:06:33.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.288+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb95c01e030 con 0x7fb96c108490 2024-04-22T07:06:33.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.288+0000 7fb970b45700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb958039650 0x7fb95803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:33.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.288+0000 7fb970b45700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb958039650 0x7fb95803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb968006fd0 tx=0x7fb968006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:33.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.294+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb95c013070 con 0x7fb96c108490 2024-04-22T07:06:33.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.505+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}) -- 0x7fb96c199180 con 0x7fb958039650 2024-04-22T07:06:33.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.512+0000 7fb9627fc700 1 -- 172.21.15.154:0/608597284 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb96c199180 con 0x7fb958039650 2024-04-22T07:06:33.510 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mon update... 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb958039650 msgr2=0x7fb95803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb958039650 0x7fb95803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb968006fd0 tx=0x7fb968006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 msgr2=0x7fb96c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c197e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006010 tx=0x7fb95c009ec0 comp rx=0 tx=0).stop 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 shutdown_connections 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb958039650 0x7fb95803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 --2- 172.21.15.154:0/608597284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb96c108490 0x7fb96c197e20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 >> 172.21.15.154:0/608597284 conn(0x7fb96c077a30 msgr2=0x7fb96c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:33.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 shutdown_connections 2024-04-22T07:06:33.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:33.514+0000 7fb9735aa700 1 -- 172.21.15.154:0/608597284 wait complete. 2024-04-22T07:06:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:06:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:34.102 DEBUG:teuthology.orchestra.run.smithi133:mon.b> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mon.b.service 2024-04-22T07:06:34.105 DEBUG:teuthology.orchestra.run.smithi154:mon.c> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mon.c.service 2024-04-22T07:06:34.107 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T07:06:34.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T07:06:34.836 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:06:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:34 smithi113 ceph-mon[27933]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:06:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:34 smithi113 ceph-mon[27933]: Saving service mon spec with placement smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c;count:3 2024-04-22T07:06:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:34 smithi113 ceph-mon[27933]: Deploying daemon mon.c on smithi154 2024-04-22T07:06:36.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:35 smithi113 ceph-mon[27933]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 -- 172.21.15.154:0/3850010992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 msgr2=0x7f9228101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 --2- 172.21.15.154:0/3850010992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228101990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9218009910 tx=0x7f9218009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 -- 172.21.15.154:0/3850010992 shutdown_connections 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 --2- 172.21.15.154:0/3850010992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228101990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 -- 172.21.15.154:0/3850010992 >> 172.21.15.154:0/3850010992 conn(0x7f92280fce80 msgr2=0x7f92280ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 -- 172.21.15.154:0/3850010992 shutdown_connections 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.204+0000 7f9230136700 1 -- 172.21.15.154:0/3850010992 wait complete. 2024-04-22T07:06:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f9230136700 1 Processor -- start 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f9230136700 1 -- start start 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f9230136700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f9230136700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f922810b970 con 0x7f92281015b0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f922ded2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f922ded2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.154:49756/0 (socket says 172.21.15.154:49756) 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.205+0000 7f922ded2700 1 -- 172.21.15.154:0/1616885437 learned_addr learned my addr 172.21.15.154:0/1616885437 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f922ded2700 1 -- 172.21.15.154:0/1616885437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9228104f40 con 0x7f92281015b0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f922ded2700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228103050 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9218006010 tx=0x7f9218009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218027070 con 0x7f92281015b0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f921800be30 con 0x7f92281015b0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92281035f0 con 0x7f92281015b0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f921800f460 con 0x7f92281015b0 2024-04-22T07:06:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.206+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9228103ab0 con 0x7f92281015b0 2024-04-22T07:06:37.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.207+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9228102a90 con 0x7f92281015b0 2024-04-22T07:06:37.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.207+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f921802a070 con 0x7f92281015b0 2024-04-22T07:06:37.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.207+0000 7f921effd700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9214039650 0x7f921403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:37.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.207+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f921801e030 con 0x7f92281015b0 2024-04-22T07:06:37.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.208+0000 7f922d6d1700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9214039650 0x7f921403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:37.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.208+0000 7f922d6d1700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9214039650 0x7f921403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9224006fd0 tx=0x7f9224006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:37.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.213+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9218013070 con 0x7f92281015b0 2024-04-22T07:06:37.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.501+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9228104620 con 0x7f92281015b0 2024-04-22T07:06:37.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.502+0000 7f921effd700 1 -- 172.21.15.154:0/1616885437 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f9218031020 con 0x7f92281015b0 2024-04-22T07:06:37.500 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:06:37.500 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":1,"fsid":"81035712-0076-11ef-bc93-c7b262605968","modified":"2024-04-22T07:04:55.557885Z","created":"2024-04-22T07:04:55.557885Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9214039650 msgr2=0x7f921403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9214039650 0x7f921403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9224006fd0 tx=0x7f9224006e40 comp rx=0 tx=0).stop 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 msgr2=0x7f9228103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228103050 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9218006010 tx=0x7f9218009ec0 comp rx=0 tx=0).stop 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 shutdown_connections 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9214039650 0x7f921403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 --2- 172.21.15.154:0/1616885437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92281015b0 0x7f9228103050 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:37.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 >> 172.21.15.154:0/1616885437 conn(0x7f92280fce80 msgr2=0x7f922806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:37.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 shutdown_connections 2024-04-22T07:06:37.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:37.504+0000 7f9230136700 1 -- 172.21.15.154:0/1616885437 wait complete. 2024-04-22T07:06:37.503 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 1 2024-04-22T07:06:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:38 smithi113 ceph-mon[27933]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:38 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/1616885437' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:39.333 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T07:06:39.334 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T07:06:39.566 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:06:44.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.550+0000 7f2b69d07700 1 -- 172.21.15.154:0/3506392727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b4c0035c0 msgr2=0x7f2b4c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:44.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.550+0000 7f2b69d07700 1 --2- 172.21.15.154:0/3506392727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b4c0035c0 0x7f2b4c005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b64102490 tx=0x7f2b54014ec0 comp rx=0 tx=0).stop 2024-04-22T07:06:44.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.550+0000 7f2b69d07700 1 -- 172.21.15.154:0/3506392727 shutdown_connections 2024-04-22T07:06:44.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.550+0000 7f2b69d07700 1 --2- 172.21.15.154:0/3506392727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b4c0035c0 0x7f2b4c005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:44.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.550+0000 7f2b69d07700 1 -- 172.21.15.154:0/3506392727 >> 172.21.15.154:0/3506392727 conn(0x7f2b640fe050 msgr2=0x7f2b64100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:44.550 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:44.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.551+0000 7f2b69d07700 1 -- 172.21.15.154:0/3506392727 shutdown_connections 2024-04-22T07:06:44.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.551+0000 7f2b69d07700 1 -- 172.21.15.154:0/3506392727 wait complete. 2024-04-22T07:06:44.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.552+0000 7f2b69d07700 1 Processor -- start 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.552+0000 7f2b69d07700 1 -- start start 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.552+0000 7f2b69d07700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.552+0000 7f2b69d07700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b64075280 0x7f2b64075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.552+0000 7f2b69d07700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410a610 con 0x7f2b64075280 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b69d07700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410a490 con 0x7f2b4c0035c0 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57140/0 (socket says 172.21.15.154:57140) 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 -- 172.21.15.154:0/2260668087 learned_addr learned my addr 172.21.15.154:0/2260668087 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b62ffd700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b64075280 0x7f2b64075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 -- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 msgr2=0x7f2b64074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T07:06:44.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 -- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 msgr2=0x7f2b64074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T07:06:44.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:06:44.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b637fe700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:44.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b62ffd700 1 -- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 msgr2=0x7f2b64074d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:06:44.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b62ffd700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:44.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.553+0000 7f2b62ffd700 1 -- 172.21.15.154:0/2260668087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64076c60 con 0x7f2b64075280 2024-04-22T07:06:44.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.554+0000 7f2b62ffd700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b64075280 0x7f2b64075700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800b4d0 tx=0x7f2b5800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:44.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.554+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2b58011630 con 0x7f2b64075280 2024-04-22T07:06:44.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.554+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2b58011ca0 con 0x7f2b64075280 2024-04-22T07:06:44.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.554+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641a4360 con 0x7f2b64075280 2024-04-22T07:06:44.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.554+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2b58010e50 con 0x7f2b64075280 2024-04-22T07:06:44.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.554+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a48a0 con 0x7f2b64075280 2024-04-22T07:06:44.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.555+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2b58018020 con 0x7f2b64075280 2024-04-22T07:06:44.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.555+0000 7f2b60ff9700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b50039600 0x7f2b5003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:44.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.555+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2b5804d1f0 con 0x7f2b64075280 2024-04-22T07:06:44.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.555+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b64103ca0 con 0x7f2b64075280 2024-04-22T07:06:44.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.556+0000 7f2b637fe700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b50039600 0x7f2b5003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:44.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.556+0000 7f2b637fe700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b50039600 0x7f2b5003bac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400f470 tx=0x7f2b54021040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.561+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b5801f020 con 0x7f2b64075280 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: mon.a calling monitor election 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: mon.c calling monitor election 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T07:06:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: monmap epoch 2 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: last_changed 2024-04-22T07:06:39.210152+0000 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: min_mon_release 19 (squid) 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: election_strategy: 1 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: fsmap 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: mgrmap e13: x(active, since 49s) 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: overall HEALTH_OK 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:44.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:44.587 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:44.587 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:44.587 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:44.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.846+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2b6402df80 con 0x7f2b64075280 2024-04-22T07:06:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.847+0000 7f2b60ff9700 1 -- 172.21.15.154:0/2260668087 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f2b580214c0 con 0x7f2b64075280 2024-04-22T07:06:44.846 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:06:44.846 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":2,"fsid":"81035712-0076-11ef-bc93-c7b262605968","modified":"2024-04-22T07:06:39.210152Z","created":"2024-04-22T07:04:55.557885Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T07:06:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b50039600 msgr2=0x7f2b5003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b50039600 0x7f2b5003bac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400f470 tx=0x7f2b54021040 comp rx=0 tx=0).stop 2024-04-22T07:06:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b64075280 msgr2=0x7f2b64075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b64075280 0x7f2b64075700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800b4d0 tx=0x7f2b5800b890 comp rx=0 tx=0).stop 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 shutdown_connections 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b50039600 0x7f2b5003bac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b4c0035c0 0x7f2b64074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 --2- 172.21.15.154:0/2260668087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b64075280 0x7f2b64075700 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 >> 172.21.15.154:0/2260668087 conn(0x7f2b640fe050 msgr2=0x7f2b640ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 shutdown_connections 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:44.849+0000 7f2b69d07700 1 -- 172.21.15.154:0/2260668087 wait complete. 2024-04-22T07:06:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 2 2024-04-22T07:06:45.457 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:45 smithi154 ceph-mon[31862]: Deploying daemon mon.b on smithi133 2024-04-22T07:06:45.457 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:45 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/2260668087' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:45.457 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:45 smithi154 ceph-mon[31862]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:45.457 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:45 smithi113 ceph-mon[27933]: Deploying daemon mon.b on smithi133 2024-04-22T07:06:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/2260668087' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:45 smithi113 ceph-mon[27933]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:46.486 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T07:06:46.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T07:06:46.732 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:06:48.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 -- 172.21.15.154:0/3567406984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e201074e0 msgr2=0x7f0e201078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:48.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 --2- 172.21.15.154:0/3567406984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e201074e0 0x7f0e201078c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800b0a0 tx=0x7f0e1800b3b0 comp rx=0 tx=0).stop 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 -- 172.21.15.154:0/3567406984 shutdown_connections 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 --2- 172.21.15.154:0/3567406984 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e201074e0 0x7f0e201078c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 -- 172.21.15.154:0/3567406984 >> 172.21.15.154:0/3567406984 conn(0x7f0e200fcf40 msgr2=0x7f0e200ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 -- 172.21.15.154:0/3567406984 shutdown_connections 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.127+0000 7f0e282c6700 1 -- 172.21.15.154:0/3567406984 wait complete. 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.128+0000 7f0e282c6700 1 Processor -- start 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.128+0000 7f0e282c6700 1 -- start start 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.128+0000 7f0e282c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.128+0000 7f0e282c6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2019a980 0x7f0e2019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:48.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.128+0000 7f0e282c6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2010ba80 con 0x7f0e2019a980 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.128+0000 7f0e282c6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2010b900 con 0x7f0e201074e0 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57150/0 (socket says 172.21.15.154:57150) 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 -- 172.21.15.154:0/2222476141 learned_addr learned my addr 172.21.15.154:0/2222476141 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e25861700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2019a980 0x7f0e2019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 -- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 msgr2=0x7f0e2019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 -- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 msgr2=0x7f0e2019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e26062700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e25861700 1 -- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 msgr2=0x7f0e2019a440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e25861700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e25861700 1 -- 172.21.15.154:0/2222476141 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2019afc0 con 0x7f0e2019a980 2024-04-22T07:06:48.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e25861700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2019a980 0x7f0e2019ed10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000b4d0 tx=0x7f0e1000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0e10011630 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2019f250 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0e10011ca0 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0e10010e50 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.129+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2019f7b0 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.130+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e20102b90 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.130+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f0e10018020 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.131+0000 7f0e177fe700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0e0c039580 0x7f0e0c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.131+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e1004c160 con 0x7f0e2019a980 2024-04-22T07:06:48.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.131+0000 7f0e26062700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0e0c039580 0x7f0e0c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:48.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.131+0000 7f0e26062700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0e0c039580 0x7f0e0c03ba40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009850 tx=0x7f0e180096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:48.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.137+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e1001f020 con 0x7f0e2019a980 2024-04-22T07:06:48.184 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:48 smithi154 ceph-mon[31862]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(???) e0 preinit fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).mds e1 new map 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout: e1 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout: legacy client fscid: -1 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout: No filesystems configured 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-22T07:06:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-22T07:06:48.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.200+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10010530 con 0x7f0e2019a980 2024-04-22T07:06:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:48 smithi113 ceph-mon[27933]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:48.440+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0e2019fac0 con 0x7f0e2019a980 2024-04-22T07:06:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Deploying cephadm binary to smithi154 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Added host smithi154 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1618514243' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1618514243' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:48.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Saving service mon spec with placement smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c;count:3 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Deploying daemon mon.c on smithi154 2024-04-22T07:06:48.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/1616885437' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.a calling monitor election 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.c calling monitor election 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: monmap epoch 2 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: last_changed 2024-04-22T07:06:39.210152+0000 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: min_mon_release 19 (squid) 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: election_strategy: 1 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: fsmap 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:48.571 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mgrmap e13: x(active, since 49s) 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: overall HEALTH_OK 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: Deploying daemon mon.b on smithi133 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/2260668087' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 ceph-mon[31198]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 bash[31179]: ba8f14fe86288af8cc7c3e99976f14ca9d56b45cca8ade96621d9207fba1766a 2024-04-22T07:06:48.572 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:48 smithi133 systemd[1]: Started Ceph mon.b for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:06:53.208 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-04-22T07:06:53.208 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"fsid":"81035712-0076-11ef-bc93-c7b262605968","modified":"2024-04-22T07:06:48.198641Z","created":"2024-04-22T07:04:55.557885Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T07:06:53.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.209+0000 7f0e177fe700 1 -- 172.21.15.154:0/2222476141 <== mon.0 v2:172.21.15.113:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f0e1003e520 con 0x7f0e2019a980 2024-04-22T07:06:53.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0e0c039580 msgr2=0x7f0e0c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:53.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0e0c039580 0x7f0e0c03ba40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009850 tx=0x7f0e180096e0 comp rx=0 tx=0).stop 2024-04-22T07:06:53.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2019a980 msgr2=0x7f0e2019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:53.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2019a980 0x7f0e2019ed10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000b4d0 tx=0x7f0e1000b890 comp rx=0 tx=0).stop 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 shutdown_connections 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0e0c039580 0x7f0e0c03ba40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e201074e0 0x7f0e2019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 --2- 172.21.15.154:0/2222476141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2019a980 0x7f0e2019ed10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 >> 172.21.15.154:0/2222476141 conn(0x7f0e200fcf40 msgr2=0x7f0e2006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.212+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 shutdown_connections 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:53.213+0000 7f0e282c6700 1 -- 172.21.15.154:0/2222476141 wait complete. 2024-04-22T07:06:53.211 INFO:teuthology.orchestra.run.smithi154.stderr:dumped monmap epoch 3 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: mon.c calling monitor election 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: mon.a calling monitor election 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/2222476141' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.563 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: monmap epoch 3 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: last_changed 2024-04-22T07:06:48.198641+0000 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: min_mon_release 19 (squid) 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: election_strategy: 1 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: 2: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: fsmap 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:53.564 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: mgrmap e13: x(active, since 58s) 2024-04-22T07:06:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: overall HEALTH_OK 2024-04-22T07:06:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: mon.c calling monitor election 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: mon.a calling monitor election 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/2222476141' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: monmap epoch 3 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: last_changed 2024-04-22T07:06:48.198641+0000 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: min_mon_release 19 (squid) 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: election_strategy: 1 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: 2: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: fsmap 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: mgrmap e13: x(active, since 58s) 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: overall HEALTH_OK 2024-04-22T07:06:53.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.587 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.587 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.587 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:53.831 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-22T07:06:53.832 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-22T07:06:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:06:54.518 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:54.518 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:54.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:06:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: mon.c calling monitor election 2024-04-22T07:06:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: mon.a calling monitor election 2024-04-22T07:06:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/2222476141' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: monmap epoch 3 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: last_changed 2024-04-22T07:06:48.198641+0000 2024-04-22T07:06:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: min_mon_release 19 (squid) 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: election_strategy: 1 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: 2: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: fsmap 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: mgrmap e13: x(active, since 58s) 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: overall HEALTH_OK 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:55.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:06:55.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:55.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 -- 172.21.15.113:0/3178213161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 msgr2=0x7fedec101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:55.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 --2- 172.21.15.113:0/3178213161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec101990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fede8009910 tx=0x7fede8009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:55.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 -- 172.21.15.113:0/3178213161 shutdown_connections 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 --2- 172.21.15.113:0/3178213161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec101990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 -- 172.21.15.113:0/3178213161 >> 172.21.15.113:0/3178213161 conn(0x7fedec0fce80 msgr2=0x7fedec0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 -- 172.21.15.113:0/3178213161 shutdown_connections 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.683+0000 7fedf41be700 1 -- 172.21.15.113:0/3178213161 wait complete. 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 Processor -- start 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 -- start start 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec10c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedec10c6c0 0x7fedec10cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedec1a43a0 0x7fedec1a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fedec10b930 con 0x7fedec1015b0 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fedec10b7b0 con 0x7fedec10c6c0 2024-04-22T07:06:55.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf41be700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fedec10bab0 con 0x7fedec1a43a0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf1f5a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec10c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf1f5a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec10c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53050/0 (socket says 172.21.15.113:53050) 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.684+0000 7fedf1f5a700 1 -- 172.21.15.113:0/2931949611 learned_addr learned my addr 172.21.15.113:0/2931949611 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1759700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedec10c6c0 0x7fedec10cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf275b700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedec1a43a0 0x7fedec1a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1f5a700 1 -- 172.21.15.113:0/2931949611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedec1a43a0 msgr2=0x7fedec1a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1f5a700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedec1a43a0 0x7fedec1a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1f5a700 1 -- 172.21.15.113:0/2931949611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedec10c6c0 msgr2=0x7fedec10cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1f5a700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedec10c6c0 0x7fedec10cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1f5a700 1 -- 172.21.15.113:0/2931949611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec1a6ec0 con 0x7fedec1015b0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1759700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedec10c6c0 0x7fedec10cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf1f5a700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec10c180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fede800bdf0 tx=0x7fede800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8027070 con 0x7fedec1015b0 2024-04-22T07:06:55.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fede8005da0 con 0x7fedec1015b0 2024-04-22T07:06:55.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedec1a71b0 con 0x7fedec1015b0 2024-04-22T07:06:55.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede800f460 con 0x7fedec1015b0 2024-04-22T07:06:55.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.685+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedec1a7740 con 0x7fedec1015b0 2024-04-22T07:06:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.686+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fede802a070 con 0x7fedec1015b0 2024-04-22T07:06:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.686+0000 7fede2ffd700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fedd8039ae0 0x7fedd803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.686+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fede804e920 con 0x7fedec1015b0 2024-04-22T07:06:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.686+0000 7fedf1759700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fedd8039ae0 0x7fedd803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.686+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedec0640a0 con 0x7fedec1015b0 2024-04-22T07:06:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.687+0000 7fedf1759700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fedd8039ae0 0x7fedd803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feddc009910 tx=0x7feddc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:55.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.692+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fede8031070 con 0x7fedec1015b0 2024-04-22T07:06:55.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.911+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fedec1a79f0 con 0x7fedec1015b0 2024-04-22T07:06:55.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.911+0000 7fede2ffd700 1 -- 172.21.15.113:0/2931949611 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fede8013070 con 0x7fedec1015b0 2024-04-22T07:06:55.909 INFO:teuthology.orchestra.run.smithi113.stdout:# minimal ceph.conf for 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:55.909 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-22T07:06:55.909 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:55.909 INFO:teuthology.orchestra.run.smithi113.stdout: mon_host = [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-04-22T07:06:55.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fedd8039ae0 msgr2=0x7fedd803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:55.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fedd8039ae0 0x7fedd803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feddc009910 tx=0x7feddc008040 comp rx=0 tx=0).stop 2024-04-22T07:06:55.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 msgr2=0x7fedec10c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:55.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec10c180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fede800bdf0 tx=0x7fede800bed0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 shutdown_connections 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fedd8039ae0 0x7fedd803bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedec1a43a0 0x7fedec1a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedec1015b0 0x7fedec10c180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 --2- 172.21.15.113:0/2931949611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedec10c6c0 0x7fedec10cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 >> 172.21.15.113:0/2931949611 conn(0x7fedec0fce80 msgr2=0x7fedec0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 shutdown_connections 2024-04-22T07:06:55.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:06:55.913+0000 7fedf41be700 1 -- 172.21.15.113:0/2931949611 wait complete. 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: mon.b calling monitor election 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: mon.b calling monitor election 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: mon.c calling monitor election 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: mon.a calling monitor election 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: monmap epoch 3 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: last_changed 2024-04-22T07:06:48.198641+0000 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: min_mon_release 19 (squid) 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: election_strategy: 1 2024-04-22T07:06:56.094 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: 2: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: fsmap 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: mgrmap e13: x(active, since 60s) 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: overall HEALTH_OK 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.095 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:56.096 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2931949611' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: mon.b calling monitor election 2024-04-22T07:06:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: mon.b calling monitor election 2024-04-22T07:06:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: mon.c calling monitor election 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: mon.a calling monitor election 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: monmap epoch 3 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: last_changed 2024-04-22T07:06:48.198641+0000 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: min_mon_release 19 (squid) 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: election_strategy: 1 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: 2: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: fsmap 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:56.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: mgrmap e13: x(active, since 60s) 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: overall HEALTH_OK 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:56.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2931949611' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: mon.b calling monitor election 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: mon.b calling monitor election 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: mon.c calling monitor election 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: mon.a calling monitor election 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: monmap epoch 3 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:06:56.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: last_changed 2024-04-22T07:06:48.198641+0000 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: created 2024-04-22T07:04:55.557885+0000 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: min_mon_release 19 (squid) 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: election_strategy: 1 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: 0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: 1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.c 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: 2: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: fsmap 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: mgrmap e13: x(active, since 60s) 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: overall HEALTH_OK 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:56.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2931949611' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:57.076 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-22T07:06:57.076 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:57.077 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T07:06:57.105 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:57.105 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:57.170 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:06:57.171 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T07:06:57.202 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:06:57.203 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:57.270 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:06:57.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T07:06:57.296 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:57 smithi154 ceph-mon[31862]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T07:06:57.296 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:57 smithi154 ceph-mon[31862]: Reconfiguring daemon mon.a on smithi113 2024-04-22T07:06:57.296 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:57 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:57.302 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:06:57.302 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:06:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:57 smithi133 ceph-mon[31198]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T07:06:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:57 smithi133 ceph-mon[31198]: Reconfiguring daemon mon.a on smithi113 2024-04-22T07:06:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:57 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:57 smithi113 ceph-mon[27933]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T07:06:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:57 smithi113 ceph-mon[27933]: Reconfiguring daemon mon.a on smithi113 2024-04-22T07:06:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:57 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:06:57.368 INFO:tasks.cephadm:Adding mgr.x on smithi113 2024-04-22T07:06:57.368 INFO:tasks.cephadm:Adding mgr.y on smithi133 2024-04-22T07:06:57.369 INFO:tasks.cephadm:Adding mgr.z on smithi154 2024-04-22T07:06:57.369 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi113=x;smithi133=y;smithi154=z' 2024-04-22T07:06:57.652 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:06:58.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:58 smithi133 ceph-mon[31198]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:58.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:58.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:58.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:58.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:06:58.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:58.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:58 smithi154 ceph-mon[31862]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:58.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:58.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:58.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:58.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:06:58.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:58.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:58 smithi113 ceph-mon[27933]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:06:58.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:58.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:06:58.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:06:58.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:06:58.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.078+0000 7fc6ed14e700 1 -- 172.21.15.154:0/569792095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81014e0 msgr2=0x7fc6e8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.078+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/569792095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81014e0 0x7fc6e8101900 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009910 tx=0x7fc6d8009c20 comp rx=0 tx=0).stop 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 -- 172.21.15.154:0/569792095 shutdown_connections 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/569792095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81014e0 0x7fc6e8101900 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 -- 172.21.15.154:0/569792095 >> 172.21.15.154:0/569792095 conn(0x7fc6e80fca80 msgr2=0x7fc6e80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 -- 172.21.15.154:0/569792095 shutdown_connections 2024-04-22T07:06:59.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 -- 172.21.15.154:0/569792095 wait complete. 2024-04-22T07:06:59.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 Processor -- start 2024-04-22T07:06:59.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.079+0000 7fc6ed14e700 1 -- start start 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6ed14e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 0x7fc6e8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6ed14e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81989c0 0x7fc6e819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6ed14e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e819db90 0x7fc6e819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6ed14e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8107510 con 0x7fc6e81989c0 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6ed14e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8107390 con 0x7fc6e819db90 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6ed14e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8107690 con 0x7fc6e8198060 2024-04-22T07:06:59.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 0x7fc6e8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 0x7fc6e8198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:34714/0 (socket says 172.21.15.154:34714) 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 -- 172.21.15.154:0/2609191437 learned_addr learned my addr 172.21.15.154:0/2609191437 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e659c700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81989c0 0x7fc6e819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e759e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e819db90 0x7fc6e819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 -- 172.21.15.154:0/2609191437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e819db90 msgr2=0x7fc6e819ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e819db90 0x7fc6e819ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 -- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81989c0 msgr2=0x7fc6e819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81989c0 0x7fc6e819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 -- 172.21.15.154:0/2609191437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e81a08b0 con 0x7fc6e8198060 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e759e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e819db90 0x7fc6e819ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6e6d9d700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 0x7fc6e8198480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b5c0 tx=0x7fc6d800bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.080+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8026070 con 0x7fc6e8198060 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.081+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e81a0b40 con 0x7fc6e8198060 2024-04-22T07:06:59.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.081+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc6d8005da0 con 0x7fc6e8198060 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.081+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800fc90 con 0x7fc6e8198060 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.081+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e81a1020 con 0x7fc6e8198060 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.081+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e80640a0 con 0x7fc6e8198060 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.082+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fc6d8029070 con 0x7fc6e8198060 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.082+0000 7fc6cffff700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc6d0039ae0 0x7fc6d003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.082+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc6d804d3d0 con 0x7fc6e8198060 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.082+0000 7fc6e659c700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc6d0039ae0 0x7fc6d003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:59.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.082+0000 7fc6e659c700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc6d0039ae0 0x7fc6d003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009ad0 tx=0x7fc6dc009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:59.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.087+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6d8013070 con 0x7fc6e8198060 2024-04-22T07:06:59.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.303+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi113=x;smithi133=y;smithi154=z", "target": ["mon-mgr", ""]}) -- 0x7fc6e8062f10 con 0x7fc6d0039ae0 2024-04-22T07:06:59.302 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:59 smithi154 ceph-mon[31862]: Reconfiguring mon.b (monmap changed)... 2024-04-22T07:06:59.302 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:06:59 smithi154 ceph-mon[31862]: Reconfiguring daemon mon.b on smithi133 2024-04-22T07:06:59.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.312+0000 7fc6cffff700 1 -- 172.21.15.154:0/2609191437 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc6e8062f10 con 0x7fc6d0039ae0 2024-04-22T07:06:59.310 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mgr update... 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc6d0039ae0 msgr2=0x7fc6d003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc6d0039ae0 0x7fc6d003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009ad0 tx=0x7fc6dc009210 comp rx=0 tx=0).stop 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 msgr2=0x7fc6e8198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 0x7fc6e8198480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b5c0 tx=0x7fc6d800bdc0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 shutdown_connections 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc6d0039ae0 0x7fc6d003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e819db90 0x7fc6e819ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc6e8198060 0x7fc6e8198480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 --2- 172.21.15.154:0/2609191437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6e81989c0 0x7fc6e819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:59.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 >> 172.21.15.154:0/2609191437 conn(0x7fc6e80fca80 msgr2=0x7fc6e818ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:59.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 shutdown_connections 2024-04-22T07:06:59.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:06:59.314+0000 7fc6ed14e700 1 -- 172.21.15.154:0/2609191437 wait complete. 2024-04-22T07:06:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:59 smithi133 ceph-mon[31198]: Reconfiguring mon.b (monmap changed)... 2024-04-22T07:06:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:06:59 smithi133 ceph-mon[31198]: Reconfiguring daemon mon.b on smithi133 2024-04-22T07:06:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:59 smithi113 ceph-mon[27933]: Reconfiguring mon.b (monmap changed)... 2024-04-22T07:06:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:06:59 smithi113 ceph-mon[27933]: Reconfiguring daemon mon.b on smithi133 2024-04-22T07:06:59.934 DEBUG:teuthology.orchestra.run.smithi133:mgr.y> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.y.service 2024-04-22T07:06:59.937 DEBUG:teuthology.orchestra.run.smithi154:mgr.z> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.z.service 2024-04-22T07:06:59.939 INFO:tasks.cephadm:Deploying OSDs... 2024-04-22T07:06:59.939 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:06:59.939 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:06:59.958 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:06:59.959 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-04-22T07:07:00.015 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-22 07:06:05.128292538 +0000 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-22 07:00:40.932116250 +0000 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-22 07:00:40.932116250 +0000 2024-04-22T07:07:00.016 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-22 07:00:40.932116250 +0000 2024-04-22T07:07:00.016 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:07:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-22T07:07:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-22T07:07:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000230299 s, 2.2 MB/s 2024-04-22T07:07:00.084 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:07:00.141 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-04-22T07:07:00.198 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-22 07:06:05.154291978 +0000 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-22 07:00:41.270108960 +0000 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-22 07:00:41.270108960 +0000 2024-04-22T07:07:00.199 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-22 07:00:41.270108960 +0000 2024-04-22T07:07:00.199 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:07:00.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-22T07:07:00.262 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-22T07:07:00.262 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000223646 s, 2.3 MB/s 2024-04-22T07:07:00.263 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: Reconfiguring mon.c (monmap changed)... 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:07:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:00.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: Reconfiguring daemon mon.c on smithi154 2024-04-22T07:07:00.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: Reconfiguring mon.c (monmap changed)... 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: Reconfiguring daemon mon.c on smithi154 2024-04-22T07:07:00.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.320 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: Reconfiguring mon.c (monmap changed)... 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T07:07:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:00.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: Reconfiguring daemon mon.c on smithi154 2024-04-22T07:07:00.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:00.349 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:07:00.349 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-22 07:06:05.178291460 +0000 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-22 07:00:41.604101757 +0000 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-22 07:00:41.604101757 +0000 2024-04-22T07:07:00.350 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-22 07:00:41.604101757 +0000 2024-04-22T07:07:00.350 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:07:00.412 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-22T07:07:00.413 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-22T07:07:00.413 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000210904 s, 2.4 MB/s 2024-04-22T07:07:00.414 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:07:00.470 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-04-22T07:07:00.525 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:07:00.525 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:00.525 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T07:07:00.525 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:00.525 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:00.525 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-22 07:06:05.208290814 +0000 2024-04-22T07:07:00.526 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-22 07:00:41.936094597 +0000 2024-04-22T07:07:00.526 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-22 07:00:41.936094597 +0000 2024-04-22T07:07:00.526 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-22 07:00:41.936094597 +0000 2024-04-22T07:07:00.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:07:00.588 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-22T07:07:00.589 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-22T07:07:00.589 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000217566 s, 2.4 MB/s 2024-04-22T07:07:00.590 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:07:00.646 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-22T07:07:00.646 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:07:00.665 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:07:00.665 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-04-22T07:07:00.722 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-22 07:06:47.627559662 +0000 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-22 07:01:46.024456799 +0000 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-22 07:01:46.024456799 +0000 2024-04-22T07:07:00.723 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-22 07:01:46.024456799 +0000 2024-04-22T07:07:00.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:07:00.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-22T07:07:00.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-22T07:07:00.789 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000261564 s, 2.0 MB/s 2024-04-22T07:07:00.790 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:07:00.848 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-04-22T07:07:00.905 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:07:00.905 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:00.905 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T07:07:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-22 07:06:47.627559662 +0000 2024-04-22T07:07:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-22 07:01:46.381450111 +0000 2024-04-22T07:07:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-22 07:01:46.381450111 +0000 2024-04-22T07:07:00.906 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-22 07:01:46.381450111 +0000 2024-04-22T07:07:00.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:07:00.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-22T07:07:00.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-22T07:07:00.970 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000234023 s, 2.2 MB/s 2024-04-22T07:07:00.971 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:07:01.029 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-04-22T07:07:01.085 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:07:01.085 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:01.085 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-22T07:07:01.085 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:01.085 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:01.086 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-22 07:06:47.628559644 +0000 2024-04-22T07:07:01.086 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-22 07:01:46.728443611 +0000 2024-04-22T07:07:01.086 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-22 07:01:46.728443611 +0000 2024-04-22T07:07:01.086 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-22 07:01:46.728443611 +0000 2024-04-22T07:07:01.086 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:07:01.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-22T07:07:01.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-22T07:07:01.149 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000204658 s, 2.5 MB/s 2024-04-22T07:07:01.150 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:07:01.206 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-04-22T07:07:01.261 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-22 07:06:47.628559644 +0000 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-22 07:01:47.058437428 +0000 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-22 07:01:47.058437428 +0000 2024-04-22T07:07:01.262 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-22 07:01:47.058437428 +0000 2024-04-22T07:07:01.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:07:01.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-22T07:07:01.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-22T07:07:01.324 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000229737 s, 2.2 MB/s 2024-04-22T07:07:01.325 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:07:01.381 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-04-22T07:07:01.381 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:07:01.397 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:07:01.398 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-22 07:06:38.637102143 +0000 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-22 07:00:58.515039839 +0000 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-22 07:00:58.515039839 +0000 2024-04-22T07:07:01.453 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-22 07:00:58.515039839 +0000 2024-04-22T07:07:01.453 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:07:01.513 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:01 smithi154 ceph-mon[31862]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi113=x;smithi133=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:01.513 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:01 smithi154 ceph-mon[31862]: Saving service mgr spec with placement smithi113=x;smithi133=y;smithi154=z;count:3 2024-04-22T07:07:01.514 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:01.514 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:01.514 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:01.516 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-22T07:07:01.516 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-22T07:07:01.516 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000272694 s, 1.9 MB/s 2024-04-22T07:07:01.517 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:07:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:01 smithi133 ceph-mon[31198]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi113=x;smithi133=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:01 smithi133 ceph-mon[31198]: Saving service mgr spec with placement smithi113=x;smithi133=y;smithi154=z;count:3 2024-04-22T07:07:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:01.574 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-04-22T07:07:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:01 smithi113 ceph-mon[27933]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi113=x;smithi133=y;smithi154=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:01 smithi113 ceph-mon[27933]: Saving service mgr spec with placement smithi113=x;smithi133=y;smithi154=z;count:3 2024-04-22T07:07:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-22 07:06:38.637102143 +0000 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-22 07:00:58.865030665 +0000 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-22 07:00:58.865030665 +0000 2024-04-22T07:07:01.629 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-22 07:00:58.865030665 +0000 2024-04-22T07:07:01.629 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:07:01.690 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-22T07:07:01.690 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-22T07:07:01.690 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000203413 s, 2.5 MB/s 2024-04-22T07:07:01.691 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:07:01.747 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-04-22T07:07:01.802 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:07:01.802 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:01.802 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-22T07:07:01.802 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:01.803 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:01.803 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-22 07:06:38.637102143 +0000 2024-04-22T07:07:01.803 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-22 07:00:59.225021232 +0000 2024-04-22T07:07:01.803 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-22 07:00:59.225021232 +0000 2024-04-22T07:07:01.803 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-22 07:00:59.225021232 +0000 2024-04-22T07:07:01.803 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:07:01.866 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-22T07:07:01.867 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-22T07:07:01.867 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000199895 s, 2.6 MB/s 2024-04-22T07:07:01.868 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:07:01.924 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-04-22T07:07:01.979 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:07:01.979 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:07:01.979 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T07:07:01.980 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:07:01.980 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:07:01.980 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-04-22 07:06:38.638102117 +0000 2024-04-22T07:07:01.980 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-04-22 07:00:59.568012245 +0000 2024-04-22T07:07:01.980 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-04-22 07:00:59.568012245 +0000 2024-04-22T07:07:01.980 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-04-22 07:00:59.568012245 +0000 2024-04-22T07:07:01.980 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:07:02.042 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-04-22T07:07:02.042 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-04-22T07:07:02.042 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000207692 s, 2.5 MB/s 2024-04-22T07:07:02.043 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:07:02.099 INFO:tasks.cephadm:Deploying osd.0 on smithi113 with /dev/vg_nvme/lv_4... 2024-04-22T07:07:02.099 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T07:07:02.336 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:07:02.389 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:02.389 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:02.389 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:02.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:02.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:02.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T07:07:02.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:02.390 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:02.457 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:02.458 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:02.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T07:07:02.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:02.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:03.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:03 smithi154 ceph-mon[31862]: Deploying daemon mgr.z on smithi154 2024-04-22T07:07:03.483 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:03 smithi113 ceph-mon[27933]: Deploying daemon mgr.z on smithi154 2024-04-22T07:07:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:03 smithi133 ceph-mon[31198]: Deploying daemon mgr.z on smithi154 2024-04-22T07:07:03.816 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:03 smithi154 systemd[1]: Starting Ceph mgr.z for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:07:04.118 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:03 smithi154 podman[33701]: 2024-04-22 07:07:03.734101974 +0000 UTC m=+0.017940900 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:07:04.119 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:03 smithi154 podman[33701]: 2024-04-22 07:07:03.839495945 +0000 UTC m=+0.123334860 container create 45a3c77739e9f55fdb2d9dc8421be8b0f517f3f627872bcc3b1d2bd066776095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:07:04.119 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 podman[33701]: 2024-04-22 07:07:04.046381694 +0000 UTC m=+0.330220618 container init 45a3c77739e9f55fdb2d9dc8421be8b0f517f3f627872bcc3b1d2bd066776095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5) 2024-04-22T07:07:04.119 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 podman[33701]: 2024-04-22 07:07:04.048868027 +0000 UTC m=+0.332706952 container start 45a3c77739e9f55fdb2d9dc8421be8b0f517f3f627872bcc3b1d2bd066776095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-22T07:07:04.119 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-mgr[33720]: -- 172.21.15.154:0/1500290807 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598e37885a0 con 0x5598e375c800 2024-04-22T07:07:04.299 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T07:07:04.299 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T07:07:04.299 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-22T07:07:04.299 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-22T07:07:04.299 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00862779 s, 1.2 GB/s 2024-04-22T07:07:04.299 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-22T07:07:04.394 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-mon[31862]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:04.395 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 bash[33701]: 45a3c77739e9f55fdb2d9dc8421be8b0f517f3f627872bcc3b1d2bd066776095 2024-04-22T07:07:04.395 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 systemd[1]: Started Ceph mgr.z for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:07:04.395 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:04.211+0000 7f0314fb8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:07:04.395 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:04.396+0000 7f0314fb8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:07:04.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:04 smithi133 ceph-mon[31198]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:04.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:04 smithi113 ceph-mon[27933]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:04.804 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:04.499+0000 7f0314fb8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:07:04.879 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_4 2024-04-22T07:07:05.066 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:04.807+0000 7f0314fb8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:07:05.066 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:04.891+0000 7f0314fb8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:07:05.114 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:07:05.492 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:05.493 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: Deploying daemon mgr.y on smithi133 2024-04-22T07:07:05.494 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:05 smithi154 ceph-mon[31862]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:05.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:05.541 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: Deploying daemon mgr.y on smithi133 2024-04-22T07:07:05.541 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:05 smithi113 ceph-mon[27933]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: Deploying daemon mgr.y on smithi133 2024-04-22T07:07:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:05 smithi133 ceph-mon[31198]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:06.448 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:06.118+0000 7f0314fb8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:07:06.448 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:06.288+0000 7f0314fb8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:07:06.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 podman[32567]: 2024-04-22 07:07:06.432131157 +0000 UTC m=+0.018069725 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:07:06.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- 172.21.15.113:0/1759339048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078103d90 msgr2=0x7fe0781041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:06.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/1759339048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078103d90 0x7fe0781041f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe068009990 tx=0x7fe068009ca0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- 172.21.15.113:0/1759339048 shutdown_connections 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/1759339048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe078104730 0x7fe07810afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/1759339048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe078100db0 0x7fe0781011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/1759339048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078103d90 0x7fe0781041f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- 172.21.15.113:0/1759339048 >> 172.21.15.113:0/1759339048 conn(0x7fe0780fc710 msgr2=0x7fe0780feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- 172.21.15.113:0/1759339048 shutdown_connections 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- 172.21.15.113:0/1759339048 wait complete. 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 Processor -- start 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- start start 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 0x7fe078198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe078103d90 0x7fe0781991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe078104730 0x7fe0781a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e690 con 0x7fe078100db0 2024-04-22T07:07:06.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e510 con 0x7fe078104730 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.616+0000 7fe07f3e5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e810 con 0x7fe078103d90 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 0x7fe078198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 0x7fe078198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55782/0 (socket says 172.21.15.113:55782) 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 -- 172.21.15.113:0/166886026 learned_addr learned my addr 172.21.15.113:0/166886026 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07c980700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe078103d90 0x7fe0781991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d982700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe078104730 0x7fe0781a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 -- 172.21.15.113:0/166886026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe078103d90 msgr2=0x7fe0781991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe078103d90 0x7fe0781991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 -- 172.21.15.113:0/166886026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe078104730 msgr2=0x7fe0781a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe078104730 0x7fe0781a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 -- 172.21.15.113:0/166886026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0781a08f0 con 0x7fe078100db0 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07d181700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 0x7fe078198c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe07400d980 tx=0x7fe07400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074017030 con 0x7fe078100db0 2024-04-22T07:07:06.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe0740044f0 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0781a0be0 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074003a00 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.617+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a1140 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.618+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fe07401a070 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.618+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe078069730 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.618+0000 7fe06e7fc700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe064039890 0x7fe06403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.618+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe07404c240 con 0x7fe078100db0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.618+0000 7fe07c980700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe064039890 0x7fe06403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:06.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.619+0000 7fe07c980700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe064039890 0x7fe06403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe06800b5c0 tx=0x7fe068005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:07:06.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.623+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe07400e3b0 con 0x7fe078100db0 2024-04-22T07:07:06.749 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:06.463+0000 7f0314fb8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:07:06.749 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:06.751+0000 7f0314fb8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:07:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:06.832+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe078062e80 con 0x7fe064039890 2024-04-22T07:07:06.873 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 podman[32567]: 2024-04-22 07:07:06.72134828 +0000 UTC m=+0.307286838 container init 2e7396d59a130640786ff1cda5b1495df59cd106369799d7998644d26457873c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD) 2024-04-22T07:07:06.873 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 podman[32567]: 2024-04-22 07:07:06.72381843 +0000 UTC m=+0.309756986 container start 2e7396d59a130640786ff1cda5b1495df59cd106369799d7998644d26457873c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T07:07:06.873 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-mgr[32586]: -- 172.21.15.133:0/929765494 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562cf98525a0 con 0x562cf9827000 2024-04-22T07:07:06.873 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 bash[32567]: 2e7396d59a130640786ff1cda5b1495df59cd106369799d7998644d26457873c 2024-04-22T07:07:06.873 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 systemd[1]: Started Ceph mgr.y for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:07:07.051 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:06.844+0000 7f0314fb8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:07:07.051 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:07.051 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:07.051 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:07.051 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:07.051 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:06 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:06 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:06 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:06 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:06 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:06 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:07.170 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:07.170 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:07.170 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:07.170 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:07.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:07.171 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:06.874+0000 7f7e2cba6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:07:07.171 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:07.063+0000 7f7e2cba6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:07:07.171 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:07.172+0000 7f7e2cba6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:07:07.316 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:07.052+0000 7f0314fb8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:07:07.316 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:07.254+0000 7f0314fb8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:07:07.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:07.501+0000 7f7e2cba6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:07:07.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:07.600+0000 7f7e2cba6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:07:08.000 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:07 smithi113 ceph-mon[27933]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:08.001 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:08.001 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:08.001 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:07 smithi113 ceph-mon[27933]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:08.001 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-mon[31198]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-mon[31198]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:08.315 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:07.903+0000 7f0314fb8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:07:08.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-mon[31862]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:08.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:08.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:08.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-mon[31862]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:08.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:08.636 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:08.338+0000 7f0314fb8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:07:08.636 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:08.435+0000 7f0314fb8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:07:08.636 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:08.638+0000 7f0314fb8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:07:09.175 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:08.839+0000 7f7e2cba6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:07:09.175 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:09.008+0000 7f7e2cba6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:07:09.175 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:09.178+0000 7f7e2cba6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:09.462 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:09.463 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:09.463 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: Reconfiguring daemon mgr.x on smithi113 2024-04-22T07:07:09.463 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-mon[31198]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:09.486 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:09.155+0000 7f0314fb8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:07:09.486 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:09.248+0000 7f0314fb8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:07:09.486 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.486 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: Reconfiguring daemon mgr.x on smithi113 2024-04-22T07:07:09.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-mon[31862]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:09.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: Reconfiguring daemon mgr.x on smithi113 2024-04-22T07:07:09.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:09 smithi113 ceph-mon[27933]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:09.757 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:09.464+0000 7f7e2cba6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:07:09.757 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:09.556+0000 7f7e2cba6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:07:09.757 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:09.759+0000 7f7e2cba6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:07:09.816 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:09.488+0000 7f0314fb8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:07:09.816 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:09.620+0000 7f0314fb8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:07:09.816 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:09.714+0000 7f0314fb8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:07:10.064 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:09.949+0000 7f7e2cba6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:07:10.129 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:09.829+0000 7f0314fb8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:07:10.129 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:10.131+0000 7f0314fb8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:07:10.722 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1532123127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b292ee-908e-46a7-8b87-733e426fe61b"}]: dispatch 2024-04-22T07:07:10.723 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1532123127' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b292ee-908e-46a7-8b87-733e426fe61b"}]': finished 2024-04-22T07:07:10.723 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-mon[31862]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T07:07:10.723 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:10.723 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3355390520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:07:10.723 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:10.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:10.544+0000 7f7e2cba6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:07:10.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1532123127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b292ee-908e-46a7-8b87-733e426fe61b"}]: dispatch 2024-04-22T07:07:10.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1532123127' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b292ee-908e-46a7-8b87-733e426fe61b"}]': finished 2024-04-22T07:07:10.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-mon[31198]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T07:07:10.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:10.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3355390520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:07:10.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:10.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:10 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1532123127' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b292ee-908e-46a7-8b87-733e426fe61b"}]: dispatch 2024-04-22T07:07:10.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:10 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1532123127' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b292ee-908e-46a7-8b87-733e426fe61b"}]': finished 2024-04-22T07:07:10.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:10 smithi113 ceph-mon[27933]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T07:07:10.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:10.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:10 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3355390520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:07:10.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:11.066 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:10.724+0000 7f0314fb8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:07:11.066 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:07:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:07:10.937+0000 7f0314fb8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:07:11.259 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:10.964+0000 7f7e2cba6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:07:11.259 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:11.058+0000 7f7e2cba6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:07:11.259 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:11.258+0000 7f7e2cba6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:07:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:11.404+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7fe074003ba0 con 0x7fe078100db0 2024-04-22T07:07:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-mon[31198]: Standby manager daemon z started 2024-04-22T07:07:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T07:07:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:07:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T07:07:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:07:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-mon[31198]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:11 smithi154 ceph-mon[31862]: Standby manager daemon z started 2024-04-22T07:07:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:11 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T07:07:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:11 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:07:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:11 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T07:07:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:11 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:07:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:11 smithi154 ceph-mon[31862]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:11.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:11 smithi113 ceph-mon[27933]: Standby manager daemon z started 2024-04-22T07:07:11.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:11 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T07:07:11.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:11 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:07:11.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:11 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T07:07:11.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:11 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/66586827' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:07:11.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:11 smithi113 ceph-mon[27933]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:12.064 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:11.791+0000 7f7e2cba6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:07:12.064 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:11.882+0000 7f7e2cba6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:07:12.401 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:12.122+0000 7f7e2cba6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:07:12.401 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:12.252+0000 7f7e2cba6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:07:12.401 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:12.345+0000 7f7e2cba6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:07:12.782 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-mon[31198]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-22T07:07:12.782 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T07:07:12.783 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:12.459+0000 7f7e2cba6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:07:12.783 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:12.775+0000 7f7e2cba6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:07:12.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:12 smithi154 ceph-mon[31862]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-22T07:07:12.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:12 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T07:07:12.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:12 smithi113 ceph-mon[27933]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-22T07:07:12.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:12 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T07:07:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:13 smithi133 ceph-mon[31198]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:13.815 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:13.367+0000 7f7e2cba6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:07:13.815 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:07:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:07:13.579+0000 7f7e2cba6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:07:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:13 smithi154 ceph-mon[31862]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:13 smithi113 ceph-mon[27933]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:14.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:14.423+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mon.0 v2:172.21.15.113:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fe07400e5d0 con 0x7fe078100db0 2024-04-22T07:07:14.713 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:14 smithi113 ceph-mon[27933]: Standby manager daemon y started 2024-04-22T07:07:14.713 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:14 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T07:07:14.713 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:14 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:07:14.713 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:14 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T07:07:14.713 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:14 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:07:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:14 smithi133 ceph-mon[31198]: Standby manager daemon y started 2024-04-22T07:07:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:14 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T07:07:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:14 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:07:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:14 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T07:07:14.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:14 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:07:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:14 smithi154 ceph-mon[31862]: Standby manager daemon y started 2024-04-22T07:07:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:14 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T07:07:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:14 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:07:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:14 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T07:07:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:14 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/843699666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:15.537 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:15.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:15.538 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-22T07:07:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T07:07:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:15.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:15.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:15.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:18 smithi133 ceph-mon[31198]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:18 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T07:07:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:18 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:18.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:18 smithi154 ceph-mon[31862]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:18.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:18 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T07:07:18.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:18 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:18 smithi113 ceph-mon[27933]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T07:07:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:19.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:19 smithi133 ceph-mon[31198]: Deploying daemon osd.0 on smithi113 2024-04-22T07:07:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:19 smithi154 ceph-mon[31862]: Deploying daemon osd.0 on smithi113 2024-04-22T07:07:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:19 smithi113 ceph-mon[27933]: Deploying daemon osd.0 on smithi113 2024-04-22T07:07:20.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:20 smithi133 ceph-mon[31198]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:20.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:20 smithi154 ceph-mon[31862]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:20.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:20 smithi113 ceph-mon[27933]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:22.144 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:22 smithi113 ceph-mon[27933]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:22.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:22 smithi154 ceph-mon[31862]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:22.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:22 smithi133 ceph-mon[31198]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:24.174 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:24.174 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:24.174 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:24.174 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:23 smithi113 ceph-mon[27933]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:24.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:24.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:24.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:24.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:23 smithi133 ceph-mon[31198]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:24.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:24.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:24.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:24.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:23 smithi154 ceph-mon[31862]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:26.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:26 smithi113 ceph-mon[27933]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:26.300 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:26.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:26.301 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:26 smithi133 ceph-mon[31198]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:26 smithi154 ceph-mon[31862]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:26.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.643+0000 7fe06e7fc700 1 -- 172.21.15.113:0/166886026 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe078062e80 con 0x7fe064039890 2024-04-22T07:07:26.642 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 0 on host 'smithi113' 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe064039890 msgr2=0x7fe06403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe064039890 0x7fe06403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe06800b5c0 tx=0x7fe068005fb0 comp rx=0 tx=0).stop 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 msgr2=0x7fe078198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 0x7fe078198c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe07400d980 tx=0x7fe07400dc90 comp rx=0 tx=0).stop 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 shutdown_connections 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe064039890 0x7fe06403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe078104730 0x7fe0781a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe078100db0 0x7fe078198c70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.645+0000 7fe07f3e5700 1 --2- 172.21.15.113:0/166886026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe078103d90 0x7fe0781991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.646+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 >> 172.21.15.113:0/166886026 conn(0x7fe0780fc710 msgr2=0x7fe078107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.646+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 shutdown_connections 2024-04-22T07:07:26.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:07:26.646+0000 7fe07f3e5700 1 -- 172.21.15.113:0/166886026 wait complete. 2024-04-22T07:07:27.227 DEBUG:teuthology.orchestra.run.smithi113:osd.0> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.0.service 2024-04-22T07:07:27.229 INFO:tasks.cephadm:Deploying osd.1 on smithi133 with /dev/vg_nvme/lv_4... 2024-04-22T07:07:27.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T07:07:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:07:27.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:27.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:27.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:27.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:27 smithi113 ceph-mon[27933]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T07:07:27.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:27 smithi113 ceph-mon[27933]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:27.907 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:27 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:27.908 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:27 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:27.908 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:27 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:27.908 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:27 smithi133 ceph-mon[31198]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T07:07:27.908 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:27 smithi133 ceph-mon[31198]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:28.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:27 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:28.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:27 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:28.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:27 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:28.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:27 smithi154 ceph-mon[31862]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T07:07:28.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:27 smithi154 ceph-mon[31862]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:28.755 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:28 smithi113 ceph-mon[27933]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T07:07:28.755 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:28 smithi113 ceph-mon[27933]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T07:07:28.755 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:28 smithi113 ceph-mon[27933]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:07:28.755 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:28.755 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:07:28 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0[40027]: 2024-04-22T07:07:28.615+0000 7f9613273700 -1 osd.0 0 waiting for initial osdmap 2024-04-22T07:07:28.755 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:07:28 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0[40027]: 2024-04-22T07:07:28.618+0000 7f960f066700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:07:28.789 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:28 smithi133 ceph-mon[31198]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T07:07:28.789 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:28 smithi133 ceph-mon[31198]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T07:07:28.789 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:28 smithi133 ceph-mon[31198]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:07:28.789 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:29.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:28 smithi154 ceph-mon[31862]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T07:07:29.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:28 smithi154 ceph-mon[31862]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T07:07:29.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:28 smithi154 ceph-mon[31862]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:07:29.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:29.416 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T07:07:29.416 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T07:07:29.416 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-22T07:07:29.416 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-22T07:07:29.416 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01156 s, 907 MB/s 2024-04-22T07:07:29.416 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-22T07:07:29.975 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:29 smithi113 ceph-mon[27933]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:07:29.976 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:29 smithi113 ceph-mon[27933]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T07:07:29.976 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:29 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:29.976 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:29 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:29.976 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:29 smithi113 ceph-mon[27933]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:30.020 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:29 smithi133 ceph-mon[31198]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:07:30.020 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:29 smithi133 ceph-mon[31198]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T07:07:30.020 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:29 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:30.020 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:29 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:30.020 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:29 smithi133 ceph-mon[31198]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:30.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-04-22T07:07:30.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:29 smithi154 ceph-mon[31862]: from='osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:07:30.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:29 smithi154 ceph-mon[31862]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T07:07:30.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:29 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:30.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:29 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:30.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:29 smithi154 ceph-mon[31862]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T07:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:07:30.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:07:30.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:07:30.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:30.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] boot 2024-04-22T07:07:30.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi113 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi113 to 17088M 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:30.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:07:31.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] boot 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi113 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi113 to 17088M 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:31.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:31.068 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:07:31.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: osd.0 [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] boot 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi113 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi113 to 17088M 2024-04-22T07:07:31.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.086 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:31.086 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:31.086 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:31.772 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:31 smithi133 ceph-mon[31198]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T07:07:31.772 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:31 smithi133 ceph-mon[31198]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 -- 172.21.15.133:0/3580270605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c103c90 msgr2=0x7f638c1040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 --2- 172.21.15.133:0/3580270605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c103c90 0x7f638c1040f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f63880099c0 tx=0x7f6388009cd0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 -- 172.21.15.133:0/3580270605 shutdown_connections 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 --2- 172.21.15.133:0/3580270605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c104630 0x7f638c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 --2- 172.21.15.133:0/3580270605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c102a90 0x7f638c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 --2- 172.21.15.133:0/3580270605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c103c90 0x7f638c1040f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.802+0000 7f6393937700 1 -- 172.21.15.133:0/3580270605 >> 172.21.15.133:0/3580270605 conn(0x7f638c0fe030 msgr2=0x7f638c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 -- 172.21.15.133:0/3580270605 shutdown_connections 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 -- 172.21.15.133:0/3580270605 wait complete. 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 Processor -- start 2024-04-22T07:07:31.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 -- start start 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 0x7f638c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c103c90 0x7f638c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c104630 0x7f638c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f638c10e670 con 0x7f638c104630 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f638c10e4f0 con 0x7f638c102a90 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f6393937700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f638c10e7f0 con 0x7f638c103c90 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f63916d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 0x7f638c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f63916d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 0x7f638c198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55812/0 (socket says 172.21.15.133:55812) 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.803+0000 7f63916d3700 1 -- 172.21.15.133:0/4228755390 learned_addr learned my addr 172.21.15.133:0/4228755390 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f6390ed2700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c103c90 0x7f638c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f6391ed4700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c104630 0x7f638c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63916d3700 1 -- 172.21.15.133:0/4228755390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c103c90 msgr2=0x7f638c199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:31.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63916d3700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c103c90 0x7f638c199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63916d3700 1 -- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c104630 msgr2=0x7f638c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63916d3700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c104630 0x7f638c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63916d3700 1 -- 172.21.15.133:0/4228755390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c1a0860 con 0x7f638c102a90 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f6390ed2700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c103c90 0x7f638c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f6391ed4700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c104630 0x7f638c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63916d3700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 0x7f638c198c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f637c00c990 tx=0x7f637c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c007b00 con 0x7f638c102a90 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f637c007ca0 con 0x7f638c102a90 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c1a0b50 con 0x7f638c102a90 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0186f0 con 0x7f638c102a90 2024-04-22T07:07:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.804+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c1a1090 con 0x7f638c102a90 2024-04-22T07:07:31.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.805+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638c069730 con 0x7f638c102a90 2024-04-22T07:07:31.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.806+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f637c018890 con 0x7f638c102a90 2024-04-22T07:07:31.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.807+0000 7f63827fc700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f63780a3bb0 0x7f63780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:31.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.807+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f637c0ce9a0 con 0x7f638c102a90 2024-04-22T07:07:31.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.807+0000 7f6390ed2700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f63780a3bb0 0x7f63780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:31.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.807+0000 7f6390ed2700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f63780a3bb0 0x7f63780a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6388006010 tx=0x7f63880038d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:07:31.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:31.811+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f637c014030 con 0x7f638c102a90 2024-04-22T07:07:32.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:32.015+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f638c062e80 con 0x7f63780a3bb0 2024-04-22T07:07:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:31 smithi154 ceph-mon[31862]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T07:07:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:31 smithi154 ceph-mon[31862]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:31 smithi113 ceph-mon[27933]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T07:07:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:31 smithi113 ceph-mon[27933]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:32.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:32 smithi133 ceph-mon[31198]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:32.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:32.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:32.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:33.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:32 smithi154 ceph-mon[31862]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:33.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:33.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:33.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:32 smithi113 ceph-mon[27933]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:33.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:33 smithi133 ceph-mon[31198]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:34.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:33 smithi154 ceph-mon[31862]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:33 smithi113 ceph-mon[27933]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:35.037 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:34 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/2178641286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]: dispatch 2024-04-22T07:07:35.037 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:34 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]: dispatch 2024-04-22T07:07:35.037 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:34 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]': finished 2024-04-22T07:07:35.037 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:34 smithi133 ceph-mon[31198]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T07:07:35.037 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:35.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:34 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/2178641286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]: dispatch 2024-04-22T07:07:35.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:34 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]: dispatch 2024-04-22T07:07:35.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:34 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]': finished 2024-04-22T07:07:35.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:34 smithi154 ceph-mon[31862]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T07:07:35.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:34 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/2178641286' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]: dispatch 2024-04-22T07:07:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:34 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]: dispatch 2024-04-22T07:07:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:34 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fed6e4a7-b083-483a-b8ec-2b567c9f6e92"}]': finished 2024-04-22T07:07:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:34 smithi113 ceph-mon[27933]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T07:07:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:36.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:35 smithi133 ceph-mon[31198]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:36.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:35 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/1293016625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:07:36.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:35 smithi154 ceph-mon[31862]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:36.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:35 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/1293016625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:07:36.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:35 smithi113 ceph-mon[27933]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:36.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:35 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/1293016625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:07:38.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:38 smithi133 ceph-mon[31198]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:38.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:38 smithi154 ceph-mon[31862]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:38 smithi113 ceph-mon[27933]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:40.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:40 smithi133 ceph-mon[31198]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:40.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:40 smithi154 ceph-mon[31862]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:40.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:40 smithi113 ceph-mon[27933]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:42 smithi133 ceph-mon[31198]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T07:07:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:42 smithi113 ceph-mon[27933]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T07:07:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:42 smithi154 ceph-mon[31862]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T07:07:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:43.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:43 smithi113 ceph-mon[27933]: Deploying daemon osd.1 on smithi133 2024-04-22T07:07:43.378 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:43 smithi133 ceph-mon[31198]: Deploying daemon osd.1 on smithi133 2024-04-22T07:07:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:43 smithi154 ceph-mon[31862]: Deploying daemon osd.1 on smithi133 2024-04-22T07:07:44.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:44 smithi133 ceph-mon[31198]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:44.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:44 smithi113 ceph-mon[27933]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:44 smithi154 ceph-mon[31862]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:46.300 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:46 smithi133 ceph-mon[31198]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:46.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:46 smithi113 ceph-mon[27933]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:46 smithi154 ceph-mon[31862]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:48.169 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:48 smithi133 ceph-mon[31198]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:48.169 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:48.169 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:48.170 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:48 smithi113 ceph-mon[27933]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:48 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:48 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:48 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:48 smithi154 ceph-mon[31862]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:48 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:48 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:48 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:50 smithi113 ceph-mon[27933]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.376 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:50 smithi133 ceph-mon[31198]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:50.376 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.376 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.376 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:50.376 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:50.376 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:50 smithi154 ceph-mon[31862]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.730+0000 7f63827fc700 1 -- 172.21.15.133:0/4228755390 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f638c062e80 con 0x7f63780a3bb0 2024-04-22T07:07:50.729 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 1 on host 'smithi133' 2024-04-22T07:07:50.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f63780a3bb0 msgr2=0x7f63780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:50.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f63780a3bb0 0x7f63780a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6388006010 tx=0x7f63880038d0 comp rx=0 tx=0).stop 2024-04-22T07:07:50.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 msgr2=0x7f638c198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:50.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 0x7f638c198c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f637c00c990 tx=0x7f637c00cd50 comp rx=0 tx=0).stop 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 shutdown_connections 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f63780a3bb0 0x7f63780a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f638c104630 0x7f638c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638c102a90 0x7f638c198c10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 --2- 172.21.15.133:0/4228755390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f638c103c90 0x7f638c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 >> 172.21.15.133:0/4228755390 conn(0x7f638c0fe030 msgr2=0x7f638c106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 shutdown_connections 2024-04-22T07:07:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:07:50.733+0000 7f6393937700 1 -- 172.21.15.133:0/4228755390 wait complete. 2024-04-22T07:07:51.386 DEBUG:teuthology.orchestra.run.smithi133:osd.1> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.1.service 2024-04-22T07:07:51.389 INFO:tasks.cephadm:Deploying osd.2 on smithi154 with /dev/vg_nvme/lv_4... 2024-04-22T07:07:51.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T07:07:51.633 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:07:51.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:51.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:51.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:51.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:51 smithi154 ceph-mon[31862]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:51.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:51 smithi154 ceph-mon[31862]: from='osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T07:07:51.978 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:51 smithi154 ceph-mon[31862]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T07:07:52.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:52.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:52.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:52.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:51 smithi133 ceph-mon[31198]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:52.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:51 smithi133 ceph-mon[31198]: from='osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T07:07:52.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:51 smithi133 ceph-mon[31198]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T07:07:52.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:52.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:52.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:07:52.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:51 smithi113 ceph-mon[27933]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:52.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:51 smithi113 ceph-mon[27933]: from='osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T07:07:52.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:51 smithi113 ceph-mon[27933]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T07:07:52.752 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:52 smithi133 ceph-mon[31198]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T07:07:52.752 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:52 smithi133 ceph-mon[31198]: from='osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:07:52.753 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:52 smithi133 ceph-mon[31198]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T07:07:52.753 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:52.753 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:52 smithi133 ceph-mon[31198]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:07:52.956 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:52 smithi154 ceph-mon[31862]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T07:07:52.956 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:52 smithi154 ceph-mon[31862]: from='osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:07:52.956 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:52 smithi154 ceph-mon[31862]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T07:07:52.957 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:52.957 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:52 smithi154 ceph-mon[31862]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:07:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:52 smithi113 ceph-mon[27933]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T07:07:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:52 smithi113 ceph-mon[27933]: from='osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:07:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:52 smithi113 ceph-mon[27933]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T07:07:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:52 smithi113 ceph-mon[27933]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:07:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T07:07:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T07:07:53.506 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-22T07:07:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-22T07:07:53.506 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112261 s, 934 MB/s 2024-04-22T07:07:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-22T07:07:53.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-mon[31198]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:07:53.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-mon[31198]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T07:07:53.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:53.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:53.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-mon[31198]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:53.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-mon[31198]: from='osd.1 ' entity='osd.1' 2024-04-22T07:07:53.974 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1[35803]: 2024-04-22T07:07:53.609+0000 7ff38dec9700 -1 osd.1 0 waiting for initial osdmap 2024-04-22T07:07:53.975 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:07:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1[35803]: 2024-04-22T07:07:53.791+0000 7ff385cb4700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:07:53.985 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:53 smithi113 ceph-mon[27933]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:07:53.986 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:53 smithi113 ceph-mon[27933]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T07:07:53.986 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:53.986 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:53.986 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:53 smithi113 ceph-mon[27933]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:53.986 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:53 smithi113 ceph-mon[27933]: from='osd.1 ' entity='osd.1' 2024-04-22T07:07:54.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:53 smithi154 ceph-mon[31862]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:07:54.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:53 smithi154 ceph-mon[31862]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T07:07:54.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:54.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:54.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:53 smithi154 ceph-mon[31862]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:54.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:53 smithi154 ceph-mon[31862]: from='osd.1 ' entity='osd.1' 2024-04-22T07:07:54.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_4 2024-04-22T07:07:54.349 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:07:54.773 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:07:54.773 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:07:54.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:54.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:54.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:54.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:07:54.774 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:54.775 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:54.775 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:54.775 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:54.775 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:55.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:55.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:07:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:07:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:07:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:55 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi133 2024-04-22T07:07:55.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:55 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi133 to 17088M 2024-04-22T07:07:55.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:55 smithi113 ceph-mon[27933]: osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] boot 2024-04-22T07:07:55.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:55 smithi113 ceph-mon[27933]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T07:07:55.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:55 smithi113 ceph-mon[27933]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:55.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 -- 172.21.15.154:0/2038548877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c104640 msgr2=0x7fad6c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:55.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 --2- 172.21.15.154:0/2038548877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c104640 0x7fad6c10aef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fad6400b0a0 tx=0x7fad6400b3b0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 -- 172.21.15.154:0/2038548877 shutdown_connections 2024-04-22T07:07:55.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 --2- 172.21.15.154:0/2038548877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c104640 0x7fad6c10aef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 --2- 172.21.15.154:0/2038548877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c102aa0 0x7fad6c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 --2- 172.21.15.154:0/2038548877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c103ca0 0x7fad6c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 -- 172.21.15.154:0/2038548877 >> 172.21.15.154:0/2038548877 conn(0x7fad6c0fe040 msgr2=0x7fad6c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:07:55.842 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:55 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi133 2024-04-22T07:07:55.842 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:55 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi133 to 17088M 2024-04-22T07:07:55.842 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:55 smithi154 ceph-mon[31862]: osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] boot 2024-04-22T07:07:55.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:55 smithi154 ceph-mon[31862]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T07:07:55.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:55.843 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:55 smithi154 ceph-mon[31862]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 -- 172.21.15.154:0/2038548877 shutdown_connections 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.844+0000 7fad744a3700 1 -- 172.21.15.154:0/2038548877 wait complete. 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 Processor -- start 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 -- start start 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c102aa0 0x7fad6c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 0x7fad6c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c104640 0x7fad6c1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10e660 con 0x7fad6c102aa0 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10e4e0 con 0x7fad6c104640 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad744a3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10e7e0 con 0x7fad6c103ca0 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 0x7fad6c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad7223f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c102aa0 0x7fad6c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:55.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 0x7fad6c1990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49286/0 (socket says 172.21.15.154:49286) 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 -- 172.21.15.154:0/2658330390 learned_addr learned my addr 172.21.15.154:0/2658330390 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad72a40700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c104640 0x7fad6c1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 -- 172.21.15.154:0/2658330390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c104640 msgr2=0x7fad6c1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c104640 0x7fad6c1a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 -- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c102aa0 msgr2=0x7fad6c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c102aa0 0x7fad6c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad71a3e700 1 -- 172.21.15.154:0/2658330390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c1a07e0 con 0x7fad6c103ca0 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad72a40700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c104640 0x7fad6c1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.845+0000 7fad7223f700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c102aa0 0x7fad6c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.846+0000 7fad71a3e700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 0x7fad6c1990c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00e9d0 tx=0x7fad5c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.846+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c00cbd0 con 0x7fad6c103ca0 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.846+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c1a0ad0 con 0x7fad6c103ca0 2024-04-22T07:07:55.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.846+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad5c00cd70 con 0x7fad6c103ca0 2024-04-22T07:07:55.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.846+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c018710 con 0x7fad6c103ca0 2024-04-22T07:07:55.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.846+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c1a1060 con 0x7fad6c103ca0 2024-04-22T07:07:55.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.847+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c069730 con 0x7fad6c103ca0 2024-04-22T07:07:55.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.848+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fad5c010430 con 0x7fad6c103ca0 2024-04-22T07:07:55.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.849+0000 7fad637fe700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fad580a3c30 0x7fad580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:55.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.849+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fad5c0cf1b0 con 0x7fad6c103ca0 2024-04-22T07:07:55.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.849+0000 7fad7223f700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fad580a3c30 0x7fad580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:07:55.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.849+0000 7fad7223f700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fad580a3c30 0x7fad580a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad6c103b00 tx=0x7fad6800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:07:55.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:55.853+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad5c09d3e0 con 0x7fad6c103ca0 2024-04-22T07:07:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:07:56.060+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fad6c062f40 con 0x7fad580a3c30 2024-04-22T07:07:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:55 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi133 2024-04-22T07:07:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:55 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi133 to 17088M 2024-04-22T07:07:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:55 smithi133 ceph-mon[31198]: osd.1 [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] boot 2024-04-22T07:07:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:55 smithi133 ceph-mon[31198]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T07:07:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:07:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:55 smithi133 ceph-mon[31198]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T07:07:56.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:56 smithi154 ceph-mon[31862]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T07:07:56.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:56 smithi154 ceph-mon[31862]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:56.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:56.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:56.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:56 smithi133 ceph-mon[31198]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T07:07:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:56 smithi133 ceph-mon[31198]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:56 smithi113 ceph-mon[27933]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T07:07:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:56 smithi113 ceph-mon[27933]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:07:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:07:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:07:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:07:57.881 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:57 smithi154 ceph-mon[31862]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:07:58.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:57 smithi133 ceph-mon[31198]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:07:58.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:57 smithi113 ceph-mon[27933]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:07:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:58 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/2115680161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]: dispatch 2024-04-22T07:07:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:58 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]: dispatch 2024-04-22T07:07:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:58 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]': finished 2024-04-22T07:07:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:58 smithi133 ceph-mon[31198]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T07:07:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:07:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:58 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/2115680161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]: dispatch 2024-04-22T07:07:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:58 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]: dispatch 2024-04-22T07:07:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:58 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]': finished 2024-04-22T07:07:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:58 smithi154 ceph-mon[31862]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T07:07:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:07:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:58 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/2115680161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]: dispatch 2024-04-22T07:07:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:58 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]: dispatch 2024-04-22T07:07:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:58 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed142201-490b-4bd4-8fe4-5aeaa334c18a"}]': finished 2024-04-22T07:07:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:58 smithi113 ceph-mon[27933]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T07:07:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:00.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:59 smithi133 ceph-mon[31198]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:00.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:07:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/2843631344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:00.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:59 smithi154 ceph-mon[31862]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:00.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:07:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/2843631344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:00.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:59 smithi113 ceph-mon[27933]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:00.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:07:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/2843631344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:02.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:02 smithi154 ceph-mon[31862]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:02 smithi113 ceph-mon[27933]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:02 smithi133 ceph-mon[31198]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:04.246 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:04 smithi154 ceph-mon[31862]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:04 smithi113 ceph-mon[27933]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:04.366 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:04 smithi133 ceph-mon[31198]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:06.221 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:06 smithi154 ceph-mon[31862]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:06.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:06 smithi113 ceph-mon[27933]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:06.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:06 smithi133 ceph-mon[31198]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:07.241 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T07:08:07.241 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:07.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T07:08:07.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:07.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T07:08:07.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:08.330 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:08 smithi154 ceph-mon[31862]: Deploying daemon osd.2 on smithi154 2024-04-22T07:08:08.331 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:08 smithi154 ceph-mon[31862]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:08.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:08 smithi113 ceph-mon[27933]: Deploying daemon osd.2 on smithi154 2024-04-22T07:08:08.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:08 smithi113 ceph-mon[27933]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:08.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:08 smithi133 ceph-mon[31198]: Deploying daemon osd.2 on smithi154 2024-04-22T07:08:08.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:08 smithi133 ceph-mon[31198]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:10.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:10 smithi154 ceph-mon[31862]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:10.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:10 smithi113 ceph-mon[27933]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:10 smithi133 ceph-mon[31198]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:12.337 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:12 smithi154 ceph-mon[31862]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:12.337 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:12 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:12.337 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:12 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:12.337 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:12 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:12 smithi133 ceph-mon[31198]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:12 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:12 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:12 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:12.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:12 smithi113 ceph-mon[27933]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:12.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:12 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:12.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:12 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:12.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:12 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:14.298 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:14 smithi154 ceph-mon[31862]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:14.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:14 smithi133 ceph-mon[31198]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:14 smithi113 ceph-mon[27933]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:15.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.326+0000 7fad637fe700 1 -- 172.21.15.154:0/2658330390 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fad6c062f40 con 0x7fad580a3c30 2024-04-22T07:08:15.325 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 2 on host 'smithi154' 2024-04-22T07:08:15.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fad580a3c30 msgr2=0x7fad580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:15.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fad580a3c30 0x7fad580a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fad6c103b00 tx=0x7fad6800a3b0 comp rx=0 tx=0).stop 2024-04-22T07:08:15.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 msgr2=0x7fad6c1990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 0x7fad6c1990c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00e9d0 tx=0x7fad5c00ece0 comp rx=0 tx=0).stop 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 shutdown_connections 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fad580a3c30 0x7fad580a60f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad6c104640 0x7fad6c1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad6c102aa0 0x7fad6c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 --2- 172.21.15.154:0/2658330390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad6c103ca0 0x7fad6c1990c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 >> 172.21.15.154:0/2658330390 conn(0x7fad6c0fe040 msgr2=0x7fad6c106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 shutdown_connections 2024-04-22T07:08:15.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:08:15.328+0000 7fad744a3700 1 -- 172.21.15.154:0/2658330390 wait complete. 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:15.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:15.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:15 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:15.620 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:15 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:15.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:15 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:15.955 DEBUG:teuthology.orchestra.run.smithi154:osd.2> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.2.service 2024-04-22T07:08:15.957 INFO:tasks.cephadm:Deploying osd.3 on smithi113 with /dev/vg_nvme/lv_3... 2024-04-22T07:08:15.957 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T07:08:16.204 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:08:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:16 smithi154 ceph-mon[31862]: from='osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T07:08:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:16 smithi154 ceph-mon[31862]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T07:08:16.577 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:16 smithi113 ceph-mon[27933]: from='osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T07:08:16.577 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:16 smithi113 ceph-mon[27933]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T07:08:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:16 smithi133 ceph-mon[31198]: from='osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T07:08:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:16 smithi133 ceph-mon[31198]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T07:08:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:17 smithi113 ceph-mon[27933]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T07:08:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:17 smithi113 ceph-mon[27933]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T07:08:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:17 smithi113 ceph-mon[27933]: from='osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:08:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:17 smithi113 ceph-mon[27933]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:08:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:17 smithi113 ceph-mon[27933]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:17 smithi133 ceph-mon[31198]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T07:08:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:17 smithi133 ceph-mon[31198]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T07:08:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:17 smithi133 ceph-mon[31198]: from='osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:08:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:17 smithi133 ceph-mon[31198]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:08:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:17 smithi133 ceph-mon[31198]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:17.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-mon[31862]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T07:08:17.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-mon[31862]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T07:08:17.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:17.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-mon[31862]: from='osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:08:17.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-mon[31862]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:08:17.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-mon[31862]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:17.816 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2[36997]: 2024-04-22T07:08:17.396+0000 7f4d65233700 -1 osd.2 0 waiting for initial osdmap 2024-04-22T07:08:17.816 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2[36997]: 2024-04-22T07:08:17.401+0000 7f4d61026700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:08:18.211 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T07:08:18.211 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T07:08:18.211 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-22T07:08:18.211 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-22T07:08:18.211 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00818466 s, 1.3 GB/s 2024-04-22T07:08:18.211 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-22T07:08:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:18 smithi154 ceph-mon[31862]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:08:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:18 smithi154 ceph-mon[31862]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T07:08:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:18 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:18 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:18 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:18 smithi113 ceph-mon[27933]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:08:18.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:18 smithi113 ceph-mon[27933]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T07:08:18.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.792 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.792 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:18 smithi133 ceph-mon[31198]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:08:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:18 smithi133 ceph-mon[31198]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T07:08:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:18 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:18 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:18 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:18.819 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_3 2024-04-22T07:08:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] boot 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi154 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi154 to 17088M 2024-04-22T07:08:19.478 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:19.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:19.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:19.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:08:19.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:08:19.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] boot 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi154 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi154 to 17088M 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:19.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: osd.2 [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] boot 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi154 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi154 to 17088M 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:19.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:19.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:19.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-22T07:08:19.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:08:20.533 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T07:08:20.534 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 ceph-mon[27933]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T07:08:20.534 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:08:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 -- 172.21.15.113:0/3041274765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 msgr2=0x7fd6bc103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/3041274765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc103000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0099c0 tx=0x7fd6ac009cd0 comp rx=0 tx=0).stop 2024-04-22T07:08:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 -- 172.21.15.113:0/3041274765 shutdown_connections 2024-04-22T07:08:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/3041274765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6bc103540 0x7fd6bc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/3041274765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 0x7fd6bc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:20.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/3041274765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc103000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:20.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 -- 172.21.15.113:0/3041274765 >> 172.21.15.113:0/3041274765 conn(0x7fd6bc0fcf40 msgr2=0x7fd6bc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:08:20.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 -- 172.21.15.113:0/3041274765 shutdown_connections 2024-04-22T07:08:20.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.564+0000 7fd6c47a8700 1 -- 172.21.15.113:0/3041274765 wait complete. 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 Processor -- start 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 -- start start 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 0x7fd6bc06b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6bc103540 0x7fd6bc1a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc10d580 con 0x7fd6bc102ba0 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc10d400 con 0x7fd6bc1019a0 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c47a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc10d700 con 0x7fd6bc103540 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c1d43700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c1d43700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41956/0 (socket says 172.21.15.113:41956) 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c1d43700 1 -- 172.21.15.113:0/2438243788 learned_addr learned my addr 172.21.15.113:0/2438243788 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c2544700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 0x7fd6bc06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:20.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.565+0000 7fd6c2d45700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6bc103540 0x7fd6bc1a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c1d43700 1 -- 172.21.15.113:0/2438243788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6bc103540 msgr2=0x7fd6bc1a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c1d43700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6bc103540 0x7fd6bc1a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c1d43700 1 -- 172.21.15.113:0/2438243788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 msgr2=0x7fd6bc06b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c1d43700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 0x7fd6bc06b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c1d43700 1 -- 172.21.15.113:0/2438243788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc1a0950 con 0x7fd6bc102ba0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c2544700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 0x7fd6bc06b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c1d43700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc19da40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b580 tx=0x7fd6ac003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00f890 con 0x7fd6bc102ba0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd6ac01f4d0 con 0x7fd6bc102ba0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc197810 con 0x7fd6bc102ba0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac017670 con 0x7fd6bc102ba0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.566+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc197cf0 con 0x7fd6bc102ba0 2024-04-22T07:08:20.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.567+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc069730 con 0x7fd6bc102ba0 2024-04-22T07:08:20.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.568+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fd6ac01f680 con 0x7fd6bc102ba0 2024-04-22T07:08:20.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.569+0000 7fd6b37fe700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd6a80a3bb0 0x7fd6a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:20.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.569+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fd6ac0d4120 con 0x7fd6bc102ba0 2024-04-22T07:08:20.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.569+0000 7fd6c2544700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd6a80a3bb0 0x7fd6a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:20.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.570+0000 7fd6c2544700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd6a80a3bb0 0x7fd6a80a6070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc102a00 tx=0x7fd6b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:08:20.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.573+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6ac013070 con 0x7fd6bc102ba0 2024-04-22T07:08:20.672 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T07:08:20.672 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 ceph-mon[31862]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T07:08:20.672 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:08:20.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:20.799+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd6bc062e80 con 0x7fd6a80a3bb0 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44469]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44469]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44469]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44469]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44472]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44472]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44472]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:20.797 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44472]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44443]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44443]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44448]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44448]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44448]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:20.798 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:08:20 smithi113 sudo[44448]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:20.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T07:08:20.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 ceph-mon[31198]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T07:08:20.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:08:20.804 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39904]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39904]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39907]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:20.805 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39907]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39910]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39910]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39913]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39913]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:20 smithi133 sudo[39913]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41058]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41058]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41058]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41058]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41061]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41061]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41061]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:21.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41061]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.066 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41052]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T07:08:21.066 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:21.066 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41052]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:21.067 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41052]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.067 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41055]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T07:08:21.067 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41055]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T07:08:21.067 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41055]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T07:08:21.067 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:08:20 smithi154 sudo[41055]: pam_unix(sudo:session): session closed for user root 2024-04-22T07:08:21.679 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T07:08:21.679 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T07:08:21.679 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.679 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.679 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.680 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:08:21.681 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:08:21.681 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:08:21.681 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:21.681 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.681 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:21 smithi113 ceph-mon[27933]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:21.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:21.732+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd6ac09dfb0 con 0x7fd6bc102ba0 2024-04-22T07:08:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:21 smithi133 ceph-mon[31198]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T07:08:21.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T07:08:21.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T07:08:21.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:21 smithi154 ceph-mon[31862]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:22.667 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:22 smithi113 ceph-mon[27933]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T07:08:22.668 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:22 smithi113 ceph-mon[27933]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T07:08:22.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:22 smithi133 ceph-mon[31198]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T07:08:22.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:22 smithi133 ceph-mon[31198]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T07:08:22.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:22 smithi154 ceph-mon[31862]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T07:08:22.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:22 smithi154 ceph-mon[31862]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T07:08:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:23 smithi133 ceph-mon[31198]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:23.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:23 smithi154 ceph-mon[31862]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:23 smithi113 ceph-mon[27933]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:24 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/653844034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e87876-83f8-46b3-8daf-3ebf810d8b99"}]: dispatch 2024-04-22T07:08:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:24 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/653844034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06e87876-83f8-46b3-8daf-3ebf810d8b99"}]': finished 2024-04-22T07:08:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:24 smithi133 ceph-mon[31198]: osdmap e22: 4 total, 3 up, 4 in 2024-04-22T07:08:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:24 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:24.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:24 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1949045550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:24.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:24 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/653844034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e87876-83f8-46b3-8daf-3ebf810d8b99"}]: dispatch 2024-04-22T07:08:24.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:24 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/653844034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06e87876-83f8-46b3-8daf-3ebf810d8b99"}]': finished 2024-04-22T07:08:24.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:24 smithi154 ceph-mon[31862]: osdmap e22: 4 total, 3 up, 4 in 2024-04-22T07:08:24.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:24 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:24.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:24 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1949045550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:24.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:24 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/653844034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06e87876-83f8-46b3-8daf-3ebf810d8b99"}]: dispatch 2024-04-22T07:08:24.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:24 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/653844034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06e87876-83f8-46b3-8daf-3ebf810d8b99"}]': finished 2024-04-22T07:08:24.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:24 smithi113 ceph-mon[27933]: osdmap e22: 4 total, 3 up, 4 in 2024-04-22T07:08:24.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:24 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:24.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:24 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1949045550' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:25 smithi133 ceph-mon[31198]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:25.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:25 smithi154 ceph-mon[31862]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:25.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:25 smithi113 ceph-mon[27933]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:28.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:28 smithi113 ceph-mon[27933]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 131 KiB/s wr, 7 op/s 2024-04-22T07:08:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:28 smithi133 ceph-mon[31198]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 131 KiB/s wr, 7 op/s 2024-04-22T07:08:28.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:28 smithi154 ceph-mon[31862]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 131 KiB/s wr, 7 op/s 2024-04-22T07:08:30.221 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:30 smithi113 ceph-mon[27933]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 98 KiB/s wr, 5 op/s 2024-04-22T07:08:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:30 smithi133 ceph-mon[31198]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 98 KiB/s wr, 5 op/s 2024-04-22T07:08:30.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:30 smithi154 ceph-mon[31862]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 98 KiB/s wr, 5 op/s 2024-04-22T07:08:32.164 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:32 smithi113 ceph-mon[27933]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-04-22T07:08:32.165 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T07:08:32.165 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:32 smithi133 ceph-mon[31198]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-04-22T07:08:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T07:08:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:32.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:32 smithi154 ceph-mon[31862]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-04-22T07:08:32.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T07:08:32.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:33.254 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:33 smithi113 ceph-mon[27933]: Deploying daemon osd.3 on smithi113 2024-04-22T07:08:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:33 smithi133 ceph-mon[31198]: Deploying daemon osd.3 on smithi113 2024-04-22T07:08:33.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:33 smithi154 ceph-mon[31862]: Deploying daemon osd.3 on smithi113 2024-04-22T07:08:34.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:34 smithi113 ceph-mon[27933]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 78 KiB/s wr, 4 op/s 2024-04-22T07:08:34.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:34 smithi133 ceph-mon[31198]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 78 KiB/s wr, 4 op/s 2024-04-22T07:08:34.467 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:34 smithi154 ceph-mon[31862]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 78 KiB/s wr, 4 op/s 2024-04-22T07:08:36.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:36 smithi113 ceph-mon[27933]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-22T07:08:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:36 smithi133 ceph-mon[31198]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-22T07:08:36.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:36 smithi154 ceph-mon[31862]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-22T07:08:37.981 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:37.981 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:37.981 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:37.981 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:37 smithi113 ceph-mon[27933]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 65 KiB/s wr, 3 op/s 2024-04-22T07:08:38.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:38.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:38.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:38.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:37 smithi133 ceph-mon[31198]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 65 KiB/s wr, 3 op/s 2024-04-22T07:08:38.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:38.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:38.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:38.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:37 smithi154 ceph-mon[31862]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 65 KiB/s wr, 3 op/s 2024-04-22T07:08:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 3 on host 'smithi113' 2024-04-22T07:08:39.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.878+0000 7fd6b37fe700 1 -- 172.21.15.113:0/2438243788 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd6bc062e80 con 0x7fd6a80a3bb0 2024-04-22T07:08:39.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.880+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd6a80a3bb0 msgr2=0x7fd6a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:39.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.880+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd6a80a3bb0 0x7fd6a80a6070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc102a00 tx=0x7fd6b8009260 comp rx=0 tx=0).stop 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.880+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 msgr2=0x7fd6bc19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.880+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc19da40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b580 tx=0x7fd6ac003680 comp rx=0 tx=0).stop 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.880+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 shutdown_connections 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd6a80a3bb0 0x7fd6a80a6070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6bc103540 0x7fd6bc1a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6bc1019a0 0x7fd6bc06b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 --2- 172.21.15.113:0/2438243788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6bc102ba0 0x7fd6bc19da40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 >> 172.21.15.113:0/2438243788 conn(0x7fd6bc0fcf40 msgr2=0x7fd6bc105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 shutdown_connections 2024-04-22T07:08:39.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:08:39.881+0000 7fd6c47a8700 1 -- 172.21.15.113:0/2438243788 wait complete. 2024-04-22T07:08:40.219 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:40 smithi113 ceph-mon[27933]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:40.219 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:40 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:40.219 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:40 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:40.509 DEBUG:teuthology.orchestra.run.smithi113:osd.3> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.3.service 2024-04-22T07:08:40.511 INFO:tasks.cephadm:Deploying osd.4 on smithi133 with /dev/vg_nvme/lv_3... 2024-04-22T07:08:40.511 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T07:08:40.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:40 smithi133 ceph-mon[31198]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:40.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:40 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:40.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:40 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:40.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:40 smithi154 ceph-mon[31862]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:40.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:40 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:40.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:40 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:08:41.174 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:41 smithi133 ceph-mon[31198]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T07:08:41.174 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:41 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:41.175 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:41 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:41 smithi113 ceph-mon[27933]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T07:08:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:41 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:41 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:41.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:41 smithi154 ceph-mon[31862]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T07:08:41.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:41 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:41.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:41 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:42 smithi133 ceph-mon[31198]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:42 smithi133 ceph-mon[31198]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T07:08:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:42 smithi133 ceph-mon[31198]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T07:08:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:42 smithi133 ceph-mon[31198]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:08:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:42.325 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-mon[27933]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:42.325 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-mon[27933]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T07:08:42.325 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-mon[27933]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T07:08:42.325 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-mon[27933]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:08:42.325 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:42.326 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:08:42.090+0000 7f269bc09700 -1 osd.3 0 waiting for initial osdmap 2024-04-22T07:08:42.326 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:08:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:08:42.095+0000 7f26979fc700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:08:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T07:08:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T07:08:42.520 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-22T07:08:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-22T07:08:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114366 s, 917 MB/s 2024-04-22T07:08:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-22T07:08:42.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:42 smithi154 ceph-mon[31862]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:42 smithi154 ceph-mon[31862]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T07:08:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:42 smithi154 ceph-mon[31862]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T07:08:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:42 smithi154 ceph-mon[31862]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:08:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:43.174 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:43 smithi133 ceph-mon[31198]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:08:43.174 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:43 smithi133 ceph-mon[31198]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T07:08:43.175 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:43 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:43.175 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:43 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:43.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-04-22T07:08:43.474 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:43 smithi113 ceph-mon[27933]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:08:43.475 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:43 smithi113 ceph-mon[27933]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T07:08:43.475 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:43 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:43.475 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:43 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:43.481 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:08:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:43 smithi154 ceph-mon[31862]: from='osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:08:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:43 smithi154 ceph-mon[31862]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T07:08:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:43 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:43 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.247 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] boot 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: osdmap e25: 4 total, 4 up, 4 in 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.248 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:44.249 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:44.249 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.249 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:44.249 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:44.249 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:44.249 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:08:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:08:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] boot 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: osdmap e25: 4 total, 4 up, 4 in 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:44.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:44.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: osd.3 [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] boot 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: osdmap e25: 4 total, 4 up, 4 in 2024-04-22T07:08:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:08:44.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:08:44.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 -- 172.21.15.133:0/1070048982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20041019e0 msgr2=0x7f2004101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:44.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 --2- 172.21.15.133:0/1070048982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20041019e0 0x7f2004101e00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20000099c0 tx=0x7f2000009cd0 comp rx=0 tx=0).stop 2024-04-22T07:08:44.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 -- 172.21.15.133:0/1070048982 shutdown_connections 2024-04-22T07:08:44.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 --2- 172.21.15.133:0/1070048982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2004103580 0x7f2004109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:44.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 --2- 172.21.15.133:0/1070048982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20041019e0 0x7f2004101e00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 --2- 172.21.15.133:0/1070048982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2004102be0 0x7f2004103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 -- 172.21.15.133:0/1070048982 >> 172.21.15.133:0/1070048982 conn(0x7f20040fcfa0 msgr2=0x7f20040ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 -- 172.21.15.133:0/1070048982 shutdown_connections 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 -- 172.21.15.133:0/1070048982 wait complete. 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 Processor -- start 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.972+0000 7f200c129700 1 -- start start 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20041019e0 0x7f2004105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2004102be0 0x7f2004103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 0x7f2004104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f200410d590 con 0x7f20041019e0 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f200410d410 con 0x7f2004103580 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f200410d710 con 0x7f2004102be0 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 0x7f2004104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 0x7f2004104500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55670/0 (socket says 172.21.15.133:55670) 2024-04-22T07:08:44.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 -- 172.21.15.133:0/638590061 learned_addr learned my addr 172.21.15.133:0/638590061 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f2009ec5700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20041019e0 0x7f2004105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f20096c4700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2004102be0 0x7f2004103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 -- 172.21.15.133:0/638590061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2004102be0 msgr2=0x7f2004103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2004102be0 0x7f2004103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 -- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20041019e0 msgr2=0x7f2004105970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20041019e0 0x7f2004105970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 -- 172.21.15.133:0/638590061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2004104db0 con 0x7f2004103580 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200a6c6700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 0x7f2004104500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00ef30 tx=0x7f1ffc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0092e0 con 0x7f2004103580 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1ffc004bb0 con 0x7f2004103580 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20041a2e70 con 0x7f2004103580 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc004280 con 0x7f2004103580 2024-04-22T07:08:44.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.973+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a32d0 con 0x7f2004103580 2024-04-22T07:08:44.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.974+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2004069730 con 0x7f2004103580 2024-04-22T07:08:44.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.975+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1ffc01b030 con 0x7f2004103580 2024-04-22T07:08:44.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.976+0000 7f1ffaffd700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ff00a3c40 0x7f1ff00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:44.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.976+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f1ffc0cece0 con 0x7f2004103580 2024-04-22T07:08:44.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.976+0000 7f2009ec5700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ff00a3c40 0x7f1ff00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:08:44.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.977+0000 7f2009ec5700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ff00a3c40 0x7f1ff00a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2000006010 tx=0x7f2000005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:08:44.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:44.981+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ffc022020 con 0x7f2004103580 2024-04-22T07:08:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:08:45.190+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f20040ff180 con 0x7f1ff00a3c40 2024-04-22T07:08:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:45 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi113 2024-04-22T07:08:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:45 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi113 to 8544M 2024-04-22T07:08:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:45 smithi133 ceph-mon[31198]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T07:08:45.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:45 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi113 2024-04-22T07:08:45.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:45 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi113 to 8544M 2024-04-22T07:08:45.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:45 smithi154 ceph-mon[31862]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T07:08:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:45 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi113 2024-04-22T07:08:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:45 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi113 to 8544M 2024-04-22T07:08:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:45 smithi113 ceph-mon[27933]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T07:08:46.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:46 smithi133 ceph-mon[31198]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:46.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:46 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:08:46.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:46 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:08:46.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:46 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:46.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:46 smithi133 ceph-mon[31198]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T07:08:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:46 smithi154 ceph-mon[31862]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:46 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:08:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:46 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:08:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:46 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:46 smithi154 ceph-mon[31862]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T07:08:46.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:46 smithi113 ceph-mon[27933]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:46.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:46 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:08:46.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:46 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:08:46.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:46 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:46.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:46 smithi113 ceph-mon[27933]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T07:08:47.283 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:47 smithi133 ceph-mon[31198]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:08:47.284 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:47 smithi133 ceph-mon[31198]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T07:08:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:47 smithi154 ceph-mon[31862]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:08:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:47 smithi154 ceph-mon[31862]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T07:08:47.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:47 smithi113 ceph-mon[27933]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:08:47.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:47 smithi113 ceph-mon[27933]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T07:08:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:48 smithi133 ceph-mon[31198]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-22T07:08:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/3971759008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]: dispatch 2024-04-22T07:08:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:48 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]: dispatch 2024-04-22T07:08:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:48 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]': finished 2024-04-22T07:08:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:48 smithi133 ceph-mon[31198]: osdmap e30: 5 total, 4 up, 5 in 2024-04-22T07:08:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:08:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:48 smithi154 ceph-mon[31862]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-22T07:08:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:48 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/3971759008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]: dispatch 2024-04-22T07:08:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:48 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]: dispatch 2024-04-22T07:08:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:48 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]': finished 2024-04-22T07:08:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:48 smithi154 ceph-mon[31862]: osdmap e30: 5 total, 4 up, 5 in 2024-04-22T07:08:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:48 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:08:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:48 smithi113 ceph-mon[27933]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-22T07:08:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:48 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/3971759008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]: dispatch 2024-04-22T07:08:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:48 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]: dispatch 2024-04-22T07:08:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:48 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35753be4-5e96-47bf-8527-b7c4baeb873e"}]': finished 2024-04-22T07:08:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:48 smithi113 ceph-mon[27933]: osdmap e30: 5 total, 4 up, 5 in 2024-04-22T07:08:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:48 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:08:49.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:49 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/790380005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:49.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:49 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/790380005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:49.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:49 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/790380005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:08:50.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:50 smithi133 ceph-mon[31198]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-22T07:08:50.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:50 smithi154 ceph-mon[31862]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-22T07:08:50.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:50 smithi113 ceph-mon[27933]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-22T07:08:52.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:52 smithi133 ceph-mon[31198]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T07:08:52.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:52 smithi154 ceph-mon[31862]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T07:08:52.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:52 smithi113 ceph-mon[27933]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T07:08:54.386 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:54 smithi113 ceph-mon[27933]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-22T07:08:54.426 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:54 smithi133 ceph-mon[31198]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-22T07:08:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:54 smithi154 ceph-mon[31862]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-22T07:08:56.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:56 smithi133 ceph-mon[31198]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:56.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T07:08:56.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:56 smithi154 ceph-mon[31862]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T07:08:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:56 smithi113 ceph-mon[27933]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T07:08:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:08:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:57 smithi133 ceph-mon[31198]: Deploying daemon osd.4 on smithi133 2024-04-22T07:08:57.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:57 smithi154 ceph-mon[31862]: Deploying daemon osd.4 on smithi133 2024-04-22T07:08:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:57 smithi113 ceph-mon[27933]: Deploying daemon osd.4 on smithi133 2024-04-22T07:08:58.469 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:08:58 smithi133 ceph-mon[31198]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:58.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:08:58 smithi154 ceph-mon[31862]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:08:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:08:58 smithi113 ceph-mon[27933]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:00.395 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:00 smithi133 ceph-mon[31198]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:00 smithi154 ceph-mon[31862]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:00 smithi113 ceph-mon[27933]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:02.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:02.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:02.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:02.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:01 smithi133 ceph-mon[31198]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:02.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:02.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:02.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:02.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:01 smithi154 ceph-mon[31862]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:01 smithi113 ceph-mon[27933]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:04.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:04 smithi133 ceph-mon[31198]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:04.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:04 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:04 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:04 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:04.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:04 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:04.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:04 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:04 smithi113 ceph-mon[27933]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:04 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:04 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:04 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:04 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:04.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:04 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:04 smithi154 ceph-mon[31862]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:04 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:04 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:04 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:04 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:04 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:04.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.663+0000 7f1ffaffd700 1 -- 172.21.15.133:0/638590061 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f20040ff180 con 0x7f1ff00a3c40 2024-04-22T07:09:04.661 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 4 on host 'smithi133' 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ff00a3c40 msgr2=0x7f1ff00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ff00a3c40 0x7f1ff00a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2000006010 tx=0x7f2000005bc0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 msgr2=0x7f2004104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 0x7f2004104500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00ef30 tx=0x7f1ffc00c3a0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 shutdown_connections 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ff00a3c40 0x7f1ff00a6100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2004103580 0x7f2004104500 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20041019e0 0x7f2004105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 --2- 172.21.15.133:0/638590061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2004102be0 0x7f2004103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 >> 172.21.15.133:0/638590061 conn(0x7f20040fcfa0 msgr2=0x7f20040fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:09:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.666+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 shutdown_connections 2024-04-22T07:09:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:04.667+0000 7f200c129700 1 -- 172.21.15.133:0/638590061 wait complete. 2024-04-22T07:09:05.250 DEBUG:teuthology.orchestra.run.smithi133:osd.4> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.4.service 2024-04-22T07:09:05.252 INFO:tasks.cephadm:Deploying osd.5 on smithi154 with /dev/vg_nvme/lv_3... 2024-04-22T07:09:05.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T07:09:05.499 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:09:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:05 smithi133 ceph-mon[31198]: from='osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T07:09:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:05 smithi133 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T07:09:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:05 smithi133 ceph-mon[31198]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:05.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:05.940 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:05.941 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:05.941 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:05 smithi154 ceph-mon[31862]: from='osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T07:09:05.941 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:05 smithi154 ceph-mon[31862]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T07:09:05.941 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:05 smithi154 ceph-mon[31862]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:06.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:06.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:06.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:06.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:05 smithi113 ceph-mon[27933]: from='osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T07:09:06.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:05 smithi113 ceph-mon[27933]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T07:09:06.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:05 smithi113 ceph-mon[27933]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:06.802 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:06 smithi154 ceph-mon[31862]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T07:09:06.802 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:06 smithi154 ceph-mon[31862]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T07:09:06.802 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:06 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:06.802 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:06 smithi154 ceph-mon[31862]: from='osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:09:06.802 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:06 smithi154 ceph-mon[31862]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:09:06.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T07:09:06.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-mon[31198]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T07:09:06.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:06.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-mon[31198]: from='osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:09:06.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:09:06.880 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:09:06.668+0000 7f49630ee700 -1 osd.4 0 waiting for initial osdmap 2024-04-22T07:09:06.880 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:09:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:09:06.677+0000 7f495eee1700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:09:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:06 smithi113 ceph-mon[27933]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T07:09:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:06 smithi113 ceph-mon[27933]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T07:09:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:06 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:06 smithi113 ceph-mon[27933]: from='osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:09:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:06 smithi113 ceph-mon[27933]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:09:07.386 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T07:09:07.386 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T07:09:07.386 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-22T07:09:07.386 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-22T07:09:07.387 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0158083 s, 663 MB/s 2024-04-22T07:09:07.387 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-22T07:09:07.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:07 smithi133 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:09:07.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:07 smithi133 ceph-mon[31198]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T07:09:07.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:07.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:07.804 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:07 smithi133 ceph-mon[31198]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:07.991 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:07 smithi154 ceph-mon[31862]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:09:07.991 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:07 smithi154 ceph-mon[31862]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T07:09:07.991 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:07.991 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:07.991 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:07 smithi154 ceph-mon[31862]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:08.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_3 2024-04-22T07:09:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:07 smithi113 ceph-mon[27933]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:09:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:07 smithi113 ceph-mon[27933]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T07:09:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:08.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:07 smithi113 ceph-mon[27933]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-22T07:09:08.270 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:09:08.938 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:09:08.938 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:09:08.938 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:08.938 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] boot 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: osdmap e33: 5 total, 5 up, 5 in 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi133 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi133 to 8544M 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:08.939 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:08 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:09:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:09:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] boot 2024-04-22T07:09:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: osdmap e33: 5 total, 5 up, 5 in 2024-04-22T07:09:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi133 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi133 to 8544M 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:09.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:09.066 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:08 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:09:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:09:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: osd.4 [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] boot 2024-04-22T07:09:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: osdmap e33: 5 total, 5 up, 5 in 2024-04-22T07:09:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi133 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi133 to 8544M 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:09.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:08 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:09.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 -- 172.21.15.154:0/1325555888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84102ba0 msgr2=0x7fdf84103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:09.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/1325555888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84102ba0 0x7fdf84103000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009990 tx=0x7fdf80009ca0 comp rx=0 tx=0).stop 2024-04-22T07:09:09.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 -- 172.21.15.154:0/1325555888 shutdown_connections 2024-04-22T07:09:09.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/1325555888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84103540 0x7fdf84109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:09.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/1325555888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf841019a0 0x7fdf84101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:09.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/1325555888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84102ba0 0x7fdf84103000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 -- 172.21.15.154:0/1325555888 >> 172.21.15.154:0/1325555888 conn(0x7fdf840fcf60 msgr2=0x7fdf840ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 -- 172.21.15.154:0/1325555888 shutdown_connections 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.839+0000 7fdf8bc33700 1 -- 172.21.15.154:0/1325555888 wait complete. 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 Processor -- start 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 -- start start 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf841019a0 0x7fdf84105a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84102ba0 0x7fdf84104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 0x7fdf84104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8410d550 con 0x7fdf841019a0 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8410d3d0 con 0x7fdf84102ba0 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8bc33700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8410d6d0 con 0x7fdf84103540 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8a1d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 0x7fdf84104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8a1d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 0x7fdf84104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48682/0 (socket says 172.21.15.154:48682) 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.840+0000 7fdf8a1d0700 1 -- 172.21.15.154:0/2804895927 learned_addr learned my addr 172.21.15.154:0/2804895927 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:09:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf899cf700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf841019a0 0x7fdf84105a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8a1d0700 1 -- 172.21.15.154:0/2804895927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84102ba0 msgr2=0x7fdf84104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8a1d0700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84102ba0 0x7fdf84104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8a1d0700 1 -- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf841019a0 msgr2=0x7fdf84105a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8a1d0700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf841019a0 0x7fdf84105a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8a1d0700 1 -- 172.21.15.154:0/2804895927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf84104e40 con 0x7fdf84103540 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8a1d0700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 0x7fdf84104590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c007ee0 tx=0x7fdf7c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00dbd0 con 0x7fdf84103540 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdf7c00dd70 con 0x7fdf84103540 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf841a2e20 con 0x7fdf84103540 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0133f0 con 0x7fdf84103540 2024-04-22T07:09:09.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.841+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf841a3280 con 0x7fdf84103540 2024-04-22T07:09:09.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.842+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf84069730 con 0x7fdf84103540 2024-04-22T07:09:09.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.843+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdf7c01a020 con 0x7fdf84103540 2024-04-22T07:09:09.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.843+0000 7fdf7affd700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdf700a3c40 0x7fdf700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:09.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.843+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fdf7c0cec60 con 0x7fdf84103540 2024-04-22T07:09:09.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.844+0000 7fdf899cf700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdf700a3c40 0x7fdf700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:09.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.844+0000 7fdf899cf700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdf700a3c40 0x7fdf700a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf84102a00 tx=0x7fdf74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:09:09.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:09.848+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf7c010070 con 0x7fdf84103540 2024-04-22T07:09:10.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:10.052+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdf8406b8c0 con 0x7fdf700a3c40 2024-04-22T07:09:10.050 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:09 smithi154 ceph-mon[31862]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T07:09:10.051 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:09 smithi154 ceph-mon[31862]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:10.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:09 smithi133 ceph-mon[31198]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T07:09:10.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:09 smithi133 ceph-mon[31198]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:10.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:09 smithi113 ceph-mon[27933]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T07:09:10.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:09 smithi113 ceph-mon[27933]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:10.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:10 smithi154 ceph-mon[31862]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:09:10.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:09:10.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:09:10.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:11.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:10 smithi133 ceph-mon[31198]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:09:11.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:09:11.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:09:11.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:10 smithi113 ceph-mon[27933]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:09:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:09:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:09:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:11.842 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:11 smithi154 ceph-mon[31862]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:12.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:11 smithi133 ceph-mon[31198]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:12.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:11 smithi113 ceph-mon[27933]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:12 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/1910406815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]: dispatch 2024-04-22T07:09:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:12 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]: dispatch 2024-04-22T07:09:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:12 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]': finished 2024-04-22T07:09:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:12 smithi133 ceph-mon[31198]: osdmap e35: 6 total, 5 up, 6 in 2024-04-22T07:09:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:12 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:13.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:12 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/1910406815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]: dispatch 2024-04-22T07:09:13.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:12 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]: dispatch 2024-04-22T07:09:13.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:12 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]': finished 2024-04-22T07:09:13.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:12 smithi154 ceph-mon[31862]: osdmap e35: 6 total, 5 up, 6 in 2024-04-22T07:09:13.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:12 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:13.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:12 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/1910406815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]: dispatch 2024-04-22T07:09:13.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:12 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]: dispatch 2024-04-22T07:09:13.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:12 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724"}]': finished 2024-04-22T07:09:13.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:12 smithi113 ceph-mon[27933]: osdmap e35: 6 total, 5 up, 6 in 2024-04-22T07:09:13.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:12 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:14.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:13 smithi133 ceph-mon[31198]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:14.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:13 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/2077916092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:09:14.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:13 smithi154 ceph-mon[31862]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:14.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:13 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/2077916092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:09:14.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:13 smithi113 ceph-mon[27933]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:14.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:13 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/2077916092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:09:16.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:16 smithi113 ceph-mon[27933]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:16 smithi133 ceph-mon[31198]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:16.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:16 smithi154 ceph-mon[31862]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:18 smithi133 ceph-mon[31198]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:18 smithi113 ceph-mon[27933]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:18.380 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:18 smithi154 ceph-mon[31862]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:20.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:20 smithi154 ceph-mon[31862]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:20.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:20 smithi133 ceph-mon[31198]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:20.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:20 smithi113 ceph-mon[27933]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:21.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T07:09:21.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:21.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:21 smithi154 ceph-mon[31862]: Deploying daemon osd.5 on smithi154 2024-04-22T07:09:21.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T07:09:21.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:21.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:21 smithi133 ceph-mon[31198]: Deploying daemon osd.5 on smithi154 2024-04-22T07:09:21.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T07:09:21.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:21.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:21 smithi113 ceph-mon[27933]: Deploying daemon osd.5 on smithi154 2024-04-22T07:09:22.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:22 smithi133 ceph-mon[31198]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:22.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:22 smithi154 ceph-mon[31862]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:22.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:22 smithi113 ceph-mon[27933]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:24.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:24 smithi154 ceph-mon[31862]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:24.386 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:24 smithi113 ceph-mon[27933]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:24.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:24 smithi133 ceph-mon[31198]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:26.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:26 smithi154 ceph-mon[31862]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:26.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:26.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:26.371 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:26 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:26 smithi133 ceph-mon[31198]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:26 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:26 smithi113 ceph-mon[27933]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:26 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:28.171 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:28 smithi154 ceph-mon[31862]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:28 smithi133 ceph-mon[31198]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:28 smithi113 ceph-mon[27933]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:29.121 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 5 on host 'smithi154' 2024-04-22T07:09:29.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.122+0000 7fdf7affd700 1 -- 172.21.15.154:0/2804895927 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf8406b8c0 con 0x7fdf700a3c40 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdf700a3c40 msgr2=0x7fdf700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdf700a3c40 0x7fdf700a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf84102a00 tx=0x7fdf74009260 comp rx=0 tx=0).stop 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 msgr2=0x7fdf84104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 0x7fdf84104590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c007ee0 tx=0x7fdf7c00d3b0 comp rx=0 tx=0).stop 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 shutdown_connections 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdf700a3c40 0x7fdf700a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf84103540 0x7fdf84104590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf841019a0 0x7fdf84105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 --2- 172.21.15.154:0/2804895927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84102ba0 0x7fdf84104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 >> 172.21.15.154:0/2804895927 conn(0x7fdf840fcf60 msgr2=0x7fdf8406b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 shutdown_connections 2024-04-22T07:09:29.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:09:29.125+0000 7fdf8bc33700 1 -- 172.21.15.154:0/2804895927 wait complete. 2024-04-22T07:09:29.740 DEBUG:teuthology.orchestra.run.smithi154:osd.5> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.5.service 2024-04-22T07:09:29.742 INFO:tasks.cephadm:Deploying osd.6 on smithi113 with /dev/vg_nvme/lv_2... 2024-04-22T07:09:29.742 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T07:09:29.992 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:09:30.351 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:30.351 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.351 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.351 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: from='osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T07:09:30.351 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T07:09:30.351 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.352 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:30 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.487 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:30.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: from='osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T07:09:30.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T07:09:30.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.488 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:30 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: from='osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:30 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:31.390 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:31 smithi113 ceph-mon[27933]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T07:09:31.390 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:31 smithi113 ceph-mon[27933]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T07:09:31.390 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:31.390 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:31 smithi113 ceph-mon[27933]: from='osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:09:31.390 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:31 smithi113 ceph-mon[27933]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:09:31.420 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-mon[31862]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T07:09:31.420 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-mon[31862]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T07:09:31.420 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:31.420 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-mon[31862]: from='osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:09:31.420 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-mon[31862]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:09:31.421 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:09:31.122+0000 7f82ec3c5700 -1 osd.5 0 waiting for initial osdmap 2024-04-22T07:09:31.421 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:09:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:09:31.129+0000 7f82e81b8700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:09:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:31 smithi133 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T07:09:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:31 smithi133 ceph-mon[31198]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T07:09:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:31 smithi133 ceph-mon[31198]: from='osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:09:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:31 smithi133 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:09:31.946 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T07:09:31.946 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T07:09:31.946 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-22T07:09:31.946 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-22T07:09:31.946 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0101116 s, 1.0 GB/s 2024-04-22T07:09:31.946 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-22T07:09:32.507 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:32 smithi154 ceph-mon[31862]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:32.507 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:32 smithi154 ceph-mon[31862]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:09:32.507 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:32 smithi154 ceph-mon[31862]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T07:09:32.507 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:32.508 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:32 smithi133 ceph-mon[31198]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:32 smithi133 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:09:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:32 smithi133 ceph-mon[31198]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T07:09:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:32 smithi113 ceph-mon[27933]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:32 smithi113 ceph-mon[27933]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:09:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:32 smithi113 ceph-mon[27933]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T07:09:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:32.654 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_2 2024-04-22T07:09:32.895 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:09:33.303 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:33 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:09:33.303 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:33 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:09:33.303 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:33 smithi113 ceph-mon[27933]: osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] boot 2024-04-22T07:09:33.303 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:33 smithi113 ceph-mon[27933]: osdmap e38: 6 total, 6 up, 6 in 2024-04-22T07:09:33.303 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:33.304 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:33 smithi113 ceph-mon[27933]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T07:09:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:33 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:09:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:33 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:09:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:33 smithi133 ceph-mon[31198]: osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] boot 2024-04-22T07:09:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:33 smithi133 ceph-mon[31198]: osdmap e38: 6 total, 6 up, 6 in 2024-04-22T07:09:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:33 smithi133 ceph-mon[31198]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T07:09:33.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:33 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:09:33.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:33 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:09:33.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:33 smithi154 ceph-mon[31862]: osd.5 [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] boot 2024-04-22T07:09:33.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:33 smithi154 ceph-mon[31862]: osdmap e38: 6 total, 6 up, 6 in 2024-04-22T07:09:33.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:09:33.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:33 smithi154 ceph-mon[31862]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi154 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi154 to 8544M 2024-04-22T07:09:34.443 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.444 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:34 smithi113 ceph-mon[27933]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T07:09:34.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 -- 172.21.15.113:0/628044549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304104650 msgr2=0x7f830410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:34.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 --2- 172.21.15.113:0/628044549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304104650 0x7f830410af00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00b0d0 tx=0x7f82fc00b3e0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 -- 172.21.15.113:0/628044549 shutdown_connections 2024-04-22T07:09:34.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 --2- 172.21.15.113:0/628044549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304104650 0x7f830410af00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 --2- 172.21.15.113:0/628044549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8304102ab0 0x7f8304102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 --2- 172.21.15.113:0/628044549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304103cb0 0x7f8304104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 -- 172.21.15.113:0/628044549 >> 172.21.15.113:0/628044549 conn(0x7f83040fe010 msgr2=0x7f8304100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:09:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 -- 172.21.15.113:0/628044549 shutdown_connections 2024-04-22T07:09:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.465+0000 7f830b2c5700 1 -- 172.21.15.113:0/628044549 wait complete. 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 Processor -- start 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 -- start start 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304102ab0 0x7f8304198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 0x7f8304199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8304104650 0x7f83041a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f830410e690 con 0x7f8304103cb0 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f830410e510 con 0x7f8304102ab0 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f830b2c5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f830410e810 con 0x7f8304104650 2024-04-22T07:09:34.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f8308860700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 0x7f8304199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f8308860700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 0x7f8304199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50036/0 (socket says 172.21.15.113:50036) 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f8308860700 1 -- 172.21.15.113:0/1144959626 learned_addr learned my addr 172.21.15.113:0/1144959626 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f8309061700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304102ab0 0x7f8304198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.466+0000 7f8309862700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8304104650 0x7f83041a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8308860700 1 -- 172.21.15.113:0/1144959626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8304104650 msgr2=0x7f83041a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8308860700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8304104650 0x7f83041a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8308860700 1 -- 172.21.15.113:0/1144959626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304102ab0 msgr2=0x7f8304198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8308860700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304102ab0 0x7f8304198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8308860700 1 -- 172.21.15.113:0/1144959626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83041a0840 con 0x7f8304103cb0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8309061700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304102ab0 0x7f8304198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f8308860700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 0x7f8304199120 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f830000c9e0 tx=0x7f830000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300007ac0 con 0x7f8304103cb0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8300007c60 con 0x7f8304103cb0 2024-04-22T07:09:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83041a0b30 con 0x7f8304103cb0 2024-04-22T07:09:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000186f0 con 0x7f8304103cb0 2024-04-22T07:09:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.467+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83041a1070 con 0x7f8304103cb0 2024-04-22T07:09:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.468+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8304069730 con 0x7f8304103cb0 2024-04-22T07:09:34.468 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi154 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi154 to 8544M 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.469 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:34 smithi154 ceph-mon[31862]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi154 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi154 to 8544M 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:34.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:34.471 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:34.471 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:34.471 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:34 smithi133 ceph-mon[31198]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T07:09:34.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.470+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f830000f9b0 con 0x7f8304103cb0 2024-04-22T07:09:34.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.470+0000 7f82f67fc700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f82f00a3c40 0x7f82f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.470+0000 7f8309061700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f82f00a3c40 0x7f82f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:34.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.471+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f83000cf7f0 con 0x7f8304103cb0 2024-04-22T07:09:34.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.471+0000 7f8309061700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f82f00a3c40 0x7f82f00a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82f8005950 tx=0x7f82f8005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:09:34.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.474+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8300014030 con 0x7f8304103cb0 2024-04-22T07:09:34.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:34.681+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8304062e80 con 0x7f82f00a3c40 2024-04-22T07:09:35.439 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:35 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:09:35.439 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:35 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:09:35.439 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:35 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:35.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:35 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:09:35.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:35 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:09:35.483 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:35 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:35 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:09:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:35 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:09:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:35 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:36.266 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:36 smithi113 ceph-mon[27933]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:09:36.266 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:36 smithi113 ceph-mon[27933]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:36.266 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:36 smithi113 ceph-mon[27933]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T07:09:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:36 smithi133 ceph-mon[31198]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:09:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:36 smithi133 ceph-mon[31198]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:36 smithi133 ceph-mon[31198]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T07:09:36.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:36 smithi154 ceph-mon[31862]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:09:36.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:36 smithi154 ceph-mon[31862]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:36.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:36 smithi154 ceph-mon[31862]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T07:09:37.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:37 smithi133 ceph-mon[31198]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T07:09:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:37 smithi154 ceph-mon[31862]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T07:09:37.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:37 smithi113 ceph-mon[27933]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T07:09:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:38 smithi133 ceph-mon[31198]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:09:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:38 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3145809234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6196d5c6-a3a0-407c-af4a-3ca38ca997fc"}]: dispatch 2024-04-22T07:09:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:38 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3145809234' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6196d5c6-a3a0-407c-af4a-3ca38ca997fc"}]': finished 2024-04-22T07:09:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:38 smithi133 ceph-mon[31198]: osdmap e43: 7 total, 6 up, 7 in 2024-04-22T07:09:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:38 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:38 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2238399677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:09:38.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:38 smithi154 ceph-mon[31862]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:09:38.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:38 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3145809234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6196d5c6-a3a0-407c-af4a-3ca38ca997fc"}]: dispatch 2024-04-22T07:09:38.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:38 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3145809234' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6196d5c6-a3a0-407c-af4a-3ca38ca997fc"}]': finished 2024-04-22T07:09:38.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:38 smithi154 ceph-mon[31862]: osdmap e43: 7 total, 6 up, 7 in 2024-04-22T07:09:38.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:38 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:38.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:38 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2238399677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:09:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:38 smithi113 ceph-mon[27933]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:09:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:38 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3145809234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6196d5c6-a3a0-407c-af4a-3ca38ca997fc"}]: dispatch 2024-04-22T07:09:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:38 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3145809234' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6196d5c6-a3a0-407c-af4a-3ca38ca997fc"}]': finished 2024-04-22T07:09:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:38 smithi113 ceph-mon[27933]: osdmap e43: 7 total, 6 up, 7 in 2024-04-22T07:09:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:38 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:38 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2238399677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:09:39.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:39 smithi133 ceph-mon[31198]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-22T07:09:39.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:39 smithi154 ceph-mon[31862]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-22T07:09:39.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:39 smithi113 ceph-mon[27933]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-22T07:09:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:42 smithi113 ceph-mon[27933]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T07:09:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:42 smithi133 ceph-mon[31198]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T07:09:42.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:42 smithi154 ceph-mon[31862]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T07:09:44.233 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:44 smithi113 ceph-mon[27933]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-22T07:09:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:44 smithi133 ceph-mon[31198]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-22T07:09:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:44 smithi154 ceph-mon[31862]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-22T07:09:45.551 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T07:09:45.551 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T07:09:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:45.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T07:09:45.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:46.427 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:46 smithi113 ceph-mon[27933]: Deploying daemon osd.6 on smithi113 2024-04-22T07:09:46.427 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:46 smithi113 ceph-mon[27933]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:46.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:46 smithi133 ceph-mon[31198]: Deploying daemon osd.6 on smithi113 2024-04-22T07:09:46.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:46 smithi133 ceph-mon[31198]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:46.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:46 smithi154 ceph-mon[31862]: Deploying daemon osd.6 on smithi113 2024-04-22T07:09:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:46 smithi154 ceph-mon[31862]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:48 smithi113 ceph-mon[27933]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:48 smithi133 ceph-mon[31198]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:48.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:48 smithi154 ceph-mon[31862]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:50.262 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:50 smithi113 ceph-mon[27933]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:50.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:50 smithi133 ceph-mon[31198]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:50.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:50 smithi154 ceph-mon[31862]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:51.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:51 smithi113 ceph-mon[27933]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:51 smithi133 ceph-mon[31198]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:51.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:51.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:51.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:51.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:51 smithi154 ceph-mon[31862]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:54.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:54 smithi113 ceph-mon[27933]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.437+0000 7f82f67fc700 1 -- 172.21.15.113:0/1144959626 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8304062e80 con 0x7f82f00a3c40 2024-04-22T07:09:54.436 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 6 on host 'smithi113' 2024-04-22T07:09:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f82f00a3c40 msgr2=0x7f82f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f82f00a3c40 0x7f82f00a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82f8005950 tx=0x7f82f8005f50 comp rx=0 tx=0).stop 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 msgr2=0x7f8304199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 0x7f8304199120 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f830000c9e0 tx=0x7f830000cda0 comp rx=0 tx=0).stop 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 shutdown_connections 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f82f00a3c40 0x7f82f00a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8304104650 0x7f83041a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8304102ab0 0x7f8304198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 --2- 172.21.15.113:0/1144959626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304103cb0 0x7f8304199120 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 >> 172.21.15.113:0/1144959626 conn(0x7f83040fe010 msgr2=0x7f8304106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 shutdown_connections 2024-04-22T07:09:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:09:54.439+0000 7f830b2c5700 1 -- 172.21.15.113:0/1144959626 wait complete. 2024-04-22T07:09:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:54 smithi133 ceph-mon[31198]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:54 smithi154 ceph-mon[31862]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.067 DEBUG:teuthology.orchestra.run.smithi113:osd.6> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.6.service 2024-04-22T07:09:55.069 INFO:tasks.cephadm:Deploying osd.7 on smithi133 with /dev/vg_nvme/lv_2... 2024-04-22T07:09:55.070 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T07:09:55.316 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:09:55.593 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.593 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.593 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:55.593 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:55 smithi113 ceph-mon[27933]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T07:09:55.593 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:55 smithi113 ceph-mon[27933]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:55.653 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.653 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.653 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:55.653 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:55 smithi133 ceph-mon[31198]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T07:09:55.653 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:55 smithi133 ceph-mon[31198]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:55.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:55.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:09:55.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:55 smithi154 ceph-mon[31862]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T07:09:55.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:55 smithi154 ceph-mon[31862]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:56 smithi113 ceph-mon[27933]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T07:09:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:56 smithi113 ceph-mon[27933]: osdmap e44: 7 total, 6 up, 7 in 2024-04-22T07:09:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:56 smithi113 ceph-mon[27933]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:09:56.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:56.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:56 smithi133 ceph-mon[31198]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T07:09:56.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:56 smithi133 ceph-mon[31198]: osdmap e44: 7 total, 6 up, 7 in 2024-04-22T07:09:56.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:56 smithi133 ceph-mon[31198]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:09:56.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:56.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:56 smithi154 ceph-mon[31862]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T07:09:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:56 smithi154 ceph-mon[31862]: osdmap e44: 7 total, 6 up, 7 in 2024-04-22T07:09:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:56 smithi154 ceph-mon[31862]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:09:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.332 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T07:09:57.333 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T07:09:57.333 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-22T07:09:57.333 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-22T07:09:57.333 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114924 s, 912 MB/s 2024-04-22T07:09:57.333 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-22T07:09:57.408 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:09:57.133+0000 7f425ebd6700 -1 osd.6 0 waiting for initial osdmap 2024-04-22T07:09:57.408 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:09:57.227+0000 7f42569c1700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:09:57.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:57 smithi133 ceph-mon[31198]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:09:57.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:57 smithi133 ceph-mon[31198]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T07:09:57.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:57 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:57 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:57 smithi133 ceph-mon[31198]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:57.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:57 smithi133 ceph-mon[31198]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' 2024-04-22T07:09:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:57 smithi154 ceph-mon[31862]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:09:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:57 smithi154 ceph-mon[31862]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T07:09:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:57 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:57 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:57 smithi154 ceph-mon[31862]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:57 smithi154 ceph-mon[31862]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' 2024-04-22T07:09:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-mon[27933]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:09:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-mon[27933]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T07:09:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-mon[27933]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-22T07:09:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:57 smithi113 ceph-mon[27933]: from='osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993]' entity='osd.6' 2024-04-22T07:09:57.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-04-22T07:09:58.137 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] boot 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: osdmap e46: 7 total, 7 up, 7 in 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi113 2024-04-22T07:09:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi113 to 5696M 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:09:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] boot 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: osdmap e46: 7 total, 7 up, 7 in 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi113 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi113 to 5696M 2024-04-22T07:09:58.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:58.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:58.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:09:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:09:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:09:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: osd.6 [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] boot 2024-04-22T07:09:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: osdmap e46: 7 total, 7 up, 7 in 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi113 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi113 to 5696M 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:09:58.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:09:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:09:59.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 -- 172.21.15.133:0/2945432610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0105e50 msgr2=0x7f24f0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 --2- 172.21.15.133:0/2945432610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0105e50 0x7f24f0108240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009990 tx=0x7f24e4009ca0 comp rx=0 tx=0).stop 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 -- 172.21.15.133:0/2945432610 shutdown_connections 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 --2- 172.21.15.133:0/2945432610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24f0108780 0x7f24f010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 --2- 172.21.15.133:0/2945432610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f0103520 0x7f24f0105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 --2- 172.21.15.133:0/2945432610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0105e50 0x7f24f0108240 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 -- 172.21.15.133:0/2945432610 >> 172.21.15.133:0/2945432610 conn(0x7f24f00fcf30 msgr2=0x7f24f00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 -- 172.21.15.133:0/2945432610 shutdown_connections 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.793+0000 7f24f6841700 1 -- 172.21.15.133:0/2945432610 wait complete. 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 Processor -- start 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 -- start start 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f0103520 0x7f24f0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24f0105e50 0x7f24f0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:59.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 0x7f24f01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010e6d0 con 0x7f24f0103520 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010e550 con 0x7f24f0108780 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f6841700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010e850 con 0x7f24f0105e50 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f4dde700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 0x7f24f01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24effff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f0103520 0x7f24f0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f4dde700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 0x7f24f01a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59042/0 (socket says 172.21.15.133:59042) 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24f4dde700 1 -- 172.21.15.133:0/1350752097 learned_addr learned my addr 172.21.15.133:0/1350752097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.794+0000 7f24ef7fe700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24f0105e50 0x7f24f0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f4dde700 1 -- 172.21.15.133:0/1350752097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24f0105e50 msgr2=0x7f24f0199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f4dde700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24f0105e50 0x7f24f0199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f4dde700 1 -- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f0103520 msgr2=0x7f24f0198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f4dde700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f0103520 0x7f24f0198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f4dde700 1 -- 172.21.15.133:0/1350752097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f01a0950 con 0x7f24f0108780 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f4dde700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 0x7f24f01a0280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b9e0 tx=0x7f24e800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:09:59.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e800d8e0 con 0x7f24f0108780 2024-04-22T07:09:59.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f01a0c40 con 0x7f24f0108780 2024-04-22T07:09:59.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f24e8015420 con 0x7f24f0108780 2024-04-22T07:09:59.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8014610 con 0x7f24f0108780 2024-04-22T07:09:59.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.795+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a1180 con 0x7f24f0108780 2024-04-22T07:09:59.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.796+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0069730 con 0x7f24f0108780 2024-04-22T07:09:59.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.797+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f24e80159a0 con 0x7f24f0108780 2024-04-22T07:09:59.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.797+0000 7f24ed7fa700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24d80a3c40 0x7f24d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:59.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.798+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f24e80cefb0 con 0x7f24f0108780 2024-04-22T07:09:59.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.798+0000 7f24effff700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24d80a3c40 0x7f24d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:09:59.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.798+0000 7f24effff700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24d80a3c40 0x7f24d80a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f24f019a1e0 tx=0x7f24e000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:09:59.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:09:59.802+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24e809c6a0 con 0x7f24f0108780 2024-04-22T07:10:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:00.005+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f24f0062e80 con 0x7f24d80a3c40 2024-04-22T07:10:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:00 smithi133 ceph-mon[31198]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:00 smithi133 ceph-mon[31198]: osdmap e47: 7 total, 7 up, 7 in 2024-04-22T07:10:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:00 smithi133 ceph-mon[31198]: overall HEALTH_OK 2024-04-22T07:10:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:00 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:00 smithi154 ceph-mon[31862]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:00 smithi154 ceph-mon[31862]: osdmap e47: 7 total, 7 up, 7 in 2024-04-22T07:10:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:00 smithi154 ceph-mon[31862]: overall HEALTH_OK 2024-04-22T07:10:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:00 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:00 smithi113 ceph-mon[27933]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:00 smithi113 ceph-mon[27933]: osdmap e47: 7 total, 7 up, 7 in 2024-04-22T07:10:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:00 smithi113 ceph-mon[27933]: overall HEALTH_OK 2024-04-22T07:10:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:00 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:01 smithi133 ceph-mon[31198]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:01 smithi154 ceph-mon[31862]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:01 smithi113 ceph-mon[27933]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:02.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:02 smithi133 ceph-mon[31198]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:02 smithi154 ceph-mon[31862]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:02 smithi113 ceph-mon[27933]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:03 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/920614441' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]: dispatch 2024-04-22T07:10:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:03 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]: dispatch 2024-04-22T07:10:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:03 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]': finished 2024-04-22T07:10:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:03 smithi133 ceph-mon[31198]: osdmap e48: 8 total, 7 up, 8 in 2024-04-22T07:10:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:03 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:03 smithi133 ceph-mon[31198]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:03 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/920614441' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]: dispatch 2024-04-22T07:10:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:03 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]: dispatch 2024-04-22T07:10:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:03 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]': finished 2024-04-22T07:10:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:03 smithi154 ceph-mon[31862]: osdmap e48: 8 total, 7 up, 8 in 2024-04-22T07:10:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:03 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:03 smithi154 ceph-mon[31862]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:03 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/920614441' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]: dispatch 2024-04-22T07:10:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:03 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]: dispatch 2024-04-22T07:10:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:03 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da26f96f-5fe9-4081-8152-7396392c2754"}]': finished 2024-04-22T07:10:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:03 smithi113 ceph-mon[27933]: osdmap e48: 8 total, 7 up, 8 in 2024-04-22T07:10:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:03 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:03.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:03 smithi113 ceph-mon[27933]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:04.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/3064128224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:04.466 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/3064128224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:04.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/3064128224' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:05.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:05 smithi133 ceph-mon[31198]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:05.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:05 smithi154 ceph-mon[31862]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:05 smithi113 ceph-mon[27933]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:08.288 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:08 smithi133 ceph-mon[31198]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:08.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:08 smithi154 ceph-mon[31862]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:08.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:08 smithi113 ceph-mon[27933]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:10.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:10 smithi133 ceph-mon[31198]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:10.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:10 smithi154 ceph-mon[31862]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:10.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:10 smithi113 ceph-mon[27933]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:11.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:11 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T07:10:11.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:11 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:11.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:11 smithi133 ceph-mon[31198]: Deploying daemon osd.7 on smithi133 2024-04-22T07:10:11.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:11 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T07:10:11.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:11 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:11.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:11 smithi154 ceph-mon[31862]: Deploying daemon osd.7 on smithi133 2024-04-22T07:10:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:11 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T07:10:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:11 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:11 smithi113 ceph-mon[27933]: Deploying daemon osd.7 on smithi133 2024-04-22T07:10:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:12 smithi133 ceph-mon[31198]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:12.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:12 smithi154 ceph-mon[31862]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:12.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:12 smithi113 ceph-mon[27933]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:14.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:14 smithi133 ceph-mon[31198]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:14.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:14 smithi154 ceph-mon[31862]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:14 smithi113 ceph-mon[27933]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:16.232 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:16 smithi133 ceph-mon[31198]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:16.233 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:16.233 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:16.233 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:16.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:16 smithi154 ceph-mon[31862]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:16.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:16 smithi113 ceph-mon[27933]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:16.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:16.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:16.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:18.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:18 smithi133 ceph-mon[31198]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:18 smithi154 ceph-mon[31862]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:18.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:18 smithi113 ceph-mon[27933]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:19.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.430+0000 7f24ed7fa700 1 -- 172.21.15.133:0/1350752097 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f24f0062e80 con 0x7f24d80a3c40 2024-04-22T07:10:19.429 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 7 on host 'smithi133' 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.433+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24d80a3c40 msgr2=0x7f24d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.433+0000 7f24f6841700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24d80a3c40 0x7f24d80a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f24f019a1e0 tx=0x7f24e000b410 comp rx=0 tx=0).stop 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 msgr2=0x7f24f01a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 0x7f24f01a0280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b9e0 tx=0x7f24e800bcf0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 shutdown_connections 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24d80a3c40 0x7f24d80a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f0108780 0x7f24f01a0280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24f0103520 0x7f24f0198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 --2- 172.21.15.133:0/1350752097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24f0105e50 0x7f24f0199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 >> 172.21.15.133:0/1350752097 conn(0x7f24f00fcf30 msgr2=0x7f24f0103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:10:19.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 shutdown_connections 2024-04-22T07:10:19.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:10:19.434+0000 7f24f6841700 1 -- 172.21.15.133:0/1350752097 wait complete. 2024-04-22T07:10:19.733 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:19.734 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:19.735 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:19.735 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T07:10:19.735 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:19 smithi133 ceph-mon[31198]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:20.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T07:10:20.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:19 smithi154 ceph-mon[31862]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T07:10:20.068 DEBUG:teuthology.orchestra.run.smithi133:osd.7> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.7.service 2024-04-22T07:10:20.070 INFO:tasks.cephadm:Deploying osd.8 on smithi154 with /dev/vg_nvme/lv_2... 2024-04-22T07:10:20.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:20.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:20.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:20.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T07:10:20.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:19 smithi113 ceph-mon[27933]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T07:10:20.314 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:10:21.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:21 smithi154 ceph-mon[31862]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T07:10:21.680 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:21 smithi154 ceph-mon[31862]: osdmap e49: 8 total, 7 up, 8 in 2024-04-22T07:10:21.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:21 smithi154 ceph-mon[31862]: from='osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:10:21.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:21 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:21.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:21 smithi154 ceph-mon[31862]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:10:21.681 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:21 smithi154 ceph-mon[31862]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:21 smithi133 ceph-mon[31198]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T07:10:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:21 smithi133 ceph-mon[31198]: osdmap e49: 8 total, 7 up, 8 in 2024-04-22T07:10:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:21 smithi133 ceph-mon[31198]: from='osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:10:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:21 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:21 smithi133 ceph-mon[31198]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:10:21.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:21 smithi133 ceph-mon[31198]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:21 smithi113 ceph-mon[27933]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T07:10:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:21 smithi113 ceph-mon[27933]: osdmap e49: 8 total, 7 up, 8 in 2024-04-22T07:10:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:21 smithi113 ceph-mon[27933]: from='osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:10:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:21 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:21 smithi113 ceph-mon[27933]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:10:21.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:21 smithi113 ceph-mon[27933]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T07:10:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T07:10:22.392 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-22T07:10:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-22T07:10:22.392 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01149 s, 913 MB/s 2024-04-22T07:10:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-22T07:10:22.690 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:22 smithi154 ceph-mon[31862]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:10:22.690 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:22 smithi154 ceph-mon[31862]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T07:10:22.690 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:22.690 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:22 smithi154 ceph-mon[31862]: from='osd.7 ' entity='osd.7' 2024-04-22T07:10:22.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:22 smithi133 ceph-mon[31198]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:10:22.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:22 smithi133 ceph-mon[31198]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T07:10:22.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:22.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:22 smithi133 ceph-mon[31198]: from='osd.7 ' entity='osd.7' 2024-04-22T07:10:22.705 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:10:22 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:10:22.366+0000 7efeb48f0700 -1 osd.7 0 waiting for initial osdmap 2024-04-22T07:10:22.705 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:10:22 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:10:22.539+0000 7efeac6db700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:10:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:22 smithi113 ceph-mon[27933]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:10:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:22 smithi113 ceph-mon[27933]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T07:10:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:22 smithi113 ceph-mon[27933]: from='osd.7 ' entity='osd.7' 2024-04-22T07:10:23.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_2 2024-04-22T07:10:23.272 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi133 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.665 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi133 to 5696M 2024-04-22T07:10:23.666 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.666 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:23.666 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:23.666 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:10:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:10:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T07:10:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:23.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi133 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi133 to 5696M 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:23.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi133 2024-04-22T07:10:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi133 to 5696M 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:23.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:24.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:24 smithi154 ceph-mon[31862]: osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] boot 2024-04-22T07:10:24.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:24 smithi154 ceph-mon[31862]: osdmap e52: 8 total, 8 up, 8 in 2024-04-22T07:10:24.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:24 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:24.636 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:24 smithi154 ceph-mon[31862]: osdmap e53: 8 total, 8 up, 8 in 2024-04-22T07:10:24.735 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:24 smithi113 ceph-mon[27933]: osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] boot 2024-04-22T07:10:24.735 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:24 smithi113 ceph-mon[27933]: osdmap e52: 8 total, 8 up, 8 in 2024-04-22T07:10:24.735 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:24 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:24.736 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:24 smithi113 ceph-mon[27933]: osdmap e53: 8 total, 8 up, 8 in 2024-04-22T07:10:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:24 smithi133 ceph-mon[31198]: osd.7 [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] boot 2024-04-22T07:10:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:24 smithi133 ceph-mon[31198]: osdmap e52: 8 total, 8 up, 8 in 2024-04-22T07:10:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:24 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:10:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:24 smithi133 ceph-mon[31198]: osdmap e53: 8 total, 8 up, 8 in 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 -- 172.21.15.154:0/4179746077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 msgr2=0x7f1ad810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 --2- 172.21.15.154:0/4179746077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad810ac40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b0a0 tx=0x7f1ad000b3b0 comp rx=0 tx=0).stop 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 -- 172.21.15.154:0/4179746077 shutdown_connections 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 --2- 172.21.15.154:0/4179746077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad810ac40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 --2- 172.21.15.154:0/4179746077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ad81035d0 0x7f1ad81059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 --2- 172.21.15.154:0/4179746077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ad8105f00 0x7f1ad81082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.836+0000 7f1adf791700 1 -- 172.21.15.154:0/4179746077 >> 172.21.15.154:0/4179746077 conn(0x7f1ad80fcfa0 msgr2=0x7f1ad80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 -- 172.21.15.154:0/4179746077 shutdown_connections 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 -- 172.21.15.154:0/4179746077 wait complete. 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 Processor -- start 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 -- start start 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ad81035d0 0x7f1ad8102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ad8105f00 0x7f1ad8101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad8101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad810e720 con 0x7f1ad8105f00 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad810e5a0 con 0x7f1ad81035d0 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1adf791700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad810e8a0 con 0x7f1ad8108830 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.837+0000 7f1addd2e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad8101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad8101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39104/0 (socket says 172.21.15.154:39104) 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 -- 172.21.15.154:0/3861758156 learned_addr learned my addr 172.21.15.154:0/3861758156 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1adcd2c700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ad8105f00 0x7f1ad8101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:24.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1add52d700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ad81035d0 0x7f1ad8102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 -- 172.21.15.154:0/3861758156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ad81035d0 msgr2=0x7f1ad8102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ad81035d0 0x7f1ad8102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 -- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ad8105f00 msgr2=0x7f1ad8101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ad8105f00 0x7f1ad8101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 -- 172.21.15.154:0/3861758156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad8102370 con 0x7f1ad8108830 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1addd2e700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad8101ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b830 tx=0x7f1ad0007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000bd70 con 0x7f1ad8108830 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1ad0022070 con 0x7f1ad8108830 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad81a2dd0 con 0x7f1ad8108830 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001b7d0 con 0x7f1ad8108830 2024-04-22T07:10:24.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.838+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81a3240 con 0x7f1ad8108830 2024-04-22T07:10:24.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.839+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad8069730 con 0x7f1ad8108830 2024-04-22T07:10:24.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.841+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1ad001b970 con 0x7f1ad8108830 2024-04-22T07:10:24.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.842+0000 7f1ace7fc700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ac40a3c40 0x7f1ac40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:24.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.842+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f1ad000f020 con 0x7f1ad8108830 2024-04-22T07:10:24.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.842+0000 7f1add52d700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ac40a3c40 0x7f1ac40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:24.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.842+0000 7f1add52d700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ac40a3c40 0x7f1ac40a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4004630 tx=0x7f1ad400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:10:24.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:24.846+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ad009e490 con 0x7f1ad8108830 2024-04-22T07:10:25.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:25.053+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1ad8062e80 con 0x7f1ac40a3c40 2024-04-22T07:10:25.756 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:25 smithi113 ceph-mon[27933]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:25.756 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:25 smithi113 ceph-mon[27933]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:25.756 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:25.756 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:25.756 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:25.756 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:25 smithi113 ceph-mon[27933]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T07:10:25.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:25 smithi154 ceph-mon[31862]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:25.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:25 smithi154 ceph-mon[31862]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:25.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:25.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:25.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:25.772 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:25 smithi154 ceph-mon[31862]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T07:10:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:25 smithi133 ceph-mon[31198]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:25 smithi133 ceph-mon[31198]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:25.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:25 smithi133 ceph-mon[31198]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T07:10:27.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:27 smithi133 ceph-mon[31198]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T07:10:27.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:27 smithi133 ceph-mon[31198]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:27.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:27 smithi154 ceph-mon[31862]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T07:10:27.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:27 smithi154 ceph-mon[31862]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:27.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:27 smithi113 ceph-mon[27933]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T07:10:27.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:27 smithi113 ceph-mon[27933]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/325028149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]: dispatch 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]: dispatch 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]': finished 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: osdmap e57: 9 total, 8 up, 9 in 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:28 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/263879487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:28.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T07:10:28.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/325028149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]: dispatch 2024-04-22T07:10:28.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]: dispatch 2024-04-22T07:10:28.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]': finished 2024-04-22T07:10:28.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: osdmap e57: 9 total, 8 up, 9 in 2024-04-22T07:10:28.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:28.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:28 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/263879487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/325028149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]: dispatch 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]: dispatch 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4f5d70d-b3b8-447c-8d18-60e6ac682dc4"}]': finished 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: osdmap e57: 9 total, 8 up, 9 in 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:28 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/263879487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:29.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:29 smithi133 ceph-mon[31198]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:29.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:29 smithi154 ceph-mon[31862]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:29.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:29 smithi113 ceph-mon[27933]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:32.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:32 smithi154 ceph-mon[31862]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:32 smithi133 ceph-mon[31198]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:32 smithi113 ceph-mon[27933]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:34.276 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:34 smithi154 ceph-mon[31862]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:34.366 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:34 smithi133 ceph-mon[31198]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:34.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:34 smithi113 ceph-mon[27933]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:36.304 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:36 smithi154 ceph-mon[31862]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:36.304 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:36 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T07:10:36.304 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:36 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:36.449 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:36 smithi113 ceph-mon[27933]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:36.449 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:36 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T07:10:36.449 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:36 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:36.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:36 smithi133 ceph-mon[31198]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:36.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:36 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T07:10:36.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:36 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:37.260 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:37 smithi154 ceph-mon[31862]: Deploying daemon osd.8 on smithi154 2024-04-22T07:10:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:37 smithi113 ceph-mon[27933]: Deploying daemon osd.8 on smithi154 2024-04-22T07:10:37.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:37 smithi133 ceph-mon[31198]: Deploying daemon osd.8 on smithi154 2024-04-22T07:10:38.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:38 smithi154 ceph-mon[31862]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:38 smithi133 ceph-mon[31198]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:38.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:38 smithi113 ceph-mon[27933]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:40.432 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:40 smithi154 ceph-mon[31862]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:40.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:40 smithi133 ceph-mon[31198]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:40.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:40 smithi113 ceph-mon[27933]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:42 smithi133 ceph-mon[31198]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:42.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:42.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:42.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:42 smithi154 ceph-mon[31862]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:42.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:42 smithi113 ceph-mon[27933]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:44.311 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:44 smithi154 ceph-mon[31862]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:44.372 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 8 on host 'smithi154' 2024-04-22T07:10:44.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.374+0000 7f1ace7fc700 1 -- 172.21.15.154:0/3861758156 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ad8062e80 con 0x7f1ac40a3c40 2024-04-22T07:10:44.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ac40a3c40 msgr2=0x7f1ac40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:44.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ac40a3c40 0x7f1ac40a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4004630 tx=0x7f1ad400b3c0 comp rx=0 tx=0).stop 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 msgr2=0x7f1ad8101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad8101ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b830 tx=0x7f1ad0007d30 comp rx=0 tx=0).stop 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 shutdown_connections 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1ac40a3c40 0x7f1ac40a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ad8108830 0x7f1ad8101ac0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ad81035d0 0x7f1ad8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 --2- 172.21.15.154:0/3861758156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ad8105f00 0x7f1ad8101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 >> 172.21.15.154:0/3861758156 conn(0x7f1ad80fcfa0 msgr2=0x7f1ad8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.376+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 shutdown_connections 2024-04-22T07:10:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:10:44.377+0000 7f1adf791700 1 -- 172.21.15.154:0/3861758156 wait complete. 2024-04-22T07:10:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:44 smithi133 ceph-mon[31198]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:44 smithi113 ceph-mon[27933]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:45.066 DEBUG:teuthology.orchestra.run.smithi154:osd.8> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.8.service 2024-04-22T07:10:45.068 INFO:tasks.cephadm:Deploying osd.9 on smithi113 with /dev/vg_nvme/lv_1... 2024-04-22T07:10:45.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T07:10:45.318 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: from='osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: from='osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: from='osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T07:10:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T07:10:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-mon[31862]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T07:10:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-mon[31862]: osdmap e58: 9 total, 8 up, 9 in 2024-04-22T07:10:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-mon[31862]: from='osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:10:46.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-mon[31862]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:10:46.567 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:10:46.364+0000 7efd6257a700 -1 osd.8 0 waiting for initial osdmap 2024-04-22T07:10:46.567 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:10:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:10:46.380+0000 7efd5e36d700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:10:46.633 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:46 smithi133 ceph-mon[31198]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T07:10:46.634 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:46 smithi133 ceph-mon[31198]: osdmap e58: 9 total, 8 up, 9 in 2024-04-22T07:10:46.634 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:46 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:46.634 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:46 smithi133 ceph-mon[31198]: from='osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:10:46.634 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:46 smithi133 ceph-mon[31198]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:10:46.674 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:46 smithi113 ceph-mon[27933]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T07:10:46.674 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:46 smithi113 ceph-mon[27933]: osdmap e58: 9 total, 8 up, 9 in 2024-04-22T07:10:46.674 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:46 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:46.674 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:46 smithi113 ceph-mon[27933]: from='osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:10:46.674 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:46 smithi113 ceph-mon[27933]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:10:47.561 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T07:10:47.561 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T07:10:47.561 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-04-22T07:10:47.561 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-04-22T07:10:47.561 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00946875 s, 1.1 GB/s 2024-04-22T07:10:47.562 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-04-22T07:10:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:47 smithi154 ceph-mon[31862]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:10:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:47 smithi154 ceph-mon[31862]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T07:10:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:47.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:47 smithi154 ceph-mon[31862]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:47.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:47 smithi113 ceph-mon[27933]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:10:47.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:47 smithi113 ceph-mon[27933]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T07:10:47.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:47.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:47.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:47 smithi113 ceph-mon[27933]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:47.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:47 smithi133 ceph-mon[31198]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:10:47.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:47 smithi133 ceph-mon[31198]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T07:10:47.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:47.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:47.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:47 smithi133 ceph-mon[31198]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-22T07:10:48.184 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_1 2024-04-22T07:10:48.442 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:10:48.564 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:48 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:10:48.564 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:48 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:10:48.564 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:48 smithi113 ceph-mon[27933]: osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] boot 2024-04-22T07:10:48.564 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:48 smithi113 ceph-mon[27933]: osdmap e60: 9 total, 9 up, 9 in 2024-04-22T07:10:48.564 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:48 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:48.565 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:48 smithi113 ceph-mon[27933]: osdmap e61: 9 total, 9 up, 9 in 2024-04-22T07:10:48.687 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:48 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:10:48.687 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:48 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:10:48.687 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:48 smithi154 ceph-mon[31862]: osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] boot 2024-04-22T07:10:48.687 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:48 smithi154 ceph-mon[31862]: osdmap e60: 9 total, 9 up, 9 in 2024-04-22T07:10:48.687 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:48 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:48.687 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:48 smithi154 ceph-mon[31862]: osdmap e61: 9 total, 9 up, 9 in 2024-04-22T07:10:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:48 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:10:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:48 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:10:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:48 smithi133 ceph-mon[31198]: osd.8 [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] boot 2024-04-22T07:10:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:48 smithi133 ceph-mon[31198]: osdmap e60: 9 total, 9 up, 9 in 2024-04-22T07:10:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:48 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:10:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:48 smithi133 ceph-mon[31198]: osdmap e61: 9 total, 9 up, 9 in 2024-04-22T07:10:49.896 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi154 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi154 to 5696M 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:49.897 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:49.898 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:49.898 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:10:49.898 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:49 smithi113 ceph-mon[27933]: osdmap e62: 9 total, 9 up, 9 in 2024-04-22T07:10:50.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi154 2024-04-22T07:10:50.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:50.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi154 to 5696M 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:10:50.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:49 smithi133 ceph-mon[31198]: osdmap e62: 9 total, 9 up, 9 in 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi154 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi154 to 5696M 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:10:50.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:49 smithi154 ceph-mon[31862]: osdmap e62: 9 total, 9 up, 9 in 2024-04-22T07:10:50.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 -- 172.21.15.113:0/3174064783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8101340 msgr2=0x7fbfe8103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:50.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 --2- 172.21.15.113:0/3174064783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8101340 0x7fbfe8103770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009990 tx=0x7fbfe4009ca0 comp rx=0 tx=0).stop 2024-04-22T07:10:50.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 -- 172.21.15.113:0/3174064783 shutdown_connections 2024-04-22T07:10:50.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 --2- 172.21.15.113:0/3174064783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe8106680 0x7fbfe8106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:50.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 --2- 172.21.15.113:0/3174064783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8101340 0x7fbfe8103770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:50.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 --2- 172.21.15.113:0/3174064783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfe8103cb0 0x7fbfe8106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:50.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 -- 172.21.15.113:0/3174064783 >> 172.21.15.113:0/3174064783 conn(0x7fbfe80fcfa0 msgr2=0x7fbfe80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:10:50.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 -- 172.21.15.113:0/3174064783 shutdown_connections 2024-04-22T07:10:50.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 -- 172.21.15.113:0/3174064783 wait complete. 2024-04-22T07:10:50.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 Processor -- start 2024-04-22T07:10:50.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.115+0000 7fbfeff69700 1 -- start start 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe8101340 0x7fbfe819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfe8103cb0 0x7fbfe819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 0x7fbfe81a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8114c20 con 0x7fbfe8106680 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8114aa0 con 0x7fbfe8103cb0 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe8114da0 con 0x7fbfe8101340 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 0x7fbfe81a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 0x7fbfe81a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44346/0 (socket says 172.21.15.113:44346) 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 -- 172.21.15.113:0/1631673665 learned_addr learned my addr 172.21.15.113:0/1631673665 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfedd05700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe8101340 0x7fbfe819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfed504700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfe8103cb0 0x7fbfe819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 -- 172.21.15.113:0/1631673665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe8101340 msgr2=0x7fbfe819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:50.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe8101340 0x7fbfe819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 -- 172.21.15.113:0/1631673665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfe8103cb0 msgr2=0x7fbfe819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfe8103cb0 0x7fbfe819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 -- 172.21.15.113:0/1631673665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe81a6ec0 con 0x7fbfe8106680 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfee506700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 0x7fbfe81a6820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0007f10 tx=0x7fbfe000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0004bb0 con 0x7fbfe8106680 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbfe0004d50 con 0x7fbfe8106680 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe81a71b0 con 0x7fbfe8106680 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.116+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00133f0 con 0x7fbfe8106680 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.117+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81a7740 con 0x7fbfe8106680 2024-04-22T07:10:50.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.117+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe8069730 con 0x7fbfe8106680 2024-04-22T07:10:50.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.118+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbfe001a020 con 0x7fbfe8106680 2024-04-22T07:10:50.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.119+0000 7fbfdeffd700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfd40a3cc0 0x7fbfd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:50.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.119+0000 7fbfedd05700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfd40a3cc0 0x7fbfd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:10:50.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.119+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7fbfe00cf050 con 0x7fbfe8106680 2024-04-22T07:10:50.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.119+0000 7fbfedd05700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfd40a3cc0 0x7fbfd40a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b5c0 tx=0x7fbfe4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:10:50.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.124+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfe0010070 con 0x7fbfe8106680 2024-04-22T07:10:50.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:10:50.332+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbfe8062f40 con 0x7fbfd40a3cc0 2024-04-22T07:10:51.554 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:51 smithi113 ceph-mon[27933]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T07:10:51.554 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:51 smithi113 ceph-mon[27933]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:51.554 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:51.554 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:51.554 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:51.555 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:51 smithi113 ceph-mon[27933]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:10:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:51 smithi133 ceph-mon[31198]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T07:10:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:51 smithi133 ceph-mon[31198]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:51 smithi133 ceph-mon[31198]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:10:51.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:51 smithi154 ceph-mon[31862]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T07:10:51.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:51 smithi154 ceph-mon[31862]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:10:51.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:10:51.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:10:51.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:10:51.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:51 smithi154 ceph-mon[31862]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:10:52.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:52 smithi133 ceph-mon[31198]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T07:10:52.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:52 smithi154 ceph-mon[31862]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T07:10:52.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:52 smithi113 ceph-mon[27933]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T07:10:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:53 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1880164820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9a6649c-b736-4f95-986a-2ff0463c0995"}]: dispatch 2024-04-22T07:10:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:53 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1880164820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9a6649c-b736-4f95-986a-2ff0463c0995"}]': finished 2024-04-22T07:10:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:53 smithi133 ceph-mon[31198]: osdmap e65: 10 total, 9 up, 10 in 2024-04-22T07:10:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:53 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:10:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:53 smithi133 ceph-mon[31198]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:10:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1880164820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9a6649c-b736-4f95-986a-2ff0463c0995"}]: dispatch 2024-04-22T07:10:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1880164820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9a6649c-b736-4f95-986a-2ff0463c0995"}]': finished 2024-04-22T07:10:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:53 smithi154 ceph-mon[31862]: osdmap e65: 10 total, 9 up, 10 in 2024-04-22T07:10:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:53 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:10:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:53 smithi154 ceph-mon[31862]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:10:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1880164820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9a6649c-b736-4f95-986a-2ff0463c0995"}]: dispatch 2024-04-22T07:10:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1880164820' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9a6649c-b736-4f95-986a-2ff0463c0995"}]': finished 2024-04-22T07:10:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:53 smithi113 ceph-mon[27933]: osdmap e65: 10 total, 9 up, 10 in 2024-04-22T07:10:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:53 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:10:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:53 smithi113 ceph-mon[27933]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:10:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:54 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1580845861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:54.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:54 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1580845861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:54.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:54 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1580845861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:10:55.324 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:10:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:10:55.011+0000 7f1dcb12e700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-22T07:10:55.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:55 smithi113 ceph-mon[27933]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-22T07:10:55.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:55 smithi133 ceph-mon[31198]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-22T07:10:55.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:55 smithi154 ceph-mon[31862]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-22T07:10:58.431 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:10:58 smithi113 ceph-mon[27933]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:10:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:10:58 smithi133 ceph-mon[31198]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:10:58.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:10:58 smithi154 ceph-mon[31862]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:00 smithi113 ceph-mon[27933]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:00 smithi133 ceph-mon[31198]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:00.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:00 smithi154 ceph-mon[31862]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:01.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T07:11:01.479 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T07:11:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T07:11:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:02.453 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:02 smithi113 ceph-mon[27933]: Deploying daemon osd.9 on smithi113 2024-04-22T07:11:02.453 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:02 smithi113 ceph-mon[27933]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:02.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:02 smithi133 ceph-mon[31198]: Deploying daemon osd.9 on smithi113 2024-04-22T07:11:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:02 smithi133 ceph-mon[31198]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:02 smithi154 ceph-mon[31862]: Deploying daemon osd.9 on smithi113 2024-04-22T07:11:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:02 smithi154 ceph-mon[31862]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:04.299 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:04 smithi113 ceph-mon[27933]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:04.366 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:04 smithi133 ceph-mon[31198]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:04.368 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:04 smithi154 ceph-mon[31862]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:06.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:06 smithi113 ceph-mon[27933]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:06.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:06 smithi133 ceph-mon[31198]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:06.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:06 smithi154 ceph-mon[31862]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:07.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:07.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:07.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:07 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:07.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:07 smithi154 ceph-mon[31862]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:07 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:07 smithi113 ceph-mon[27933]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:08.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:08.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:08.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:07 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:08.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:07 smithi133 ceph-mon[31198]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:10.400 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:10 smithi113 ceph-mon[27933]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:10.400 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.400 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.400 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:10.401 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:10.401 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:10 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.487 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 9 on host 'smithi113' 2024-04-22T07:11:10.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.489+0000 7fbfdeffd700 1 -- 172.21.15.113:0/1631673665 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfe8062f40 con 0x7fbfd40a3cc0 2024-04-22T07:11:10.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.491+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfd40a3cc0 msgr2=0x7fbfd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:10.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.491+0000 7fbfeff69700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfd40a3cc0 0x7fbfd40a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b5c0 tx=0x7fbfe4005fb0 comp rx=0 tx=0).stop 2024-04-22T07:11:10.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.491+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 msgr2=0x7fbfe81a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:10.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.491+0000 7fbfeff69700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 0x7fbfe81a6820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0007f10 tx=0x7fbfe000d3b0 comp rx=0 tx=0).stop 2024-04-22T07:11:10.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 shutdown_connections 2024-04-22T07:11:10.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfd40a3cc0 0x7fbfd40a6180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:10.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfe8106680 0x7fbfe81a6820 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:10.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe8101340 0x7fbfe819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:10.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 --2- 172.21.15.113:0/1631673665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfe8103cb0 0x7fbfe819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:10.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 >> 172.21.15.113:0/1631673665 conn(0x7fbfe80fcfa0 msgr2=0x7fbfe8104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:11:10.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 shutdown_connections 2024-04-22T07:11:10.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:11:10.492+0000 7fbfeff69700 1 -- 172.21.15.113:0/1631673665 wait complete. 2024-04-22T07:11:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:10 smithi133 ceph-mon[31198]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:10 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:10 smithi154 ceph-mon[31862]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:10 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.181 DEBUG:teuthology.orchestra.run.smithi113:osd.9> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.9.service 2024-04-22T07:11:11.183 INFO:tasks.cephadm:Deploying osd.10 on smithi133 with /dev/vg_nvme/lv_1... 2024-04-22T07:11:11.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T07:11:11.429 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:11:11.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:11 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:11 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:11 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:11.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:11 smithi133 ceph-mon[31198]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T07:11:11.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:11 smithi133 ceph-mon[31198]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:11.699 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:11 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.700 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:11 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.700 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:11 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:11.700 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:11 smithi113 ceph-mon[27933]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T07:11:11.700 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:11 smithi113 ceph-mon[27933]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:11 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:11 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:11 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:11 smithi154 ceph-mon[31862]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T07:11:11.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:11 smithi154 ceph-mon[31862]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:12.728 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:12 smithi133 ceph-mon[31198]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T07:11:12.728 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:12 smithi133 ceph-mon[31198]: osdmap e66: 10 total, 9 up, 10 in 2024-04-22T07:11:12.728 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:12 smithi133 ceph-mon[31198]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:11:12.728 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:12 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:12.729 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:12 smithi113 ceph-mon[27933]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T07:11:12.729 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:12 smithi113 ceph-mon[27933]: osdmap e66: 10 total, 9 up, 10 in 2024-04-22T07:11:12.729 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:12 smithi113 ceph-mon[27933]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:11:12.729 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:12 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:12 smithi154 ceph-mon[31862]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T07:11:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:12 smithi154 ceph-mon[31862]: osdmap e66: 10 total, 9 up, 10 in 2024-04-22T07:11:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:12 smithi154 ceph-mon[31862]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-04-22T07:11:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:12 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.438 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T07:11:13.439 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T07:11:13.439 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-22T07:11:13.439 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-22T07:11:13.439 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0152829 s, 686 MB/s 2024-04-22T07:11:13.439 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-22T07:11:13.457 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:11:13.194+0000 7f8b79f53700 -1 osd.9 0 waiting for initial osdmap 2024-04-22T07:11:13.457 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:11:13.233+0000 7f8b71d3e700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:11:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:13 smithi133 ceph-mon[31198]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:11:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:13 smithi133 ceph-mon[31198]: osdmap e67: 10 total, 9 up, 10 in 2024-04-22T07:11:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:13 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:13 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:13 smithi133 ceph-mon[31198]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:13.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:13 smithi133 ceph-mon[31198]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' 2024-04-22T07:11:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:13 smithi154 ceph-mon[31862]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:11:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:13 smithi154 ceph-mon[31862]: osdmap e67: 10 total, 9 up, 10 in 2024-04-22T07:11:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:13 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:13 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:13 smithi154 ceph-mon[31862]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:13.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:13 smithi154 ceph-mon[31862]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' 2024-04-22T07:11:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-mon[27933]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-04-22T07:11:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-mon[27933]: osdmap e67: 10 total, 9 up, 10 in 2024-04-22T07:11:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:13.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-mon[27933]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-22T07:11:13.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:13 smithi113 ceph-mon[27933]: from='osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950]' entity='osd.9' 2024-04-22T07:11:14.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-04-22T07:11:14.316 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:11:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:11:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] boot 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: osdmap e68: 10 total, 10 up, 10 in 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:14.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:14.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:14 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.767 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:11:14.767 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:11:14.767 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:14.767 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] boot 2024-04-22T07:11:14.767 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: osdmap e68: 10 total, 10 up, 10 in 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:14.768 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:14 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: osd.9 [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] boot 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: osdmap e68: 10 total, 10 up, 10 in 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:14.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:14.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:14.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:14.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:14 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:15.748 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:15 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi113 2024-04-22T07:11:15.748 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:15 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi113 to 4272M 2024-04-22T07:11:15.749 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:15 smithi133 ceph-mon[31198]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:15.749 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:15 smithi133 ceph-mon[31198]: osdmap e69: 10 total, 10 up, 10 in 2024-04-22T07:11:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:15 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi113 2024-04-22T07:11:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:15 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi113 to 4272M 2024-04-22T07:11:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:15 smithi154 ceph-mon[31862]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:15.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:15 smithi154 ceph-mon[31862]: osdmap e69: 10 total, 10 up, 10 in 2024-04-22T07:11:15.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:15 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi113 2024-04-22T07:11:15.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:15 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi113 to 4272M 2024-04-22T07:11:15.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:15 smithi113 ceph-mon[27933]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:15.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:15 smithi113 ceph-mon[27933]: osdmap e69: 10 total, 10 up, 10 in 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 -- 172.21.15.133:0/3037254424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084102ac0 msgr2=0x7ff084102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 --2- 172.21.15.133:0/3037254424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084102ac0 0x7ff084102ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff0740099c0 tx=0x7ff074009cd0 comp rx=0 tx=0).stop 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 -- 172.21.15.133:0/3037254424 shutdown_connections 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 --2- 172.21.15.133:0/3037254424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff084104660 0x7ff08410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 --2- 172.21.15.133:0/3037254424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084102ac0 0x7ff084102ee0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 --2- 172.21.15.133:0/3037254424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff084103cc0 0x7ff084104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 -- 172.21.15.133:0/3037254424 >> 172.21.15.133:0/3037254424 conn(0x7ff0840fe060 msgr2=0x7ff0841004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:11:15.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 -- 172.21.15.133:0/3037254424 shutdown_connections 2024-04-22T07:11:15.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.853+0000 7ff088a95700 1 -- 172.21.15.133:0/3037254424 wait complete. 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 Processor -- start 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 -- start start 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff084102ac0 0x7ff0840767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff084103cc0 0x7ff084074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 0x7ff084075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410e690 con 0x7ff084103cc0 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410e510 con 0x7ff084104660 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff088a95700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410e810 con 0x7ff084102ac0 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 0x7ff084075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 0x7ff084075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46654/0 (socket says 172.21.15.133:46654) 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 -- 172.21.15.133:0/1420131640 learned_addr learned my addr 172.21.15.133:0/1420131640 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-22T07:11:15.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff081d9b700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff084103cc0 0x7ff084074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff08259c700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff084102ac0 0x7ff0840767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 -- 172.21.15.133:0/1420131640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff084102ac0 msgr2=0x7ff0840767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff084102ac0 0x7ff0840767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 -- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff084103cc0 msgr2=0x7ff084074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff084103cc0 0x7ff084074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 -- 172.21.15.133:0/1420131640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff084075c20 con 0x7ff084104660 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff082d9d700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 0x7ff084075370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00ef30 tx=0x7ff07c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.854+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0092e0 con 0x7ff084104660 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.855+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0841a2db0 con 0x7ff084104660 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.855+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff07c004bb0 con 0x7ff084104660 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.855+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c004280 con 0x7ff084104660 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.855+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841a3210 con 0x7ff084104660 2024-04-22T07:11:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.855+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08402df80 con 0x7ff084104660 2024-04-22T07:11:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.857+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff07c01b030 con 0x7ff084104660 2024-04-22T07:11:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.857+0000 7ff0737fe700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff06c0a3c40 0x7ff06c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.857+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7ff07c0d0080 con 0x7ff084104660 2024-04-22T07:11:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.857+0000 7ff08259c700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff06c0a3c40 0x7ff06c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.858+0000 7ff08259c700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff06c0a3c40 0x7ff06c0a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b580 tx=0x7ff074005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:11:15.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:15.861+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff07c022020 con 0x7ff084104660 2024-04-22T07:11:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:16.070+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff084062e80 con 0x7ff06c0a3c40 2024-04-22T07:11:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:16 smithi133 ceph-mon[31198]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:11:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:11:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:11:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:16.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:16 smithi154 ceph-mon[31862]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:11:16.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:11:16.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:11:16.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:16 smithi113 ceph-mon[27933]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:11:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:11:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:11:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:17.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:17 smithi133 ceph-mon[31198]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:17.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:17 smithi154 ceph-mon[31862]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:17.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:17 smithi113 ceph-mon[27933]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:19.194 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/1597168669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]: dispatch 2024-04-22T07:11:19.194 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:18 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]: dispatch 2024-04-22T07:11:19.194 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:18 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]': finished 2024-04-22T07:11:19.194 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:18 smithi133 ceph-mon[31198]: osdmap e70: 11 total, 10 up, 11 in 2024-04-22T07:11:19.194 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:18 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/1597168669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]: dispatch 2024-04-22T07:11:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:18 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]: dispatch 2024-04-22T07:11:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:18 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]': finished 2024-04-22T07:11:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:18 smithi154 ceph-mon[31862]: osdmap e70: 11 total, 10 up, 11 in 2024-04-22T07:11:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:18 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/1597168669' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]: dispatch 2024-04-22T07:11:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:18 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]: dispatch 2024-04-22T07:11:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:18 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513726aa-29df-4ae7-bebc-db08c423daf4"}]': finished 2024-04-22T07:11:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:18 smithi113 ceph-mon[27933]: osdmap e70: 11 total, 10 up, 11 in 2024-04-22T07:11:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:18 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:20.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:19 smithi133 ceph-mon[31198]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:20.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:19 smithi133 ceph-mon[31198]: from='client.? 172.21.15.133:0/932619139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:11:20.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:19 smithi154 ceph-mon[31862]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:20.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:19 smithi154 ceph-mon[31862]: from='client.? 172.21.15.133:0/932619139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:11:20.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:19 smithi113 ceph-mon[27933]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:20.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:19 smithi113 ceph-mon[27933]: from='client.? 172.21.15.133:0/932619139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:11:22.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:22 smithi133 ceph-mon[31198]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:22.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:22 smithi154 ceph-mon[31862]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:22.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:22 smithi113 ceph-mon[27933]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:24.374 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:24 smithi133 ceph-mon[31198]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:24.385 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:24 smithi113 ceph-mon[27933]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:24.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:24 smithi154 ceph-mon[31862]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:26.367 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:26 smithi133 ceph-mon[31198]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:26.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:26 smithi154 ceph-mon[31862]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:26.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:26 smithi113 ceph-mon[27933]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:27.294 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:27 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T07:11:27.295 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:27 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:27.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:27 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T07:11:27.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:27 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:27.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T07:11:27.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:27 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:28.181 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:28 smithi133 ceph-mon[31198]: Deploying daemon osd.10 on smithi133 2024-04-22T07:11:28.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:28 smithi133 ceph-mon[31198]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:28.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:28 smithi154 ceph-mon[31862]: Deploying daemon osd.10 on smithi133 2024-04-22T07:11:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:28 smithi154 ceph-mon[31862]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:28 smithi113 ceph-mon[27933]: Deploying daemon osd.10 on smithi133 2024-04-22T07:11:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:28 smithi113 ceph-mon[27933]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:30.226 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:30 smithi133 ceph-mon[31198]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:30 smithi154 ceph-mon[31862]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:30 smithi113 ceph-mon[27933]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:32.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:32 smithi133 ceph-mon[31198]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:32.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:32.553 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:32.553 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:32 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:32 smithi154 ceph-mon[31862]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:32 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:32 smithi113 ceph-mon[27933]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:32 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:34.352 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:34 smithi133 ceph-mon[31198]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:34.385 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:34 smithi154 ceph-mon[31862]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:34.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:34 smithi113 ceph-mon[27933]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:34.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.989+0000 7ff0737fe700 1 -- 172.21.15.133:0/1420131640 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff084062e80 con 0x7ff06c0a3c40 2024-04-22T07:11:34.988 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 10 on host 'smithi133' 2024-04-22T07:11:34.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.991+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff06c0a3c40 msgr2=0x7ff06c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff06c0a3c40 0x7ff06c0a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b580 tx=0x7ff074005fb0 comp rx=0 tx=0).stop 2024-04-22T07:11:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 msgr2=0x7ff084075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 0x7ff084075370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00ef30 tx=0x7ff07c00c3a0 comp rx=0 tx=0).stop 2024-04-22T07:11:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 shutdown_connections 2024-04-22T07:11:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff06c0a3c40 0x7ff06c0a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff084104660 0x7ff084075370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff084102ac0 0x7ff0840767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 --2- 172.21.15.133:0/1420131640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff084103cc0 0x7ff084074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 >> 172.21.15.133:0/1420131640 conn(0x7ff0840fe060 msgr2=0x7ff084106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:11:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 shutdown_connections 2024-04-22T07:11:34.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-22T07:11:34.992+0000 7ff088a95700 1 -- 172.21.15.133:0/1420131640 wait complete. 2024-04-22T07:11:35.830 DEBUG:teuthology.orchestra.run.smithi133:osd.10> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.10.service 2024-04-22T07:11:35.832 INFO:tasks.cephadm:Deploying osd.11 on smithi154 with /dev/vg_nvme/lv_1... 2024-04-22T07:11:35.833 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T07:11:36.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:35 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:36.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:35 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:36.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:35 smithi133 ceph-mon[31198]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:36.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:35 smithi133 ceph-mon[31198]: from='osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T07:11:36.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:35 smithi133 ceph-mon[31198]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T07:11:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:11:36.193 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:35 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:36.193 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:35 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:36.193 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:35 smithi154 ceph-mon[31862]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:36.194 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:35 smithi154 ceph-mon[31862]: from='osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T07:11:36.194 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:35 smithi154 ceph-mon[31862]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T07:11:36.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:35 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:36.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:35 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:36.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:35 smithi113 ceph-mon[27933]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:36.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:35 smithi113 ceph-mon[27933]: from='osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T07:11:36.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:35 smithi113 ceph-mon[27933]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: osdmap e71: 11 total, 10 up, 11 in 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: from='osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:37.040 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:37.040 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:11:36.958+0000 7f1dc44ef700 -1 osd.10 0 waiting for initial osdmap 2024-04-22T07:11:37.040 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:11:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:11:36.972+0000 7f1dc02e2700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:11:37.212 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T07:11:37.213 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: osdmap e71: 11 total, 10 up, 11 in 2024-04-22T07:11:37.213 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:37.213 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: from='osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:11:37.213 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:11:37.213 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:37.213 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:36 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T07:11:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: osdmap e71: 11 total, 10 up, 11 in 2024-04-22T07:11:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: from='osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:11:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-22T07:11:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:36 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:38.109 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:37 smithi154 ceph-mon[31862]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:11:38.109 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:37 smithi154 ceph-mon[31862]: osdmap e72: 11 total, 10 up, 11 in 2024-04-22T07:11:38.109 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:38.109 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:38.109 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:37 smithi154 ceph-mon[31862]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:38.110 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T07:11:38.110 INFO:teuthology.orchestra.run.smithi154.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T07:11:38.110 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10+0 records in 2024-04-22T07:11:38.110 INFO:teuthology.orchestra.run.smithi154.stderr:10+0 records out 2024-04-22T07:11:38.110 INFO:teuthology.orchestra.run.smithi154.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127714 s, 821 MB/s 2024-04-22T07:11:38.110 INFO:teuthology.orchestra.run.smithi154.stderr:--> Zapping successful for: 2024-04-22T07:11:38.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:37 smithi133 ceph-mon[31198]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:11:38.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:37 smithi133 ceph-mon[31198]: osdmap e72: 11 total, 10 up, 11 in 2024-04-22T07:11:38.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:38.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:38.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:37 smithi133 ceph-mon[31198]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:37 smithi113 ceph-mon[27933]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-22T07:11:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:37 smithi113 ceph-mon[27933]: osdmap e72: 11 total, 10 up, 11 in 2024-04-22T07:11:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:37 smithi113 ceph-mon[27933]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-22T07:11:38.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_1 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] boot 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: osdmap e73: 11 total, 11 up, 11 in 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:38 smithi133 ceph-mon[31198]: osdmap e74: 11 total, 11 up, 11 in 2024-04-22T07:11:39.125 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] boot 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: osdmap e73: 11 total, 11 up, 11 in 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:39.247 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:38 smithi154 ceph-mon[31862]: osdmap e74: 11 total, 11 up, 11 in 2024-04-22T07:11:39.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:11:39.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:11:39.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:39.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: osd.10 [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] boot 2024-04-22T07:11:39.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: osdmap e73: 11 total, 11 up, 11 in 2024-04-22T07:11:39.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:11:39.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:38 smithi113 ceph-mon[27933]: osdmap e74: 11 total, 11 up, 11 in 2024-04-22T07:11:40.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:40.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: osdmap e75: 11 total, 11 up, 11 in 2024-04-22T07:11:40.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:40.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: osdmap e75: 11 total, 11 up, 11 in 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:40.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:40.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: osdmap e75: 11 total, 11 up, 11 in 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:11:40.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 -- 172.21.15.154:0/3319124385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe80754a0 msgr2=0x7fefe8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3319124385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe80754a0 0x7fefe8075920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009990 tx=0x7fefd0009ca0 comp rx=0 tx=0).stop 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 -- 172.21.15.154:0/3319124385 shutdown_connections 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3319124385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe810a990 0x7fefe810ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3319124385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8076a70 0x7fefe8074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3319124385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe80754a0 0x7fefe8075920 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 -- 172.21.15.154:0/3319124385 >> 172.21.15.154:0/3319124385 conn(0x7fefe80fe060 msgr2=0x7fefe81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 -- 172.21.15.154:0/3319124385 shutdown_connections 2024-04-22T07:11:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.856+0000 7fefeda0f700 1 -- 172.21.15.154:0/3319124385 wait complete. 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 Processor -- start 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 -- start start 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80754a0 0x7fefe819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8076a70 0x7fefe819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 0x7fefe81a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8110890 con 0x7fefe80754a0 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8110710 con 0x7fefe8076a70 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefeda0f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8110a10 con 0x7fefe810a990 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 0x7fefe81a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 0x7fefe81a2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54196/0 (socket says 172.21.15.154:54196) 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 -- 172.21.15.154:0/3864793210 learned_addr learned my addr 172.21.15.154:0/3864793210 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe6ffd700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80754a0 0x7fefe819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefde7fc700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8076a70 0x7fefe819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 -- 172.21.15.154:0/3864793210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8076a70 msgr2=0x7fefe819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8076a70 0x7fefe819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 -- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80754a0 msgr2=0x7fefe819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80754a0 0x7fefe819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.857+0000 7fefe77fe700 1 -- 172.21.15.154:0/3864793210 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe81a2ab0 con 0x7fefe810a990 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.858+0000 7fefe77fe700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 0x7fefe81a2410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fefe0007d40 tx=0x7fefe000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.858+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0016020 con 0x7fefe810a990 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.858+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fefe000acb0 con 0x7fefe810a990 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.858+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe81a2da0 con 0x7fefe810a990 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.858+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00115e0 con 0x7fefe810a990 2024-04-22T07:11:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.858+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81a3200 con 0x7fefe810a990 2024-04-22T07:11:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.859+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe8069730 con 0x7fefe810a990 2024-04-22T07:11:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.860+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fefe0019020 con 0x7fefe810a990 2024-04-22T07:11:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.860+0000 7fefeca0d700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fefd40a3c40 0x7fefd40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:11:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.861+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fefe00d0050 con 0x7fefe810a990 2024-04-22T07:11:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.861+0000 7fefe6ffd700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fefd40a3c40 0x7fefd40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:11:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.861+0000 7fefe6ffd700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fefd40a3c40 0x7fefd40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fefd8005950 tx=0x7fefd800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:11:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:40.864+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefe0020020 con 0x7fefe810a990 2024-04-22T07:11:41.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:11:41.076+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fefe8062e80 con 0x7fefd40a3c40 2024-04-22T07:11:41.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:40 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi133 2024-04-22T07:11:41.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:40 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi133 to 4272M 2024-04-22T07:11:41.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:40 smithi133 ceph-mon[31198]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T07:11:41.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:40 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi133 2024-04-22T07:11:41.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:40 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi133 to 4272M 2024-04-22T07:11:41.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:40 smithi154 ceph-mon[31862]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T07:11:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:40 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi133 2024-04-22T07:11:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:40 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi133 to 4272M 2024-04-22T07:11:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:40 smithi113 ceph-mon[27933]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T07:11:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:41 smithi133 ceph-mon[31198]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:11:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:41 smithi133 ceph-mon[31198]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:11:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:41 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:11:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:41 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:11:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:41 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:42.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:41 smithi133 ceph-mon[31198]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T07:11:42.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:41 smithi154 ceph-mon[31862]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:11:42.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:41 smithi154 ceph-mon[31862]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:11:42.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:41 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:11:42.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:41 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:11:42.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:41 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:42.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:41 smithi154 ceph-mon[31862]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T07:11:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:41 smithi113 ceph-mon[27933]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T07:11:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:41 smithi113 ceph-mon[27933]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:11:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:41 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T07:11:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:41 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T07:11:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:41 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:42.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:41 smithi113 ceph-mon[27933]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T07:11:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:44 smithi133 ceph-mon[31198]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-22T07:11:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:44 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/2142841068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]: dispatch 2024-04-22T07:11:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:44 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]: dispatch 2024-04-22T07:11:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:44 smithi133 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]': finished 2024-04-22T07:11:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:44 smithi133 ceph-mon[31198]: osdmap e78: 12 total, 11 up, 12 in 2024-04-22T07:11:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:44 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:11:44.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:44 smithi154 ceph-mon[31862]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-22T07:11:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:44 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/2142841068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]: dispatch 2024-04-22T07:11:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:44 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]: dispatch 2024-04-22T07:11:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:44 smithi154 ceph-mon[31862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]': finished 2024-04-22T07:11:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:44 smithi154 ceph-mon[31862]: osdmap e78: 12 total, 11 up, 12 in 2024-04-22T07:11:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:44 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:11:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:44 smithi113 ceph-mon[27933]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-22T07:11:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:44 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/2142841068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]: dispatch 2024-04-22T07:11:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:44 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]: dispatch 2024-04-22T07:11:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:44 smithi113 ceph-mon[27933]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0998e208-1905-409f-8e8d-7e8b7d1703dd"}]': finished 2024-04-22T07:11:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:44 smithi113 ceph-mon[27933]: osdmap e78: 12 total, 11 up, 12 in 2024-04-22T07:11:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:44 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:11:45.383 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:45 smithi133 ceph-mon[31198]: from='client.? 172.21.15.154:0/3489166630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:11:45.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:45 smithi154 ceph-mon[31862]: from='client.? 172.21.15.154:0/3489166630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:11:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.154:0/3489166630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T07:11:46.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:46 smithi133 ceph-mon[31198]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-22T07:11:46.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:46 smithi154 ceph-mon[31862]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-22T07:11:46.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:46 smithi113 ceph-mon[27933]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-22T07:11:48.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:48 smithi154 ceph-mon[31862]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:48 smithi133 ceph-mon[31198]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:48 smithi113 ceph-mon[27933]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:50.369 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:50 smithi154 ceph-mon[31862]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:50.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:50 smithi133 ceph-mon[31198]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:50.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:50 smithi113 ceph-mon[27933]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:52.408 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:52 smithi154 ceph-mon[31862]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:52.408 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T07:11:52.408 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:52.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:52 smithi133 ceph-mon[31198]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:52.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T07:11:52.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:52.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:52 smithi113 ceph-mon[27933]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:52.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T07:11:52.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:11:53.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:53 smithi154 ceph-mon[31862]: Deploying daemon osd.11 on smithi154 2024-04-22T07:11:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:53 smithi133 ceph-mon[31198]: Deploying daemon osd.11 on smithi154 2024-04-22T07:11:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:53 smithi113 ceph-mon[27933]: Deploying daemon osd.11 on smithi154 2024-04-22T07:11:54.417 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:54 smithi154 ceph-mon[31862]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:54.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:54 smithi113 ceph-mon[27933]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:54 smithi133 ceph-mon[31198]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:56.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:56 smithi154 ceph-mon[31862]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:56 smithi133 ceph-mon[31198]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:56 smithi113 ceph-mon[27933]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:58 smithi133 ceph-mon[31198]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:11:58 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:58.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:58 smithi154 ceph-mon[31862]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:11:58 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:11:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:58 smithi113 ceph-mon[27933]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:11:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:11:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:11:58 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:00.508 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:00 smithi154 ceph-mon[31862]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:00 smithi133 ceph-mon[31198]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:00.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:00 smithi113 ceph-mon[27933]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:01.341 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 11 on host 'smithi154' 2024-04-22T07:12:01.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.342+0000 7fefeca0d700 1 -- 172.21.15.154:0/3864793210 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fefe8062e80 con 0x7fefd40a3c40 2024-04-22T07:12:01.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fefd40a3c40 msgr2=0x7fefd40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:01.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fefd40a3c40 0x7fefd40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fefd8005950 tx=0x7fefd800a300 comp rx=0 tx=0).stop 2024-04-22T07:12:01.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 msgr2=0x7fefe81a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:01.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 0x7fefe81a2410 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fefe0007d40 tx=0x7fefe000a340 comp rx=0 tx=0).stop 2024-04-22T07:12:01.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 shutdown_connections 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fefd40a3c40 0x7fefd40a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fefe810a990 0x7fefe81a2410 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fefe80754a0 0x7fefe819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 --2- 172.21.15.154:0/3864793210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8076a70 0x7fefe819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 >> 172.21.15.154:0/3864793210 conn(0x7fefe80fe060 msgr2=0x7fefe8106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 shutdown_connections 2024-04-22T07:12:01.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:01.345+0000 7fefeda0f700 1 -- 172.21.15.154:0/3864793210 wait complete. 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T07:12:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:01 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:01.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T07:12:01.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T07:12:01.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:01 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:01.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:01.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:01.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:01.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T07:12:01.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T07:12:01.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:01.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:01 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:01.950 DEBUG:teuthology.orchestra.run.smithi154:osd.11> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.11.service 2024-04-22T07:12:01.952 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-22T07:12:01.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T07:12:02.200 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:12:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:02 smithi113 ceph-mon[27933]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T07:12:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:02 smithi113 ceph-mon[27933]: osdmap e79: 12 total, 11 up, 12 in 2024-04-22T07:12:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:02 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:02.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:02 smithi113 ceph-mon[27933]: from='osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:12:02.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:02 smithi113 ceph-mon[27933]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:12:02.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:02 smithi154 ceph-mon[31862]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T07:12:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:02 smithi154 ceph-mon[31862]: osdmap e79: 12 total, 11 up, 12 in 2024-04-22T07:12:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:02 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:02 smithi154 ceph-mon[31862]: from='osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:12:02.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:02 smithi154 ceph-mon[31862]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:12:03.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:02 smithi133 ceph-mon[31198]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T07:12:03.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:02 smithi133 ceph-mon[31198]: osdmap e79: 12 total, 11 up, 12 in 2024-04-22T07:12:03.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:02 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:03.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:02 smithi133 ceph-mon[31198]: from='osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:12:03.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:02 smithi133 ceph-mon[31198]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]: dispatch 2024-04-22T07:12:03.598 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:03 smithi113 ceph-mon[27933]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:12:03.598 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:03 smithi113 ceph-mon[27933]: osdmap e80: 12 total, 11 up, 12 in 2024-04-22T07:12:03.598 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:03 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:03.598 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:03 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:03.598 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:03 smithi113 ceph-mon[27933]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:03.598 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:03 smithi113 ceph-mon[27933]: from='osd.11 ' entity='osd.11' 2024-04-22T07:12:03.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-mon[31862]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:12:03.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-mon[31862]: osdmap e80: 12 total, 11 up, 12 in 2024-04-22T07:12:03.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:03.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:03.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-mon[31862]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:03.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-mon[31862]: from='osd.11 ' entity='osd.11' 2024-04-22T07:12:03.817 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:12:03.472+0000 7f5d9921e700 -1 osd.11 0 waiting for initial osdmap 2024-04-22T07:12:03.817 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:12:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:12:03.648+0000 7f5d91009700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:12:03.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 -- 172.21.15.113:0/2696218539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 msgr2=0x7fcc7c109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:03.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 --2- 172.21.15.113:0/2696218539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c109de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b0d0 tx=0x7fcc7400b3e0 comp rx=0 tx=0).stop 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 -- 172.21.15.113:0/2696218539 shutdown_connections 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 --2- 172.21.15.113:0/2696218539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c109de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 --2- 172.21.15.113:0/2696218539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c101990 0x7fcc7c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 --2- 172.21.15.113:0/2696218539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc7c102b90 0x7fcc7c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 -- 172.21.15.113:0/2696218539 >> 172.21.15.113:0/2696218539 conn(0x7fcc7c0fcf30 msgr2=0x7fcc7c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 -- 172.21.15.113:0/2696218539 shutdown_connections 2024-04-22T07:12:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.912+0000 7fcc81535700 1 -- 172.21.15.113:0/2696218539 wait complete. 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 Processor -- start 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 -- start start 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc7c101990 0x7fcc7c06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c102b90 0x7fcc7c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c10d550 con 0x7fcc7c103530 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c10d3d0 con 0x7fcc7c101990 2024-04-22T07:12:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc81535700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c10d6d0 con 0x7fcc7c102b90 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc7b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.913+0000 7fcc7b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c19ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58008/0 (socket says 172.21.15.113:58008) 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 -- 172.21.15.113:0/895437466 learned_addr learned my addr 172.21.15.113:0/895437466 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7a7fc700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c102b90 0x7fcc7c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7affd700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc7c101990 0x7fcc7c06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 -- 172.21.15.113:0/895437466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c102b90 msgr2=0x7fcc7c19d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c102b90 0x7fcc7c19d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 -- 172.21.15.113:0/895437466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc7c101990 msgr2=0x7fcc7c06bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc7c101990 0x7fcc7c06bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 -- 172.21.15.113:0/895437466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c1a0840 con 0x7fcc7c103530 2024-04-22T07:12:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc7b7fe700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c19ff70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc74007d10 tx=0x7fcc74007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74021020 con 0x7fcc7c103530 2024-04-22T07:12:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c1a0ad0 con 0x7fcc7c103530 2024-04-22T07:12:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcc7401d940 con 0x7fcc7c103530 2024-04-22T07:12:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74005520 con 0x7fcc7c103530 2024-04-22T07:12:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.914+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c1a0f80 con 0x7fcc7c103530 2024-04-22T07:12:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.915+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c069730 con 0x7fcc7c103530 2024-04-22T07:12:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.916+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcc74024020 con 0x7fcc7c103530 2024-04-22T07:12:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.917+0000 7fcc63fff700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcc640a3c40 0x7fcc640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.917+0000 7fcc7affd700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcc640a3c40 0x7fcc640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.917+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7fcc7400f020 con 0x7fcc7c103530 2024-04-22T07:12:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.917+0000 7fcc7affd700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcc640a3c40 0x7fcc640a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005950 tx=0x7fcc700058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:03.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:03.921+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc74009cb0 con 0x7fcc7c103530 2024-04-22T07:12:04.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:03 smithi133 ceph-mon[31198]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-04-22T07:12:04.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:03 smithi133 ceph-mon[31198]: osdmap e80: 12 total, 11 up, 12 in 2024-04-22T07:12:04.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:03 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:04.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:03 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:04.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:03 smithi133 ceph-mon[31198]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:04.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:03 smithi133 ceph-mon[31198]: from='osd.11 ' entity='osd.11' 2024-04-22T07:12:04.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.120+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fcc7c063650 con 0x7fcc7c103530 2024-04-22T07:12:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.120+0000 7fcc63fff700 1 -- 172.21.15.113:0/895437466 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7fcc7409e630 con 0x7fcc7c103530 2024-04-22T07:12:04.118 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcc640a3c40 msgr2=0x7fcc640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcc640a3c40 0x7fcc640a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005950 tx=0x7fcc700058e0 comp rx=0 tx=0).stop 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 msgr2=0x7fcc7c19ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c19ff70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc74007d10 tx=0x7fcc74007d40 comp rx=0 tx=0).stop 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 shutdown_connections 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcc640a3c40 0x7fcc640a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc7c103530 0x7fcc7c19ff70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc7c101990 0x7fcc7c06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 --2- 172.21.15.113:0/895437466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc7c102b90 0x7fcc7c19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 >> 172.21.15.113:0/895437466 conn(0x7fcc7c0fcf30 msgr2=0x7fcc7c105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 shutdown_connections 2024-04-22T07:12:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:04.122+0000 7fcc81535700 1 -- 172.21.15.113:0/895437466 wait complete. 2024-04-22T07:12:04.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:04 smithi113 ceph-mon[27933]: purged_snaps scrub starts 2024-04-22T07:12:04.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:04 smithi113 ceph-mon[27933]: purged_snaps scrub ok 2024-04-22T07:12:04.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:04 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:04.791 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/895437466' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T07:12:04.819 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1713769897,"num_in_osds":12,"osd_in_since":1713769903,"num_remapped_pgs":0} 2024-04-22T07:12:04.905 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:04 smithi154 ceph-mon[31862]: purged_snaps scrub starts 2024-04-22T07:12:04.906 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:04 smithi154 ceph-mon[31862]: purged_snaps scrub ok 2024-04-22T07:12:04.906 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:04 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:04.906 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/895437466' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T07:12:04.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:04 smithi133 ceph-mon[31198]: purged_snaps scrub starts 2024-04-22T07:12:04.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:04 smithi133 ceph-mon[31198]: purged_snaps scrub ok 2024-04-22T07:12:04.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:04 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:04.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/895437466' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T07:12:05.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T07:12:05.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:05.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] boot 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: osdmap e81: 12 total, 12 up, 12 in 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: Detected new or changed devices on smithi154 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi154 to 4272M 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:05.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:05.836 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:05 smithi113 ceph-mon[27933]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] boot 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: osdmap e81: 12 total, 12 up, 12 in 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: Detected new or changed devices on smithi154 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi154 to 4272M 2024-04-22T07:12:06.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:06.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:06.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:05 smithi133 ceph-mon[31198]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: osd.11 [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] boot 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: osdmap e81: 12 total, 12 up, 12 in 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: Detected new or changed devices on smithi154 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi154 to 4272M 2024-04-22T07:12:06.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:06.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:06.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:06.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:05 smithi154 ceph-mon[31862]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-22T07:12:07.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:07 smithi133 ceph-mon[31198]: osdmap e82: 12 total, 12 up, 12 in 2024-04-22T07:12:07.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:07 smithi154 ceph-mon[31862]: osdmap e82: 12 total, 12 up, 12 in 2024-04-22T07:12:07.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:07 smithi113 ceph-mon[27933]: osdmap e82: 12 total, 12 up, 12 in 2024-04-22T07:12:07.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 -- 172.21.15.113:0/2882379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0103560 msgr2=0x7f23c0109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2882379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0103560 0x7f23c0109e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b0a0 tx=0x7f23b800b3b0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 -- 172.21.15.113:0/2882379584 shutdown_connections 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2882379584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0103560 0x7f23c0109e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2882379584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c01019c0 0x7f23c0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2882379584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23c0102bc0 0x7f23c0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 -- 172.21.15.113:0/2882379584 >> 172.21.15.113:0/2882379584 conn(0x7f23c00fcfa0 msgr2=0x7f23c00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 -- 172.21.15.113:0/2882379584 shutdown_connections 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 -- 172.21.15.113:0/2882379584 wait complete. 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 Processor -- start 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.707+0000 7f23c82bd700 1 -- start start 2024-04-22T07:12:07.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c82bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c01019c0 0x7f23c019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c82bd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 0x7f23c019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c82bd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23c0103560 0x7f23c01a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c82bd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010e6b0 con 0x7f23c0102bc0 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c82bd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010e530 con 0x7f23c0103560 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c82bd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010e830 con 0x7f23c01019c0 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 0x7f23c019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 0x7f23c019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34820/0 (socket says 172.21.15.113:34820) 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 -- 172.21.15.113:0/2029473629 learned_addr learned my addr 172.21.15.113:0/2029473629 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c6059700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c01019c0 0x7f23c019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c685a700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23c0103560 0x7f23c01a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 -- 172.21.15.113:0/2029473629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c01019c0 msgr2=0x7f23c019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c01019c0 0x7f23c019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 -- 172.21.15.113:0/2029473629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23c0103560 msgr2=0x7f23c01a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23c0103560 0x7f23c01a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 -- 172.21.15.113:0/2029473629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c01a2a90 con 0x7f23c0102bc0 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23c5858700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 0x7f23c019b370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f23b000d980 tx=0x7f23b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.708+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0016070 con 0x7f23c0102bc0 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.709+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23b0009da0 con 0x7f23c0102bc0 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.709+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b000f3c0 con 0x7f23c0102bc0 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.709+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01a2d80 con 0x7f23c0102bc0 2024-04-22T07:12:07.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.709+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a3210 con 0x7f23c0102bc0 2024-04-22T07:12:07.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.710+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c01a3440 con 0x7f23c0102bc0 2024-04-22T07:12:07.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.711+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f23b0019070 con 0x7f23c0102bc0 2024-04-22T07:12:07.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.712+0000 7f23b77fe700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f23ac0a3c40 0x7f23ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:07.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.712+0000 7f23c6059700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f23ac0a3c40 0x7f23ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:07.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.712+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f23b00cf960 con 0x7f23c0102bc0 2024-04-22T07:12:07.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.712+0000 7f23c6059700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f23ac0a3c40 0x7f23ac0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23c0102a20 tx=0x7f23bc00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:07.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.716+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23b009c7e0 con 0x7f23c0102bc0 2024-04-22T07:12:07.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.914+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f23c01a3740 con 0x7f23c0102bc0 2024-04-22T07:12:07.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.914+0000 7f23b77fe700 1 -- 172.21.15.113:0/2029473629 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f23b009c0e0 con 0x7f23c0102bc0 2024-04-22T07:12:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:07.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f23ac0a3c40 msgr2=0x7f23ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f23ac0a3c40 0x7f23ac0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23c0102a20 tx=0x7f23bc00a3b0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 msgr2=0x7f23c019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 0x7f23c019b370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f23b000d980 tx=0x7f23b000dc90 comp rx=0 tx=0).stop 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 shutdown_connections 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f23ac0a3c40 0x7f23ac0a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23c0103560 0x7f23c01a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23c01019c0 0x7f23c019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 --2- 172.21.15.113:0/2029473629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23c0102bc0 0x7f23c019b370 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 >> 172.21.15.113:0/2029473629 conn(0x7f23c00fcfa0 msgr2=0x7f23c010a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 shutdown_connections 2024-04-22T07:12:07.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:07.917+0000 7f23c82bd700 1 -- 172.21.15.113:0/2029473629 wait complete. 2024-04-22T07:12:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:08 smithi133 ceph-mon[31198]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:08.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:08 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2029473629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T07:12:08.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:08 smithi154 ceph-mon[31862]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:08.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:08 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2029473629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T07:12:08.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:08 smithi113 ceph-mon[27933]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:08.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:08 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2029473629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T07:12:08.564 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713769924,"num_in_osds":12,"osd_in_since":1713769903,"num_remapped_pgs":0} 2024-04-22T07:12:08.565 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T07:12:08.843 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:12:10.161 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:10 smithi113 ceph-mon[27933]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 -- 172.21.15.113:0/3868726732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5410f4e0 msgr2=0x7f6b541118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 --2- 172.21.15.113:0/3868726732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5410f4e0 0x7f6b541118d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00b0a0 tx=0x7f6b4c00b3b0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 -- 172.21.15.113:0/3868726732 shutdown_connections 2024-04-22T07:12:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 --2- 172.21.15.113:0/3868726732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5410f4e0 0x7f6b541118d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 --2- 172.21.15.113:0/3868726732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5406afe0 0x7f6b5406b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 --2- 172.21.15.113:0/3868726732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b5406b900 0x7f6b5410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 -- 172.21.15.113:0/3868726732 >> 172.21.15.113:0/3868726732 conn(0x7f6b540788b0 msgr2=0x7f6b54078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 -- 172.21.15.113:0/3868726732 shutdown_connections 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.397+0000 7f6b59336700 1 -- 172.21.15.113:0/3868726732 wait complete. 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 Processor -- start 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 -- start start 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b5406afe0 0x7f6b5419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 0x7f6b5419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5410f4e0 0x7f6b5419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54115050 con 0x7f6b5406b900 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54114ed0 con 0x7f6b5410f4e0 2024-04-22T07:12:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b59336700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6b541151d0 con 0x7f6b5406afe0 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b527fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 0x7f6b5419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b527fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 0x7f6b5419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34826/0 (socket says 172.21.15.113:34826) 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.398+0000 7f6b527fc700 1 -- 172.21.15.113:0/1828600031 learned_addr learned my addr 172.21.15.113:0/1828600031 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b537fe700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5410f4e0 0x7f6b5419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b52ffd700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b5406afe0 0x7f6b5419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b527fc700 1 -- 172.21.15.113:0/1828600031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b5406afe0 msgr2=0x7f6b5419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b527fc700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b5406afe0 0x7f6b5419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b527fc700 1 -- 172.21.15.113:0/1828600031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5410f4e0 msgr2=0x7f6b5419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b527fc700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5410f4e0 0x7f6b5419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b527fc700 1 -- 172.21.15.113:0/1828600031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b541a0530 con 0x7f6b5406b900 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b527fc700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 0x7f6b5419ba60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800b530 tx=0x7f6b4800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48015020 con 0x7f6b5406b900 2024-04-22T07:12:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6b48011d00 con 0x7f6b5406b900 2024-04-22T07:12:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b541a0820 con 0x7f6b5406b900 2024-04-22T07:12:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800f2e0 con 0x7f6b5406b900 2024-04-22T07:12:10.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.399+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b541a80a0 con 0x7f6b5406b900 2024-04-22T07:12:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.400+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b54069730 con 0x7f6b5406b900 2024-04-22T07:12:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.401+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6b48018020 con 0x7f6b5406b900 2024-04-22T07:12:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.402+0000 7f6b3bfff700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6b3c0a3c40 0x7f6b3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.402+0000 7f6b52ffd700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6b3c0a3c40 0x7f6b3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.402+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6b480cfcf0 con 0x7f6b5406b900 2024-04-22T07:12:10.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.402+0000 7f6b52ffd700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6b3c0a3c40 0x7f6b3c0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6b44006fd0 tx=0x7f6b44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:10.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.406+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b4801f020 con 0x7f6b5406b900 2024-04-22T07:12:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:10 smithi133 ceph-mon[31198]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:10.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:10 smithi154 ceph-mon[31862]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.603+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b5402df80 con 0x7f6b5406b900 2024-04-22T07:12:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.604+0000 7f6b3bfff700 1 -- 172.21.15.113:0/1828600031 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18504 (secure 0 0 0) 0x7f6b4809cb70 con 0x7f6b5406b900 2024-04-22T07:12:10.602 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:10.602 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:06.002736+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6b3c0a3c40 msgr2=0x7f6b3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6b3c0a3c40 0x7f6b3c0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6b44006fd0 tx=0x7f6b44008040 comp rx=0 tx=0).stop 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 msgr2=0x7f6b5419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 0x7f6b5419ba60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800b530 tx=0x7f6b4800b840 comp rx=0 tx=0).stop 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 shutdown_connections 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6b3c0a3c40 0x7f6b3c0a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5410f4e0 0x7f6b5419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b5406afe0 0x7f6b5419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 --2- 172.21.15.113:0/1828600031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b5406b900 0x7f6b5419ba60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 >> 172.21.15.113:0/1828600031 conn(0x7f6b540788b0 msgr2=0x7f6b54110950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 shutdown_connections 2024-04-22T07:12:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:10.606+0000 7f6b59336700 1 -- 172.21.15.113:0/1828600031 wait complete. 2024-04-22T07:12:11.191 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:11 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1828600031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:11.221 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:12:11.221 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-22T07:12:11.457 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:12:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:11 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1828600031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:11.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:11 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1828600031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:12.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:12 smithi113 ceph-mon[27933]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:12 smithi133 ceph-mon[31198]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:12.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:12 smithi154 ceph-mon[31862]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:13.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 -- 172.21.15.113:0/2799487131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4103450 msgr2=0x7fb5e410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:13.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 --2- 172.21.15.113:0/2799487131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4103450 0x7fb5e410f8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00b0a0 tx=0x7fb5dc00b3b0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 -- 172.21.15.113:0/2799487131 shutdown_connections 2024-04-22T07:12:13.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 --2- 172.21.15.113:0/2799487131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4103450 0x7fb5e410f8b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 --2- 172.21.15.113:0/2799487131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5e4108a50 0x7fb5e4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 --2- 172.21.15.113:0/2799487131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4102a90 0x7fb5e4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 -- 172.21.15.113:0/2799487131 >> 172.21.15.113:0/2799487131 conn(0x7fb5e4079ba0 msgr2=0x7fb5e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 -- 172.21.15.113:0/2799487131 shutdown_connections 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 -- 172.21.15.113:0/2799487131 wait complete. 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.007+0000 7fb5e949a700 1 Processor -- start 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 -- start start 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4102a90 0x7fb5e41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5e4103450 0x7fb5e4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 0x7fb5e419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4112d40 con 0x7fb5e4108a50 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4112bc0 con 0x7fb5e4103450 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e949a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4112ec0 con 0x7fb5e4102a90 2024-04-22T07:12:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 0x7fb5e419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 0x7fb5e419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34844/0 (socket says 172.21.15.113:34844) 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 -- 172.21.15.113:0/3351325686 learned_addr learned my addr 172.21.15.113:0/3351325686 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e27fc700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5e4103450 0x7fb5e4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e2ffd700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4102a90 0x7fb5e41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 -- 172.21.15.113:0/3351325686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4102a90 msgr2=0x7fb5e41990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4102a90 0x7fb5e41990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 -- 172.21.15.113:0/3351325686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5e4103450 msgr2=0x7fb5e4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5e4103450 0x7fb5e4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 -- 172.21.15.113:0/3351325686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e419e100 con 0x7fb5e4108a50 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.008+0000 7fb5e37fe700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 0x7fb5e419d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00b830 tx=0x7fb5dc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.009+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc021020 con 0x7fb5e4108a50 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.009+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e419e390 con 0x7fb5e4108a50 2024-04-22T07:12:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.009+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb5dc01db40 con 0x7fb5e4108a50 2024-04-22T07:12:13.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.009+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01bc80 con 0x7fb5e4108a50 2024-04-22T07:12:13.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.009+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e41a5c70 con 0x7fb5e4108a50 2024-04-22T07:12:13.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.010+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4069730 con 0x7fb5e4108a50 2024-04-22T07:12:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.011+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb5dc024020 con 0x7fb5e4108a50 2024-04-22T07:12:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.011+0000 7fb5cbfff700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb5cc0a3c40 0x7fb5cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.011+0000 7fb5e2ffd700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb5cc0a3c40 0x7fb5cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.011+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb5dc0d13f0 con 0x7fb5e4108a50 2024-04-22T07:12:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.012+0000 7fb5e2ffd700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb5cc0a3c40 0x7fb5cc0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4005fd0 tx=0x7fb5d400af20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.016+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5dc02b020 con 0x7fb5e4108a50 2024-04-22T07:12:13.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.218+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5e402df80 con 0x7fb5e4108a50 2024-04-22T07:12:13.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.218+0000 7fb5cbfff700 1 -- 172.21.15.113:0/3351325686 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb5dc01be20 con 0x7fb5e4108a50 2024-04-22T07:12:13.216 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.220+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb5cc0a3c40 msgr2=0x7fb5cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.220+0000 7fb5e949a700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb5cc0a3c40 0x7fb5cc0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4005fd0 tx=0x7fb5d400af20 comp rx=0 tx=0).stop 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.220+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 msgr2=0x7fb5e419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.220+0000 7fb5e949a700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 0x7fb5e419d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00b830 tx=0x7fb5dc007a90 comp rx=0 tx=0).stop 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 shutdown_connections 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb5cc0a3c40 0x7fb5cc0a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e4108a50 0x7fb5e419d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5e4102a90 0x7fb5e41990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 --2- 172.21.15.113:0/3351325686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5e4103450 0x7fb5e4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 >> 172.21.15.113:0/3351325686 conn(0x7fb5e4079ba0 msgr2=0x7fb5e41015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:13.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 shutdown_connections 2024-04-22T07:12:13.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:13.221+0000 7fb5e949a700 1 -- 172.21.15.113:0/3351325686 wait complete. 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.a on smithi113 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.d on smithi113 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.g on smithi113 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.j on smithi113 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.b on smithi133 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.e on smithi133 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.h on smithi133 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.k on smithi133 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.c on smithi154 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.f on smithi154 2024-04-22T07:12:13.850 INFO:tasks.cephadm:Adding mds.i on smithi154 2024-04-22T07:12:13.851 INFO:tasks.cephadm:Adding mds.l on smithi154 2024-04-22T07:12:13.851 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l' 2024-04-22T07:12:14.118 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:12:14.231 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:14 smithi154 ceph-mon[31862]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:14.231 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:14 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3351325686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:14.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:14 smithi133 ceph-mon[31198]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:14.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:14 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3351325686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:14 smithi113 ceph-mon[27933]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:14.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:14 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3351325686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2914250895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 msgr2=0x7fc948108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2914250895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948108240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009990 tx=0x7fc93c009ca0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2914250895 shutdown_connections 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2914250895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc948108780 0x7fc94810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2914250895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc948103520 0x7fc948105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2914250895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948108240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2914250895 >> 172.21.15.154:0/2914250895 conn(0x7fc9480fcf30 msgr2=0x7fc9480ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2914250895 shutdown_connections 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2914250895 wait complete. 2024-04-22T07:12:15.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.771+0000 7fc94e1f5700 1 Processor -- start 2024-04-22T07:12:15.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 -- start start 2024-04-22T07:12:15.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc948103520 0x7fc948198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:15.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:15.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc948108780 0x7fc9481a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:15.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc94810e670 con 0x7fc948108780 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc94810e4f0 con 0x7fc948103520 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc94e1f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc94810e7f0 con 0x7fc948105e50 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54924/0 (socket says 172.21.15.154:54924) 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 -- 172.21.15.154:0/2018649704 learned_addr learned my addr 172.21.15.154:0/2018649704 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc9477fe700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc948103520 0x7fc948198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc947fff700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc948108780 0x7fc9481a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 -- 172.21.15.154:0/2018649704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc948103520 msgr2=0x7fc948198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc948103520 0x7fc948198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 -- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc948108780 msgr2=0x7fc9481a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc948108780 0x7fc9481a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.772+0000 7fc946ffd700 1 -- 172.21.15.154:0/2018649704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9481a0860 con 0x7fc948105e50 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.773+0000 7fc946ffd700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948199150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006010 tx=0x7fc93c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:15.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.773+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c027070 con 0x7fc948105e50 2024-04-22T07:12:15.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.773+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9481a0af0 con 0x7fc948105e50 2024-04-22T07:12:15.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.773+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc93c004150 con 0x7fc948105e50 2024-04-22T07:12:15.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.773+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00f460 con 0x7fc948105e50 2024-04-22T07:12:15.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.773+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9481a0f40 con 0x7fc948105e50 2024-04-22T07:12:15.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.774+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc948069730 con 0x7fc948105e50 2024-04-22T07:12:15.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.776+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc93c02a070 con 0x7fc948105e50 2024-04-22T07:12:15.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.776+0000 7fc944ff9700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9300a3c40 0x7fc9300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:15.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.776+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc93c0d10a0 con 0x7fc948105e50 2024-04-22T07:12:15.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.776+0000 7fc9477fe700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9300a3c40 0x7fc9300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:15.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.777+0000 7fc9477fe700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9300a3c40 0x7fc9300a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc94819a0f0 tx=0x7fc93800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:15.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.780+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc93c013070 con 0x7fc948105e50 2024-04-22T07:12:15.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.986+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}) -- 0x7fc948062f80 con 0x7fc9300a3c40 2024-04-22T07:12:15.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.995+0000 7fc944ff9700 1 -- 172.21.15.154:0/2018649704 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc948062f80 con 0x7fc9300a3c40 2024-04-22T07:12:15.992 INFO:teuthology.orchestra.run.smithi154.stdout:Scheduled mds.all update... 2024-04-22T07:12:15.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9300a3c40 msgr2=0x7fc9300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9300a3c40 0x7fc9300a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc94819a0f0 tx=0x7fc93800a380 comp rx=0 tx=0).stop 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 msgr2=0x7fc948199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948199150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006010 tx=0x7fc93c0047b0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 shutdown_connections 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9300a3c40 0x7fc9300a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc948108780 0x7fc9481a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc948103520 0x7fc948198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 --2- 172.21.15.154:0/2018649704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948105e50 0x7fc948199150 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 >> 172.21.15.154:0/2018649704 conn(0x7fc9480fcf30 msgr2=0x7fc948103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 shutdown_connections 2024-04-22T07:12:15.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-04-22T07:12:15.997+0000 7fc94e1f5700 1 -- 172.21.15.154:0/2018649704 wait complete. 2024-04-22T07:12:16.561 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:16 smithi154 ceph-mon[31862]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:16.561 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:16.562 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:16 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:16.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:16 smithi133 ceph-mon[31198]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:16 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:16.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:16 smithi113 ceph-mon[27933]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:16.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:16.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:16 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:16.588 DEBUG:teuthology.orchestra.run.smithi113:mds.a> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.a.service 2024-04-22T07:12:16.591 DEBUG:teuthology.orchestra.run.smithi113:mds.d> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.d.service 2024-04-22T07:12:16.635 DEBUG:teuthology.orchestra.run.smithi113:mds.g> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.g.service 2024-04-22T07:12:16.677 DEBUG:teuthology.orchestra.run.smithi113:mds.j> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.j.service 2024-04-22T07:12:16.721 DEBUG:teuthology.orchestra.run.smithi133:mds.b> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service 2024-04-22T07:12:16.724 DEBUG:teuthology.orchestra.run.smithi133:mds.e> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.e.service 2024-04-22T07:12:16.768 DEBUG:teuthology.orchestra.run.smithi133:mds.h> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.h.service 2024-04-22T07:12:16.810 DEBUG:teuthology.orchestra.run.smithi133:mds.k> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.k.service 2024-04-22T07:12:16.854 DEBUG:teuthology.orchestra.run.smithi154:mds.c> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service 2024-04-22T07:12:16.856 DEBUG:teuthology.orchestra.run.smithi154:mds.f> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service 2024-04-22T07:12:16.900 DEBUG:teuthology.orchestra.run.smithi154:mds.i> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.i.service 2024-04-22T07:12:16.942 DEBUG:teuthology.orchestra.run.smithi154:mds.l> sudo journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.l.service 2024-04-22T07:12:16.985 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-22T07:12:16.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:12:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- 172.21.15.113:0/2349166582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101140 msgr2=0x7f4cf41015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- 172.21.15.113:0/2349166582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101140 0x7f4cf41015a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce802bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- 172.21.15.113:0/2349166582 shutdown_connections 2024-04-22T07:12:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- 172.21.15.113:0/2349166582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cf4101ae0 0x7f4cf4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- 172.21.15.113:0/2349166582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101140 0x7f4cf41015a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- 172.21.15.113:0/2349166582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf41007e0 0x7f4cf4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- 172.21.15.113:0/2349166582 >> 172.21.15.113:0/2349166582 conn(0x7f4cf40fc3e0 msgr2=0x7f4cf40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- 172.21.15.113:0/2349166582 shutdown_connections 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- 172.21.15.113:0/2349166582 wait complete. 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 Processor -- start 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- start start 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cf41007e0 0x7f4cf419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf4101140 0x7f4cf419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 0x7f4cf41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf410fde0 con 0x7f4cf4101ae0 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf410fc60 con 0x7f4cf4101140 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.106+0000 7f4cfba31640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf410ff60 con 0x7f4cf41007e0 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 0x7f4cf41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 0x7f4cf41a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47958/0 (socket says 172.21.15.113:47958) 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 -- 172.21.15.113:0/3801596684 learned_addr learned my addr 172.21.15.113:0/3801596684 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf97cd640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cf41007e0 0x7f4cf419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf8fcc640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf4101140 0x7f4cf419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 -- 172.21.15.113:0/3801596684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cf41007e0 msgr2=0x7f4cf419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cf41007e0 0x7f4cf419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 -- 172.21.15.113:0/3801596684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf4101140 msgr2=0x7f4cf419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf4101140 0x7f4cf419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 -- 172.21.15.113:0/3801596684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf41a2080 con 0x7f4cf4101ae0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf8fcc640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf4101140 0x7f4cf419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf9fce640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 0x7f4cf41a1a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00e970 tx=0x7f4cec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00ccc0 con 0x7f4cf4101ae0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4cec00ce60 con 0x7f4cf4101ae0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf41a2370 con 0x7f4cf4101ae0 2024-04-22T07:12:17.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec010430 con 0x7f4cf4101ae0 2024-04-22T07:12:17.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.107+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf41a2910 con 0x7f4cf4101ae0 2024-04-22T07:12:17.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.108+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf410b6f0 con 0x7f4cf4101ae0 2024-04-22T07:12:17.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.109+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4cec005020 con 0x7f4cf4101ae0 2024-04-22T07:12:17.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.110+0000 7f4cf27fc640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4cdc0a3c30 0x7f4cdc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.110+0000 7f4cf97cd640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4cdc0a3c30 0x7f4cdc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.110+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4cec0d02d0 con 0x7f4cf4101ae0 2024-04-22T07:12:17.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.110+0000 7f4cf97cd640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4cdc0a3c30 0x7f4cdc0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005fd0 tx=0x7f4ce4009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:17.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.115+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cec09d150 con 0x7f4cf4101ae0 2024-04-22T07:12:17.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.294+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cf4062b50 con 0x7f4cf4101ae0 2024-04-22T07:12:17.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.295+0000 7f4cf27fc640 1 -- 172.21.15.113:0/3801596684 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18504 (secure 0 0 0) 0x7f4cec09ca50 con 0x7f4cf4101ae0 2024-04-22T07:12:17.293 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:17.294 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:06.002736+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:12:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4cdc0a3c30 msgr2=0x7f4cdc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4cdc0a3c30 0x7f4cdc0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005fd0 tx=0x7f4ce4009760 comp rx=0 tx=0).stop 2024-04-22T07:12:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 msgr2=0x7f4cf41a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 0x7f4cf41a1a10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00e970 tx=0x7f4cec00ed70 comp rx=0 tx=0).stop 2024-04-22T07:12:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 shutdown_connections 2024-04-22T07:12:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4cdc0a3c30 0x7f4cdc0a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cf4101ae0 0x7f4cf41a1a10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cf4101140 0x7f4cf419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 --2- 172.21.15.113:0/3801596684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cf41007e0 0x7f4cf419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 >> 172.21.15.113:0/3801596684 conn(0x7f4cf40fc3e0 msgr2=0x7f4cf40fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 shutdown_connections 2024-04-22T07:12:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.297+0000 7f4cfba31640 1 -- 172.21.15.113:0/3801596684 wait complete. 2024-04-22T07:12:17.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:12:17.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: Saving service mds.all spec with placement smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-22T07:12:17.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:17.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:17.305 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:17.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:17.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:17.306 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:17 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:17.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:12:17.307 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: Saving service mds.all spec with placement smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:17.398 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:17 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 -- 172.21.15.113:0/2756829003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 msgr2=0x7ff77c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 --2- 172.21.15.113:0/2756829003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1023f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff770009900 tx=0x7ff77002bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 -- 172.21.15.113:0/2756829003 shutdown_connections 2024-04-22T07:12:17.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 --2- 172.21.15.113:0/2756829003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 0x7ff77c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 --2- 172.21.15.113:0/2756829003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1023f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 --2- 172.21.15.113:0/2756829003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff77c100d80 0x7ff77c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 -- 172.21.15.113:0/2756829003 >> 172.21.15.113:0/2756829003 conn(0x7ff77c0fc330 msgr2=0x7ff77c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 -- 172.21.15.113:0/2756829003 shutdown_connections 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.410+0000 7ff77b59e640 1 -- 172.21.15.113:0/2756829003 wait complete. 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 Processor -- start 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 -- start start 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff77c100d80 0x7ff77c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 0x7ff77c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10daf0 con 0x7ff77c101f70 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10d970 con 0x7ff77c102930 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10dc70 con 0x7ff77c100d80 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff77c100d80 0x7ff77c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47982/0 (socket says 172.21.15.113:47982) 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 -- 172.21.15.113:0/1630348043 learned_addr learned my addr 172.21.15.113:0/1630348043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77ad9d640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 0x7ff77c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 -- 172.21.15.113:0/1630348043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff77c100d80 msgr2=0x7ff77c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff77c100d80 0x7ff77c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 -- 172.21.15.113:0/1630348043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 msgr2=0x7ff77c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 0x7ff77c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 -- 172.21.15.113:0/1630348043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c19fdb0 con 0x7ff77c101f70 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77ad9d640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 0x7ff77c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:12:17.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff779d9b640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1986d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff770005ab0 tx=0x7ff770005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770041070 con 0x7ff77c101f70 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff7700052f0 con 0x7ff77c101f70 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c1a0040 con 0x7ff77c101f70 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770003d20 con 0x7ff77c101f70 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.411+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1a05e0 con 0x7ff77c101f70 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.412+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c1080a0 con 0x7ff77c101f70 2024-04-22T07:12:17.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.413+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff770044070 con 0x7ff77c101f70 2024-04-22T07:12:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.414+0000 7ff76b7fe640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7640a3c30 0x7ff7640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.414+0000 7ff77a59c640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7640a3c30 0x7ff7640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.414+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff7700f1990 con 0x7ff77c101f70 2024-04-22T07:12:17.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.414+0000 7ff77a59c640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7640a3c30 0x7ff7640a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff77c101dd0 tx=0x7ff76c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:17.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.418+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7700be1c0 con 0x7ff77c101f70 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: Saving service mds.all spec with placement smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:17 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:17.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.596+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff77c02d7e0 con 0x7ff77c101f70 2024-04-22T07:12:17.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.596+0000 7ff76b7fe640 1 -- 172.21.15.113:0/1630348043 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7ff7700c24c0 con 0x7ff77c101f70 2024-04-22T07:12:17.594 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:12:17.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7640a3c30 msgr2=0x7ff7640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7640a3c30 0x7ff7640a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff77c101dd0 tx=0x7ff76c005f90 comp rx=0 tx=0).stop 2024-04-22T07:12:17.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 msgr2=0x7ff77c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1986d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff770005ab0 tx=0x7ff770005dd0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 shutdown_connections 2024-04-22T07:12:17.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff7640a3c30 0x7ff7640a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff77c102930 0x7ff77c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77c101f70 0x7ff77c1986d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 --2- 172.21.15.113:0/1630348043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff77c100d80 0x7ff77c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 >> 172.21.15.113:0/1630348043 conn(0x7ff77c0fc330 msgr2=0x7ff77c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 shutdown_connections 2024-04-22T07:12:17.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.598+0000 7ff77b59e640 1 -- 172.21.15.113:0/1630348043 wait complete. 2024-04-22T07:12:17.607 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2441128319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 msgr2=0x7fa1ec1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2441128319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec1092b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009ca0 tx=0x7fa1e402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2441128319 shutdown_connections 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2441128319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec1092b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2441128319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1ec101fd0 0x7fa1ec102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2441128319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec100e80 0x7fa1ec1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2441128319 >> 172.21.15.113:0/2441128319 conn(0x7fa1ec0fc410 msgr2=0x7fa1ec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2441128319 shutdown_connections 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.708+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2441128319 wait complete. 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 Processor -- start 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 -- start start 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec100e80 0x7fa1ec198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1ec101fd0 0x7fa1ec198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10cb00 con 0x7fa1ec102990 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10c980 con 0x7fa1ec100e80 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1f0e9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10cc80 con 0x7fa1ec101fd0 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47992/0 (socket says 172.21.15.113:47992) 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 -- 172.21.15.113:0/2045633388 learned_addr learned my addr 172.21.15.113:0/2045633388 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1e9d9b640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1ec101fd0 0x7fa1ec198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ea59c640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec100e80 0x7fa1ec198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 -- 172.21.15.113:0/2045633388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1ec101fd0 msgr2=0x7fa1ec198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1ec101fd0 0x7fa1ec198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 -- 172.21.15.113:0/2045633388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec100e80 msgr2=0x7fa1ec198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec100e80 0x7fa1ec198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.709+0000 7fa1ead9d640 1 -- 172.21.15.113:0/2045633388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec19fd50 con 0x7fa1ec102990 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.710+0000 7fa1ead9d640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec19f6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4005cc0 tx=0x7fa1e4032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.710+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4041020 con 0x7fa1ec102990 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.710+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1e4003e40 con 0x7fa1ec102990 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.710+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec19ffe0 con 0x7fa1ec102990 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.710+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4039040 con 0x7fa1ec102990 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.710+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1a04a0 con 0x7fa1ec102990 2024-04-22T07:12:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.711+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec1a0790 con 0x7fa1ec102990 2024-04-22T07:12:17.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.712+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa1e4044020 con 0x7fa1ec102990 2024-04-22T07:12:17.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.712+0000 7fa1db7fe640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:17.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.712+0000 7fa1ea59c640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:17.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.712+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1e40f2370 con 0x7fa1ec102990 2024-04-22T07:12:17.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.713+0000 7fa1ea59c640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005d00 tx=0x7fa1dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:17.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.717+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1e404b020 con 0x7fa1ec102990 2024-04-22T07:12:17.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.931+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1ec1a0a80 con 0x7fa1ec102990 2024-04-22T07:12:17.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.931+0000 7fa1db7fe640 1 -- 172.21.15.113:0/2045633388 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa1e40bf1f0 con 0x7fa1ec102990 2024-04-22T07:12:17.929 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:17.929 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-22T07:12:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 1 2024-04-22T07:12:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1d40a3c30 msgr2=0x7fa1d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005d00 tx=0x7fa1dc00b410 comp rx=0 tx=0).stop 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 msgr2=0x7fa1ec19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec19f6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4005cc0 tx=0x7fa1e4032950 comp rx=0 tx=0).stop 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 shutdown_connections 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1ec102990 0x7fa1ec19f6e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1ec101fd0 0x7fa1ec198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 --2- 172.21.15.113:0/2045633388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec100e80 0x7fa1ec198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 >> 172.21.15.113:0/2045633388 conn(0x7fa1ec0fc410 msgr2=0x7fa1ec0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.933+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 shutdown_connections 2024-04-22T07:12:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:17.934+0000 7fa1f0e9a640 1 -- 172.21.15.113:0/2045633388 wait complete. 2024-04-22T07:12:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:18 smithi113 ceph-mon[27933]: Deploying daemon mds.l on smithi154 2024-04-22T07:12:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:18 smithi113 ceph-mon[27933]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3801596684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1630348043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:18.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2045633388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:18.388 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:18 smithi154 ceph-mon[31862]: Deploying daemon mds.l on smithi154 2024-04-22T07:12:18.388 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:18 smithi154 ceph-mon[31862]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:18.388 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3801596684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:18.388 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1630348043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:18.388 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2045633388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:18 smithi133 ceph-mon[31198]: Deploying daemon mds.l on smithi154 2024-04-22T07:12:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:18 smithi133 ceph-mon[31198]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3801596684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1630348043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2045633388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:18.816 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:18 smithi154 systemd[1]: Starting Ceph mds.l for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:19.123 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:18 smithi154 podman[62635]: 2024-04-22 07:12:18.823063486 +0000 UTC m=+0.109121713 container create da1eb41d991c6ba8d97cc95b6369388cd883ed55aca5980727f980e1c5ee24d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:12:19.123 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:18 smithi154 podman[62635]: 2024-04-22 07:12:18.732575107 +0000 UTC m=+0.018633301 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:19.123 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:19 smithi154 podman[62635]: 2024-04-22 07:12:19.034000243 +0000 UTC m=+0.320058424 container init da1eb41d991c6ba8d97cc95b6369388cd883ed55aca5980727f980e1c5ee24d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:12:19.123 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:19 smithi154 podman[62635]: 2024-04-22 07:12:19.036622834 +0000 UTC m=+0.322681015 container start da1eb41d991c6ba8d97cc95b6369388cd883ed55aca5980727f980e1c5ee24d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-22T07:12:19.124 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:19 smithi154 ceph-mds[62654]: -- 172.21.15.154:0/1064435139 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f48a3f7a40 con 0x55f48a3cac00 2024-04-22T07:12:19.124 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l[62650]: starting mds.l at 2024-04-22T07:12:19.124 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:19 smithi154 bash[62635]: da1eb41d991c6ba8d97cc95b6369388cd883ed55aca5980727f980e1c5ee24d2 2024-04-22T07:12:19.566 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:12:19 smithi154 systemd[1]: Started Ceph mds.l for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:19.944 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:20.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 -- 172.21.15.113:0/3378392913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 msgr2=0x7f723010a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:20.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 --2- 172.21.15.113:0/3378392913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723010a0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7220009900 tx=0x7f722002bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 -- 172.21.15.113:0/3378392913 shutdown_connections 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 --2- 172.21.15.113:0/3378392913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723010a0b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 --2- 172.21.15.113:0/3378392913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7230105330 0x7f7230107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 --2- 172.21.15.113:0/3378392913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7230102a00 0x7f7230104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 -- 172.21.15.113:0/3378392913 >> 172.21.15.113:0/3378392913 conn(0x7f72300fc410 msgr2=0x7f72300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 -- 172.21.15.113:0/3378392913 shutdown_connections 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.056+0000 7f72348c0640 1 -- 172.21.15.113:0/3378392913 wait complete. 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 Processor -- start 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 -- start start 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7230102a00 0x7f72301981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7230105330 0x7f72301986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f723010dc40 con 0x7f7230107c60 2024-04-22T07:12:20.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f723010dac0 con 0x7f7230102a00 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f72348c0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f723010ddc0 con 0x7f7230105330 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48020/0 (socket says 172.21.15.113:48020) 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 -- 172.21.15.113:0/2565603066 learned_addr learned my addr 172.21.15.113:0/2565603066 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722dd9b640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7230105330 0x7f72301986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722e59c640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7230102a00 0x7f72301981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 -- 172.21.15.113:0/2565603066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7230105330 msgr2=0x7f72301986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7230105330 0x7f72301986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 -- 172.21.15.113:0/2565603066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7230102a00 msgr2=0x7f72301981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7230102a00 0x7f72301981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.057+0000 7f722ed9d640 1 -- 172.21.15.113:0/2565603066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723019fdc0 con 0x7f7230107c60 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.058+0000 7f722ed9d640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723019f750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f722002c710 tx=0x7f7220004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:20.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.058+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220043020 con 0x7f7230107c60 2024-04-22T07:12:20.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.058+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f72200052f0 con 0x7f7230107c60 2024-04-22T07:12:20.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.058+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72301a0050 con 0x7f7230107c60 2024-04-22T07:12:20.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.058+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220003dc0 con 0x7f7230107c60 2024-04-22T07:12:20.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.058+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72301a05a0 con 0x7f7230107c60 2024-04-22T07:12:20.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.059+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72300fdf00 con 0x7f7230107c60 2024-04-22T07:12:20.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.060+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7220046020 con 0x7f7230107c60 2024-04-22T07:12:20.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.061+0000 7f72277fe640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f721c0a3bb0 0x7f721c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:20.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.061+0000 7f722e59c640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f721c0a3bb0 0x7f721c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:20.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.061+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f72200f2590 con 0x7f7230107c60 2024-04-22T07:12:20.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.061+0000 7f722e59c640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f721c0a3bb0 0x7f721c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7228005c40 tx=0x7f722800a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:20.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.065+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f722004d020 con 0x7f7230107c60 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6832/819873552,v1:172.21.15.154:6833/819873552] up:boot 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: fsmap 1 up:standby 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:20.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:20 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:20.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.282+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7230062b50 con 0x7f7230107c60 2024-04-22T07:12:20.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.282+0000 7f72277fe640 1 -- 172.21.15.113:0/2565603066 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f72200bf410 con 0x7f7230107c60 2024-04-22T07:12:20.280 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:20.281 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T07:12:20.281 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 2 2024-04-22T07:12:20.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f721c0a3bb0 msgr2=0x7f721c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:20.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f721c0a3bb0 0x7f721c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7228005c40 tx=0x7f722800a780 comp rx=0 tx=0).stop 2024-04-22T07:12:20.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 msgr2=0x7f723019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:20.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723019f750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f722002c710 tx=0x7f7220004270 comp rx=0 tx=0).stop 2024-04-22T07:12:20.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 shutdown_connections 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f721c0a3bb0 0x7f721c0a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7230107c60 0x7f723019f750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7230105330 0x7f72301986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 --2- 172.21.15.113:0/2565603066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7230102a00 0x7f72301981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 >> 172.21.15.113:0/2565603066 conn(0x7f72300fc410 msgr2=0x7f7230103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 shutdown_connections 2024-04-22T07:12:20.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:20.284+0000 7f72348c0640 1 -- 172.21.15.113:0/2565603066 wait complete. 2024-04-22T07:12:20.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:20.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6832/819873552,v1:172.21.15.154:6833/819873552] up:boot 2024-04-22T07:12:20.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: fsmap 1 up:standby 2024-04-22T07:12:20.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T07:12:20.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:20.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:20.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:20 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6832/819873552,v1:172.21.15.154:6833/819873552] up:boot 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: fsmap 1 up:standby 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:20.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:20 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:21.203 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:21 smithi133 ceph-mon[31198]: Deploying daemon mds.k on smithi133 2024-04-22T07:12:21.203 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:21 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2565603066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:21.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:21 smithi113 ceph-mon[27933]: Deploying daemon mds.k on smithi133 2024-04-22T07:12:21.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:21 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2565603066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:21.564 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 systemd[1]: Starting Ceph mds.k for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:21.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:21 smithi154 ceph-mon[31862]: Deploying daemon mds.k on smithi133 2024-04-22T07:12:21.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:21 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2565603066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:21.866 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 podman[60932]: 2024-04-22 07:12:21.497380248 +0000 UTC m=+0.018535159 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:21.866 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 podman[60932]: 2024-04-22 07:12:21.616897607 +0000 UTC m=+0.138052501 container create 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-22T07:12:21.866 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 podman[60932]: 2024-04-22 07:12:21.869363444 +0000 UTC m=+0.390518337 container init 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:12:22.294 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 podman[60932]: 2024-04-22 07:12:21.872221775 +0000 UTC m=+0.393376669 container start 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 ceph-mds[60952]: -- 172.21.15.133:0/1305106812 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbb8a5fa40 con 0x55dbb8a32800 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k[60948]: starting mds.k at 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 bash[60932]: 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:12:21 smithi133 systemd[1]: Started Ceph mds.k for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:22.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:22.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:22.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:22 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:22.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:22 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 -- 172.21.15.113:0/1008747530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 msgr2=0x7f329c1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 --2- 172.21.15.113:0/1008747530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c1092a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3294009ca0 tx=0x7f329402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 -- 172.21.15.113:0/1008747530 shutdown_connections 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 --2- 172.21.15.113:0/1008747530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c1092a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 --2- 172.21.15.113:0/1008747530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f329c101fc0 0x7f329c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 --2- 172.21.15.113:0/1008747530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f329c100e70 0x7f329c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 -- 172.21.15.113:0/1008747530 >> 172.21.15.113:0/1008747530 conn(0x7f329c0fc3e0 msgr2=0x7f329c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 -- 172.21.15.113:0/1008747530 shutdown_connections 2024-04-22T07:12:22.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.397+0000 7f32a2fce640 1 -- 172.21.15.113:0/1008747530 wait complete. 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 Processor -- start 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 -- start start 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f329c100e70 0x7f329c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f329c101fc0 0x7f329c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f329c10cb10 con 0x7f329c102980 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f329c10c990 con 0x7f329c101fc0 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f329c10cc90 con 0x7f329c100e70 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48044/0 (socket says 172.21.15.113:48044) 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 -- 172.21.15.113:0/3952700145 learned_addr learned my addr 172.21.15.113:0/3952700145 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a0d6a640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f329c100e70 0x7f329c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f329bfff640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f329c101fc0 0x7f329c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:22.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 -- 172.21.15.113:0/3952700145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f329c100e70 msgr2=0x7f329c069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f329c100e70 0x7f329c069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 -- 172.21.15.113:0/3952700145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f329c101fc0 msgr2=0x7f329c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f329c101fc0 0x7f329c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 -- 172.21.15.113:0/3952700145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c19fe10 con 0x7f329c102980 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a156b640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c19f500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3294005cc0 tx=0x7f3294032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294041020 con 0x7f329c102980 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3294003e40 con 0x7f329c102980 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.398+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c1a0040 con 0x7f329c102980 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.399+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294039040 con 0x7f329c102980 2024-04-22T07:12:22.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.399+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a0500 con 0x7f329c102980 2024-04-22T07:12:22.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.400+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c108120 con 0x7f329c102980 2024-04-22T07:12:22.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.401+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3294044020 con 0x7f329c102980 2024-04-22T07:12:22.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.401+0000 7f3299ffb640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f328c0a3bb0 0x7f328c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:22.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.402+0000 7f32a0d6a640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f328c0a3bb0 0x7f328c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:22.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.402+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f32940f27c0 con 0x7f329c102980 2024-04-22T07:12:22.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.402+0000 7f32a0d6a640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f328c0a3bb0 0x7f328c0a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3290005fd0 tx=0x7f329000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:22.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.409+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f329404b020 con 0x7f329c102980 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:22.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:22 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:22.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.645+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f329c02d780 con 0x7f329c102980 2024-04-22T07:12:22.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.646+0000 7f3299ffb640 1 -- 172.21.15.113:0/3952700145 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f32940bf640 con 0x7f329c102980 2024-04-22T07:12:22.643 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:22.644 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T07:12:22.644 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 3 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f328c0a3bb0 msgr2=0x7f328c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f328c0a3bb0 0x7f328c0a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3290005fd0 tx=0x7f329000b390 comp rx=0 tx=0).stop 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 msgr2=0x7f329c19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c19f500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3294005cc0 tx=0x7f3294032950 comp rx=0 tx=0).stop 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 shutdown_connections 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f328c0a3bb0 0x7f328c0a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f329c102980 0x7f329c19f500 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f329c101fc0 0x7f329c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 --2- 172.21.15.113:0/3952700145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f329c100e70 0x7f329c069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 >> 172.21.15.113:0/3952700145 conn(0x7f329c0fc3e0 msgr2=0x7f329c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 shutdown_connections 2024-04-22T07:12:22.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:22.648+0000 7f32a2fce640 1 -- 172.21.15.113:0/3952700145 wait complete. 2024-04-22T07:12:23.286 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:23 smithi133 ceph-mon[31198]: Deploying daemon mds.j on smithi113 2024-04-22T07:12:23.286 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:23 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6832/4012646459,v1:172.21.15.133:6833/4012646459] up:boot 2024-04-22T07:12:23.286 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:23 smithi133 ceph-mon[31198]: fsmap 2 up:standby 2024-04-22T07:12:23.287 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:23 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T07:12:23.287 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:23 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3952700145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:23.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:23 smithi113 ceph-mon[27933]: Deploying daemon mds.j on smithi113 2024-04-22T07:12:23.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:23 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6832/4012646459,v1:172.21.15.133:6833/4012646459] up:boot 2024-04-22T07:12:23.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:23 smithi113 ceph-mon[27933]: fsmap 2 up:standby 2024-04-22T07:12:23.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:23 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T07:12:23.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:23 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3952700145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:23 smithi154 ceph-mon[31862]: Deploying daemon mds.j on smithi113 2024-04-22T07:12:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:23 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6832/4012646459,v1:172.21.15.133:6833/4012646459] up:boot 2024-04-22T07:12:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:23 smithi154 ceph-mon[31862]: fsmap 2 up:standby 2024-04-22T07:12:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:23 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T07:12:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:23 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3952700145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:24.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:24 smithi113 ceph-mon[27933]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:24.335 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 systemd[1]: Starting Ceph mds.j for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:24.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:24 smithi133 ceph-mon[31198]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:24.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:24 smithi154 ceph-mon[31862]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:24.657 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 -- 172.21.15.113:0/1599220209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38100e80 msgr2=0x7f4c381012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/1599220209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38100e80 0x7f4c381012a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009900 tx=0x7f4c2802bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 -- 172.21.15.113:0/1599220209 shutdown_connections 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/1599220209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c38102990 0x7f4c381092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/1599220209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 0x7f4c38102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/1599220209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38100e80 0x7f4c381012a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.761+0000 7f4c3fac1640 1 -- 172.21.15.113:0/1599220209 >> 172.21.15.113:0/1599220209 conn(0x7f4c380fc410 msgr2=0x7f4c380fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 -- 172.21.15.113:0/1599220209 shutdown_connections 2024-04-22T07:12:24.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 -- 172.21.15.113:0/1599220209 wait complete. 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 Processor -- start 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 -- start start 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c38100e80 0x7f4c38198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 0x7f4c38198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 0x7f4c3819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3810cc10 con 0x7f4c38102990 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3810ca90 con 0x7f4c38100e80 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3fac1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3810cd90 con 0x7f4c38101fd0 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3e05e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 0x7f4c3819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3e05e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 0x7f4c3819f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48068/0 (socket says 172.21.15.113:48068) 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3e05e640 1 -- 172.21.15.113:0/3803285320 learned_addr learned my addr 172.21.15.113:0/3803285320 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.762+0000 7f4c3d05c640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 0x7f4c38198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3d85d640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c38100e80 0x7f4c38198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3e05e640 1 -- 172.21.15.113:0/3803285320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 msgr2=0x7f4c38198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3e05e640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 0x7f4c38198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3e05e640 1 -- 172.21.15.113:0/3803285320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c38100e80 msgr2=0x7f4c38198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3e05e640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c38100e80 0x7f4c38198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3e05e640 1 -- 172.21.15.113:0/3803285320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381a0020 con 0x7f4c38102990 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3d05c640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 0x7f4c38198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3e05e640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 0x7f4c3819f9b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000b970 tx=0x7f4c3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30019070 con 0x7f4c38102990 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381a0310 con 0x7f4c38102990 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4c3000cb40 con 0x7f4c38102990 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30007650 con 0x7f4c38102990 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.763+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381a08e0 con 0x7f4c38102990 2024-04-22T07:12:24.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.764+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c381a0ba0 con 0x7f4c38102990 2024-04-22T07:12:24.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.765+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4c3001c070 con 0x7f4c38102990 2024-04-22T07:12:24.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.765+0000 7f4c2effd640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4c240a3bb0 0x7f4c240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:24.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.766+0000 7f4c3d85d640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4c240a3bb0 0x7f4c240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:24.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.766+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4c300cf590 con 0x7f4c38102990 2024-04-22T07:12:24.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.766+0000 7f4c3d85d640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4c240a3bb0 0x7f4c240a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c28005c40 tx=0x7f4c28005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:24.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:24.770+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c3009c410 con 0x7f4c38102990 2024-04-22T07:12:24.776 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 podman[65878]: 2024-04-22 07:12:24.391198737 +0000 UTC m=+0.017812522 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:24.777 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 podman[65878]: 2024-04-22 07:12:24.517734511 +0000 UTC m=+0.144348314 container create 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-22T07:12:24.777 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 podman[65878]: 2024-04-22 07:12:24.779355012 +0000 UTC m=+0.405968790 container init 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-22T07:12:25.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.009+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c381a0d70 con 0x7f4c38102990 2024-04-22T07:12:25.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.010+0000 7f4c2effd640 1 -- 172.21.15.113:0/3803285320 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f4c3009bd10 con 0x7f4c38102990 2024-04-22T07:12:25.007 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:25.008 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T07:12:25.008 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 3 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4c240a3bb0 msgr2=0x7f4c240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4c240a3bb0 0x7f4c240a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c28005c40 tx=0x7f4c28005fd0 comp rx=0 tx=0).stop 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 msgr2=0x7f4c3819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 0x7f4c3819f9b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000b970 tx=0x7f4c3000be30 comp rx=0 tx=0).stop 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 shutdown_connections 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4c240a3bb0 0x7f4c240a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c38102990 0x7f4c3819f9b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c38101fd0 0x7f4c38198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 --2- 172.21.15.113:0/3803285320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c38100e80 0x7f4c38198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:25.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 >> 172.21.15.113:0/3803285320 conn(0x7f4c380fc410 msgr2=0x7f4c380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:25.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 shutdown_connections 2024-04-22T07:12:25.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:25.013+0000 7f4c3fac1640 1 -- 172.21.15.113:0/3803285320 wait complete. 2024-04-22T07:12:25.084 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 podman[65878]: 2024-04-22 07:12:24.783827857 +0000 UTC m=+0.410441621 container start 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-22T07:12:25.084 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 ceph-mds[65937]: -- 172.21.15.113:0/1922457713 <== mon.0 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558668d61a40 con 0x558668d34800 2024-04-22T07:12:25.084 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j[65894]: starting mds.j at 2024-04-22T07:12:25.084 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 bash[65878]: 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 2024-04-22T07:12:25.084 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:12:24 smithi113 systemd[1]: Started Ceph mds.j for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:26.216 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: Deploying daemon mds.d on smithi113 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3803285320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.113:6834/127167986,v1:172.21.15.113:6835/127167986] up:boot 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: fsmap 3 up:standby 2024-04-22T07:12:26.217 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:25 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T07:12:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:26.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: Deploying daemon mds.d on smithi113 2024-04-22T07:12:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3803285320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.113:6834/127167986,v1:172.21.15.113:6835/127167986] up:boot 2024-04-22T07:12:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: fsmap 3 up:standby 2024-04-22T07:12:26.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:25 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: Deploying daemon mds.d on smithi113 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3803285320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:26.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.113:6834/127167986,v1:172.21.15.113:6835/127167986] up:boot 2024-04-22T07:12:26.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: fsmap 3 up:standby 2024-04-22T07:12:26.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:25 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T07:12:27.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:27.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.172+0000 7eff58bf4640 1 -- 172.21.15.113:0/310946165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103170 msgr2=0x7eff541035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.172+0000 7eff58bf4640 1 --2- 172.21.15.113:0/310946165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103170 0x7eff541035f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff48009900 tx=0x7eff4802bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 -- 172.21.15.113:0/310946165 shutdown_connections 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 --2- 172.21.15.113:0/310946165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54103b30 0x7eff5410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 --2- 172.21.15.113:0/310946165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103170 0x7eff541035f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 --2- 172.21.15.113:0/310946165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54101f80 0x7eff541023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 -- 172.21.15.113:0/310946165 >> 172.21.15.113:0/310946165 conn(0x7eff540fd530 msgr2=0x7eff540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 -- 172.21.15.113:0/310946165 shutdown_connections 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 -- 172.21.15.113:0/310946165 wait complete. 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 Processor -- start 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.173+0000 7eff58bf4640 1 -- start start 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54101f80 0x7eff541980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:27.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54103170 0x7eff541985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 0x7eff5419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410dc30 con 0x7eff54103b30 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410dab0 con 0x7eff54103170 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff5410ddb0 con 0x7eff54101f80 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 0x7eff5419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 0x7eff5419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33098/0 (socket says 172.21.15.113:33098) 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 -- 172.21.15.113:0/2984330251 learned_addr learned my addr 172.21.15.113:0/2984330251 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff5259c640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54101f80 0x7eff541980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff51d9b640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54103170 0x7eff541985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 -- 172.21.15.113:0/2984330251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54101f80 msgr2=0x7eff541980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54101f80 0x7eff541980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 -- 172.21.15.113:0/2984330251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54103170 msgr2=0x7eff541985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54103170 0x7eff541985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 -- 172.21.15.113:0/2984330251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5419fcc0 con 0x7eff54103b30 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff51d9b640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54103170 0x7eff541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff52d9d640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 0x7eff5419f650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00cad0 tx=0x7eff4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00ae20 con 0x7eff54103b30 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.175+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff4c005020 con 0x7eff54103b30 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.174+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5419ffb0 con 0x7eff54103b30 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.175+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00f6b0 con 0x7eff54103b30 2024-04-22T07:12:27.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.175+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff541a0470 con 0x7eff54103b30 2024-04-22T07:12:27.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.176+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff541091d0 con 0x7eff54103b30 2024-04-22T07:12:27.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.177+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7eff4c0051d0 con 0x7eff54103b30 2024-04-22T07:12:27.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.177+0000 7eff437fe640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff3c0a3bb0 0x7eff3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:27.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.178+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7eff4c0d0310 con 0x7eff54103b30 2024-04-22T07:12:27.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.178+0000 7eff5259c640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff3c0a3bb0 0x7eff3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:27.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.178+0000 7eff5259c640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff3c0a3bb0 0x7eff3c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff54102fd0 tx=0x7eff44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:27.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.182+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff4c014030 con 0x7eff54103b30 2024-04-22T07:12:27.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.421+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff5402d810 con 0x7eff54103b30 2024-04-22T07:12:27.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.421+0000 7eff437fe640 1 -- 172.21.15.113:0/2984330251 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7eff4c09d190 con 0x7eff54103b30 2024-04-22T07:12:27.419 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:27.419 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T07:12:27.420 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 4 2024-04-22T07:12:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.423+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff3c0a3bb0 msgr2=0x7eff3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.423+0000 7eff58bf4640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff3c0a3bb0 0x7eff3c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff54102fd0 tx=0x7eff44005f90 comp rx=0 tx=0).stop 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 msgr2=0x7eff5419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 0x7eff5419f650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00cad0 tx=0x7eff4c00cf90 comp rx=0 tx=0).stop 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 shutdown_connections 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff3c0a3bb0 0x7eff3c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff54103b30 0x7eff5419f650 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff54103170 0x7eff541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 --2- 172.21.15.113:0/2984330251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff54101f80 0x7eff541980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 >> 172.21.15.113:0/2984330251 conn(0x7eff540fd530 msgr2=0x7eff541063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 shutdown_connections 2024-04-22T07:12:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:27.424+0000 7eff58bf4640 1 -- 172.21.15.113:0/2984330251 wait complete. 2024-04-22T07:12:27.581 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:12:27 smithi113 podman[66266]: 2024-04-22 07:12:27.561296019 +0000 UTC m=+0.341596370 container start 7ca41c63c4d60f22b02068e77ba86484220edc5860284ac4ae9461b1f22dff30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-22T07:12:27.834 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:12:27 smithi113 ceph-mds[66294]: -- 172.21.15.113:0/440598714 <== mon.0 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fd2afda40 con 0x563fd2ad0c00 2024-04-22T07:12:27.834 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:12:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d[66290]: starting mds.d at 2024-04-22T07:12:27.834 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:12:27 smithi113 bash[66266]: 7ca41c63c4d60f22b02068e77ba86484220edc5860284ac4ae9461b1f22dff30 2024-04-22T07:12:27.834 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:12:27 smithi113 systemd[1]: Started Ceph mds.d for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2984330251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:28.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:28.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:28 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2984330251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:28 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2984330251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:28 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:29.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:29 smithi133 ceph-mon[31198]: Deploying daemon mds.e on smithi133 2024-04-22T07:12:29.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:29 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.113:6836/918732402,v1:172.21.15.113:6837/918732402] up:boot 2024-04-22T07:12:29.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:29 smithi133 ceph-mon[31198]: fsmap 4 up:standby 2024-04-22T07:12:29.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:29 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T07:12:29.433 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:29.459 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:29 smithi113 ceph-mon[27933]: Deploying daemon mds.e on smithi133 2024-04-22T07:12:29.459 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:29 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.113:6836/918732402,v1:172.21.15.113:6837/918732402] up:boot 2024-04-22T07:12:29.459 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:29 smithi113 ceph-mon[27933]: fsmap 4 up:standby 2024-04-22T07:12:29.459 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:29 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T07:12:29.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 -- 172.21.15.113:0/3864879314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000102a00 msgr2=0x7f9000104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:29.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 --2- 172.21.15.113:0/3864879314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000102a00 0x7f9000104df0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009ca0 tx=0x7f8ff802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 -- 172.21.15.113:0/3864879314 shutdown_connections 2024-04-22T07:12:29.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 --2- 172.21.15.113:0/3864879314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9000107c60 0x7f900010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 --2- 172.21.15.113:0/3864879314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9000105330 0x7f9000107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 --2- 172.21.15.113:0/3864879314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000102a00 0x7f9000104df0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 -- 172.21.15.113:0/3864879314 >> 172.21.15.113:0/3864879314 conn(0x7f90000fc410 msgr2=0x7f90000fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 -- 172.21.15.113:0/3864879314 shutdown_connections 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.545+0000 7f9006e81640 1 -- 172.21.15.113:0/3864879314 wait complete. 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 Processor -- start 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 -- start start 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9000102a00 0x7f90001982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9000105330 0x7f90001987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 0x7f900019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f900010dc00 con 0x7f9000107c60 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f900010da80 con 0x7f9000102a00 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9006e81640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f900010dd80 con 0x7f9000105330 2024-04-22T07:12:29.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 0x7f900019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 0x7f900019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33124/0 (socket says 172.21.15.113:33124) 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 -- 172.21.15.113:0/2218507442 learned_addr learned my addr 172.21.15.113:0/2218507442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f9004c1d640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9000102a00 0x7f90001982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f8ffffff640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9000105330 0x7f90001987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 -- 172.21.15.113:0/2218507442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9000105330 msgr2=0x7f90001987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9000105330 0x7f90001987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 -- 172.21.15.113:0/2218507442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9000102a00 msgr2=0x7f90001982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9000102a00 0x7f90001982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 -- 172.21.15.113:0/2218507442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900019fec0 con 0x7f9000107c60 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.546+0000 7f900541e640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 0x7f900019f850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000d950 tx=0x7f8ff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.547+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0017030 con 0x7f9000107c60 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.547+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90001a01b0 con 0x7f9000107c60 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.547+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8ff0004020 con 0x7f9000107c60 2024-04-22T07:12:29.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.547+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0003bc0 con 0x7f9000107c60 2024-04-22T07:12:29.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.547+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001a0700 con 0x7f9000107c60 2024-04-22T07:12:29.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.548+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90000fdf00 con 0x7f9000107c60 2024-04-22T07:12:29.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.549+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8ff001a070 con 0x7f9000107c60 2024-04-22T07:12:29.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.549+0000 7f8ffdffb640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8fec0a3bb0 0x7f8fec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:29.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.550+0000 7f9004c1d640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8fec0a3bb0 0x7f8fec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:29.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.550+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8ff00cf700 con 0x7f9000107c60 2024-04-22T07:12:29.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.550+0000 7f9004c1d640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8fec0a3bb0 0x7f8fec0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ff802c700 tx=0x7f8ff80325a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:29.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.554+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ff0021070 con 0x7f9000107c60 2024-04-22T07:12:29.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:29 smithi154 ceph-mon[31862]: Deploying daemon mds.e on smithi133 2024-04-22T07:12:29.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:29 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.113:6836/918732402,v1:172.21.15.113:6837/918732402] up:boot 2024-04-22T07:12:29.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:29 smithi154 ceph-mon[31862]: fsmap 4 up:standby 2024-04-22T07:12:29.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:29 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T07:12:29.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.770+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f900002d770 con 0x7f9000107c60 2024-04-22T07:12:29.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.771+0000 7f8ffdffb640 1 -- 172.21.15.113:0/2218507442 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f8ff009c580 con 0x7f9000107c60 2024-04-22T07:12:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:29.769 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T07:12:29.769 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 5 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8fec0a3bb0 msgr2=0x7f8fec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8fec0a3bb0 0x7f8fec0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8ff802c700 tx=0x7f8ff80325a0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 msgr2=0x7f900019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 0x7f900019f850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000d950 tx=0x7f8ff000de10 comp rx=0 tx=0).stop 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 shutdown_connections 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8fec0a3bb0 0x7f8fec0a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9000107c60 0x7f900019f850 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9000105330 0x7f90001987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 --2- 172.21.15.113:0/2218507442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9000102a00 0x7f90001982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 >> 172.21.15.113:0/2218507442 conn(0x7f90000fc410 msgr2=0x7f9000103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:29.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 shutdown_connections 2024-04-22T07:12:29.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:29.773+0000 7f9006e81640 1 -- 172.21.15.113:0/2218507442 wait complete. 2024-04-22T07:12:30.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:30 smithi133 ceph-mon[31198]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:30.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:30 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2218507442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:30 smithi154 ceph-mon[31862]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:30 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2218507442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:30 smithi113 ceph-mon[27933]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:30 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2218507442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:30.604 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:12:30 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: starting mds.e at 2024-04-22T07:12:30.604 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:12:30 smithi133 bash[61255]: f9cf8bfae3a662fc8b8f9c211a8bd498fdd9f48b4fda0dde611dd6c0266937d0 2024-04-22T07:12:31.064 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:12:30 smithi133 systemd[1]: Started Ceph mds.e for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:31.782 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:31.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 -- 172.21.15.113:0/3275508620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64101fd0 msgr2=0x7f3e64102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 --2- 172.21.15.113:0/3275508620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64101fd0 0x7f3e64102450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009900 tx=0x7f3e5802bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 -- 172.21.15.113:0/3275508620 shutdown_connections 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 --2- 172.21.15.113:0/3275508620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e64102990 0x7f3e641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 --2- 172.21.15.113:0/3275508620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64101fd0 0x7f3e64102450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 --2- 172.21.15.113:0/3275508620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64100e80 0x7f3e641012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 -- 172.21.15.113:0/3275508620 >> 172.21.15.113:0/3275508620 conn(0x7f3e640fc410 msgr2=0x7f3e640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 -- 172.21.15.113:0/3275508620 shutdown_connections 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 -- 172.21.15.113:0/3275508620 wait complete. 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.893+0000 7f3e6a575640 1 Processor -- start 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 -- start start 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e64100e80 0x7f3e6419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64101fd0 0x7f3e6419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:31.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 0x7f3e641a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410cb00 con 0x7f3e64102990 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410c980 con 0x7f3e64100e80 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e6a575640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410cc80 con 0x7f3e64101fd0 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 0x7f3e641a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 0x7f3e641a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33152/0 (socket says 172.21.15.113:33152) 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 -- 172.21.15.113:0/1313520050 learned_addr learned my addr 172.21.15.113:0/1313520050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e63fff640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e64100e80 0x7f3e6419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e637fe640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64101fd0 0x7f3e6419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 -- 172.21.15.113:0/1313520050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64101fd0 msgr2=0x7f3e6419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64101fd0 0x7f3e6419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 -- 172.21.15.113:0/1313520050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e64100e80 msgr2=0x7f3e6419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e64100e80 0x7f3e6419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 -- 172.21.15.113:0/1313520050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e641a2040 con 0x7f3e64102990 2024-04-22T07:12:31.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e637fe640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64101fd0 0x7f3e6419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.894+0000 7f3e68b12640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 0x7f3e641a19d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00eaa0 tx=0x7f3e5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.895+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00cdb0 con 0x7f3e64102990 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.895+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e641a2330 con 0x7f3e64102990 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.895+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3e5c005020 con 0x7f3e64102990 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.895+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0116d0 con 0x7f3e64102990 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.895+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e641a2830 con 0x7f3e64102990 2024-04-22T07:12:31.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.896+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e641a2af0 con 0x7f3e64102990 2024-04-22T07:12:31.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.897+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3e5c0051d0 con 0x7f3e64102990 2024-04-22T07:12:31.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.897+0000 7f3e617fa640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3e4c0a3bb0 0x7f3e4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:31.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.897+0000 7f3e63fff640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3e4c0a3bb0 0x7f3e4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:31.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.897+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3e5c0d0250 con 0x7f3e64102990 2024-04-22T07:12:31.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.898+0000 7f3e63fff640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3e4c0a3bb0 0x7f3e4c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e54005e10 tx=0x7f3e54005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:31.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:31.902+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e5c016030 con 0x7f3e64102990 2024-04-22T07:12:32.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:32.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:32.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:32.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: Deploying daemon mds.f on smithi154 2024-04-22T07:12:32.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:32.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:boot 2024-04-22T07:12:32.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: fsmap 5 up:standby 2024-04-22T07:12:32.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:31 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:12:32.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: Deploying daemon mds.f on smithi154 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:boot 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: fsmap 5 up:standby 2024-04-22T07:12:32.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:31 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:12:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:32.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: Deploying daemon mds.f on smithi154 2024-04-22T07:12:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:boot 2024-04-22T07:12:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: fsmap 5 up:standby 2024-04-22T07:12:32.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:31 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:12:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.118+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e641a2cc0 con 0x7f3e64102990 2024-04-22T07:12:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.119+0000 7f3e617fa640 1 -- 172.21.15.113:0/1313520050 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f3e5c09d0d0 con 0x7f3e64102990 2024-04-22T07:12:32.118 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:32.118 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T07:12:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 6 2024-04-22T07:12:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3e4c0a3bb0 msgr2=0x7f3e4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3e4c0a3bb0 0x7f3e4c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e54005e10 tx=0x7f3e54005da0 comp rx=0 tx=0).stop 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 msgr2=0x7f3e641a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 0x7f3e641a19d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00eaa0 tx=0x7f3e5c00ef60 comp rx=0 tx=0).stop 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 shutdown_connections 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3e4c0a3bb0 0x7f3e4c0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e64102990 0x7f3e641a19d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e64101fd0 0x7f3e6419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 --2- 172.21.15.113:0/1313520050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e64100e80 0x7f3e6419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 >> 172.21.15.113:0/1313520050 conn(0x7f3e640fc410 msgr2=0x7f3e640fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 shutdown_connections 2024-04-22T07:12:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:32.121+0000 7f3e6a575640 1 -- 172.21.15.113:0/1313520050 wait complete. 2024-04-22T07:12:32.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:32 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1313520050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:32.816 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:32 smithi154 systemd[1]: Starting Ceph mds.f for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:33.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:32 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1313520050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:32 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1313520050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:33.217 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:32 smithi154 podman[62943]: 2024-04-22 07:12:32.85695665 +0000 UTC m=+0.018498444 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:33.217 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:32 smithi154 podman[62943]: 2024-04-22 07:12:32.963936478 +0000 UTC m=+0.125478273 container create 1dd73fe23a4974c088fd820e1103b90e679e7de915c1ecab9d436b3bfba461e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:12:33.217 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:33 smithi154 podman[62943]: 2024-04-22 07:12:33.194580316 +0000 UTC m=+0.356122109 container init 1dd73fe23a4974c088fd820e1103b90e679e7de915c1ecab9d436b3bfba461e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:12:33.217 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:33 smithi154 podman[62943]: 2024-04-22 07:12:33.197263565 +0000 UTC m=+0.358805361 container start 1dd73fe23a4974c088fd820e1103b90e679e7de915c1ecab9d436b3bfba461e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True) 2024-04-22T07:12:33.217 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mds[62962]: -- 172.21.15.154:0/2870034875 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e1af6ba40 con 0x563e1af3e800 2024-04-22T07:12:33.566 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: starting mds.f at 2024-04-22T07:12:33.566 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:33 smithi154 bash[62943]: 1dd73fe23a4974c088fd820e1103b90e679e7de915c1ecab9d436b3bfba461e9 2024-04-22T07:12:33.566 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:12:33 smithi154 systemd[1]: Started Ceph mds.f for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:34.001 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:33 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:34.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:33 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:34.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:33 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:34.131 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:34.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 -- 172.21.15.113:0/2109519887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 msgr2=0x7fc93c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 --2- 172.21.15.113:0/2109519887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c1092b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc934009ca0 tx=0x7fc93402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 -- 172.21.15.113:0/2109519887 shutdown_connections 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 --2- 172.21.15.113:0/2109519887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c1092b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 --2- 172.21.15.113:0/2109519887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc93c101fd0 0x7fc93c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 --2- 172.21.15.113:0/2109519887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc93c100e80 0x7fc93c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 -- 172.21.15.113:0/2109519887 >> 172.21.15.113:0/2109519887 conn(0x7fc93c0fc410 msgr2=0x7fc93c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 -- 172.21.15.113:0/2109519887 shutdown_connections 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 -- 172.21.15.113:0/2109519887 wait complete. 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 Processor -- start 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.243+0000 7fc94092b640 1 -- start start 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc93c100e80 0x7fc93c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:34.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc93c101fd0 0x7fc93c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c10dc30 con 0x7fc93c102990 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c10dab0 con 0x7fc93c100e80 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c10ddb0 con 0x7fc93c101fd0 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33162/0 (socket says 172.21.15.113:33162) 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 -- 172.21.15.113:0/3493202398 learned_addr learned my addr 172.21.15.113:0/3493202398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc939d9b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc93c101fd0 0x7fc93c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93a59c640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc93c100e80 0x7fc93c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 -- 172.21.15.113:0/3493202398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc93c101fd0 msgr2=0x7fc93c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc93c101fd0 0x7fc93c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 -- 172.21.15.113:0/3493202398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc93c100e80 msgr2=0x7fc93c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:34.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc93c100e80 0x7fc93c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 -- 172.21.15.113:0/3493202398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c19fe80 con 0x7fc93c102990 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc93ad9d640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c19f810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc934009dc0 tx=0x7fc934032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934041020 con 0x7fc93c102990 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc934003e40 con 0x7fc93c102990 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934039040 con 0x7fc93c102990 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.244+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c1a0110 con 0x7fc93c102990 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.245+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c1a0600 con 0x7fc93c102990 2024-04-22T07:12:34.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.245+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c108130 con 0x7fc93c102990 2024-04-22T07:12:34.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.246+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc934044020 con 0x7fc93c102990 2024-04-22T07:12:34.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.247+0000 7fc92b7fe640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9240a3bb0 0x7fc9240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:34.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.247+0000 7fc93a59c640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9240a3bb0 0x7fc9240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:34.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.247+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc9340f22e0 con 0x7fc93c102990 2024-04-22T07:12:34.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.248+0000 7fc93a59c640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9240a3bb0 0x7fc9240a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00bbe0 tx=0x7fc92c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:34.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.252+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc93404b020 con 0x7fc93c102990 2024-04-22T07:12:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.468+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc93c062b50 con 0x7fc93c102990 2024-04-22T07:12:34.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.468+0000 7fc92b7fe640 1 -- 172.21.15.113:0/3493202398 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fc9340bf160 con 0x7fc93c102990 2024-04-22T07:12:34.467 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:34.467 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T07:12:34.468 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 7 2024-04-22T07:12:34.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9240a3bb0 msgr2=0x7fc9240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:34.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9240a3bb0 0x7fc9240a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00bbe0 tx=0x7fc92c00b390 comp rx=0 tx=0).stop 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 msgr2=0x7fc93c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c19f810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc934009dc0 tx=0x7fc934032950 comp rx=0 tx=0).stop 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 shutdown_connections 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc9240a3bb0 0x7fc9240a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc93c102990 0x7fc93c19f810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc93c101fd0 0x7fc93c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 --2- 172.21.15.113:0/3493202398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc93c100e80 0x7fc93c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 >> 172.21.15.113:0/3493202398 conn(0x7fc93c0fc410 msgr2=0x7fc93c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 shutdown_connections 2024-04-22T07:12:34.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:34.471+0000 7fc94092b640 1 -- 172.21.15.113:0/3493202398 wait complete. 2024-04-22T07:12:34.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:34 smithi113 ceph-mon[27933]: Deploying daemon mds.c on smithi154 2024-04-22T07:12:34.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:34 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:boot 2024-04-22T07:12:34.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:34 smithi113 ceph-mon[27933]: fsmap 6 up:standby 2024-04-22T07:12:34.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:34 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:12:34.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:34 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3493202398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:34.836 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:34 smithi133 ceph-mon[31198]: Deploying daemon mds.c on smithi154 2024-04-22T07:12:34.836 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:34 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:boot 2024-04-22T07:12:34.836 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:34 smithi133 ceph-mon[31198]: fsmap 6 up:standby 2024-04-22T07:12:34.836 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:34 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:12:34.836 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:34 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3493202398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:35.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:34 smithi154 ceph-mon[31862]: Deploying daemon mds.c on smithi154 2024-04-22T07:12:35.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:34 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:boot 2024-04-22T07:12:35.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:34 smithi154 ceph-mon[31862]: fsmap 6 up:standby 2024-04-22T07:12:35.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:34 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:12:35.008 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:34 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3493202398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:35.566 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 systemd[1]: Starting Ceph mds.c for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:35.955 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:35 smithi154 ceph-mon[31862]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:35.955 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 podman[63268]: 2024-04-22 07:12:35.606939336 +0000 UTC m=+0.018215855 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:35.955 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 podman[63268]: 2024-04-22 07:12:35.704323988 +0000 UTC m=+0.115600506 container create 3b72115b888e18c32f9321f0a102017386a5df0ddfbc2aed0220e98ef0c02088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T07:12:35.955 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 podman[63268]: 2024-04-22 07:12:35.932285145 +0000 UTC m=+0.343561663 container init 3b72115b888e18c32f9321f0a102017386a5df0ddfbc2aed0220e98ef0c02088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-22T07:12:35.955 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 podman[63268]: 2024-04-22 07:12:35.934887796 +0000 UTC m=+0.346164316 container start 3b72115b888e18c32f9321f0a102017386a5df0ddfbc2aed0220e98ef0c02088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-22T07:12:35.955 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 ceph-mds[63287]: -- 172.21.15.154:0/2594786804 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562eb2be5a40 con 0x562eb2bb8800 2024-04-22T07:12:36.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:35 smithi133 ceph-mon[31198]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:36.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:35 smithi113 ceph-mon[27933]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:36.318 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: starting mds.c at 2024-04-22T07:12:36.318 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:36 smithi154 bash[63268]: 3b72115b888e18c32f9321f0a102017386a5df0ddfbc2aed0220e98ef0c02088 2024-04-22T07:12:36.318 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:12:36 smithi154 systemd[1]: Started Ceph mds.c for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:36.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:36.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3662831340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b4107cf0 msgr2=0x7f15b410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3662831340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b4107cf0 0x7f15b410a0e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009ca0 tx=0x7f15ac02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3662831340 shutdown_connections 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3662831340 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b4107cf0 0x7f15b410a0e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3662831340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15b41053c0 0x7f15b41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3662831340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15b4102a90 0x7f15b4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3662831340 >> 172.21.15.113:0/3662831340 conn(0x7f15b40fc3e0 msgr2=0x7f15b40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3662831340 shutdown_connections 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3662831340 wait complete. 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 Processor -- start 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.592+0000 7f15bb7f4640 1 -- start start 2024-04-22T07:12:36.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15b4102a90 0x7f15b4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 0x7f15b4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15b4107cf0 0x7f15b419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f15b410dbe0 con 0x7f15b41053c0 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f15b410da60 con 0x7f15b4107cf0 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f15b410dd60 con 0x7f15b4102a90 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 0x7f15b4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 0x7f15b4198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48320/0 (socket says 172.21.15.113:48320) 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 -- 172.21.15.113:0/3210717310 learned_addr learned my addr 172.21.15.113:0/3210717310 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b9590640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15b4102a90 0x7f15b4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b9d91640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15b4107cf0 0x7f15b419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:36.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 -- 172.21.15.113:0/3210717310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15b4102a90 msgr2=0x7f15b4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15b4102a90 0x7f15b4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 -- 172.21.15.113:0/3210717310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15b4107cf0 msgr2=0x7f15b419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15b4107cf0 0x7f15b419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 -- 172.21.15.113:0/3210717310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b419fe50 con 0x7f15b41053c0 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15b8d8f640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 0x7f15b4198770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c970 tx=0x7f15b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0007bb0 con 0x7f15b41053c0 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f15b0007d50 con 0x7f15b41053c0 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b41a0140 con 0x7f15b41053c0 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b000f450 con 0x7f15b41053c0 2024-04-22T07:12:36.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.593+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41a0690 con 0x7f15b41053c0 2024-04-22T07:12:36.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.594+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b40fded0 con 0x7f15b41053c0 2024-04-22T07:12:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.595+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f15b00046c0 con 0x7f15b41053c0 2024-04-22T07:12:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.596+0000 7f15aa7fc640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f15a00a3bb0 0x7f15a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.596+0000 7f15b9590640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f15a00a3bb0 0x7f15a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.596+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f15b00cff00 con 0x7f15b41053c0 2024-04-22T07:12:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.596+0000 7f15b9590640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f15a00a3bb0 0x7f15a00a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009bd0 tx=0x7f15a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:36.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.600+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15b0014030 con 0x7f15b41053c0 2024-04-22T07:12:36.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.815+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15b402d810 con 0x7f15b41053c0 2024-04-22T07:12:36.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.815+0000 7f15aa7fc640 1 -- 172.21.15.113:0/3210717310 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f15b009c730 con 0x7f15b41053c0 2024-04-22T07:12:36.814 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T07:12:36.814 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 8 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f15a00a3bb0 msgr2=0x7f15a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f15a00a3bb0 0x7f15a00a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009bd0 tx=0x7f15a4005f90 comp rx=0 tx=0).stop 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 msgr2=0x7f15b4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 0x7f15b4198770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c970 tx=0x7f15b000ce30 comp rx=0 tx=0).stop 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 shutdown_connections 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f15a00a3bb0 0x7f15a00a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15b4107cf0 0x7f15b419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15b41053c0 0x7f15b4198770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 --2- 172.21.15.113:0/3210717310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15b4102a90 0x7f15b4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:36.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 >> 172.21.15.113:0/3210717310 conn(0x7f15b40fc3e0 msgr2=0x7f15b41033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:36.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 shutdown_connections 2024-04-22T07:12:36.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:36.817+0000 7f15bb7f4640 1 -- 172.21.15.113:0/3210717310 wait complete. 2024-04-22T07:12:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:37.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: Deploying daemon mds.h on smithi133 2024-04-22T07:12:37.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:boot 2024-04-22T07:12:37.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: fsmap 7 up:standby 2024-04-22T07:12:37.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:12:37.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:37 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3210717310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: Deploying daemon mds.h on smithi133 2024-04-22T07:12:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:boot 2024-04-22T07:12:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: fsmap 7 up:standby 2024-04-22T07:12:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:12:37.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:37 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3210717310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: Deploying daemon mds.h on smithi133 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:boot 2024-04-22T07:12:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: fsmap 7 up:standby 2024-04-22T07:12:37.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:12:37.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:37 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3210717310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:38.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:38 smithi133 ceph-mon[31198]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:38.314 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 systemd[1]: Starting Ceph mds.h for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:38 smithi113 ceph-mon[27933]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:38.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:38 smithi154 ceph-mon[31862]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:38.726 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 podman[61574]: 2024-04-22 07:12:38.435182225 +0000 UTC m=+0.102940240 container create 9a81b14fc52ea01479e3415df909a31c27a38eb2223e9a8c500513adaa6a5d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:12:38.727 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 podman[61574]: 2024-04-22 07:12:38.35100745 +0000 UTC m=+0.018765471 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:38.727 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 podman[61574]: 2024-04-22 07:12:38.654281612 +0000 UTC m=+0.322039621 container init 9a81b14fc52ea01479e3415df909a31c27a38eb2223e9a8c500513adaa6a5d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-22T07:12:38.727 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 podman[61574]: 2024-04-22 07:12:38.656922748 +0000 UTC m=+0.324680765 container start 9a81b14fc52ea01479e3415df909a31c27a38eb2223e9a8c500513adaa6a5d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-22T07:12:38.727 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 ceph-mds[61593]: -- 172.21.15.133:0/1357411943 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557aef425a40 con 0x557aef3f8800 2024-04-22T07:12:38.727 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: starting mds.h at 2024-04-22T07:12:38.727 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 bash[61574]: 9a81b14fc52ea01479e3415df909a31c27a38eb2223e9a8c500513adaa6a5d89 2024-04-22T07:12:38.827 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 -- 172.21.15.113:0/3322321495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984101fc0 msgr2=0x7fc984102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/3322321495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984101fc0 0x7fc984102440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc974009900 tx=0x7fc97402bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 -- 172.21.15.113:0/3322321495 shutdown_connections 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/3322321495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc984102980 0x7fc9841092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/3322321495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984101fc0 0x7fc984102440 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/3322321495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984100e70 0x7fc984101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 -- 172.21.15.113:0/3322321495 >> 172.21.15.113:0/3322321495 conn(0x7fc9840fc3e0 msgr2=0x7fc9840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 -- 172.21.15.113:0/3322321495 shutdown_connections 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 -- 172.21.15.113:0/3322321495 wait complete. 2024-04-22T07:12:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 Processor -- start 2024-04-22T07:12:38.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.939+0000 7fc98a4f1640 1 -- start start 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 0x7fc98410b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc984101fc0 0x7fc984109e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984102980 0x7fc98410a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410dc20 con 0x7fc984100e70 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410daa0 con 0x7fc984102980 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc98410dda0 con 0x7fc984101fc0 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 0x7fc98410b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 0x7fc98410b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48334/0 (socket says 172.21.15.113:48334) 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 -- 172.21.15.113:0/1379840253 learned_addr learned my addr 172.21.15.113:0/1379840253 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc9837fe640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc984101fc0 0x7fc984109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc988a8e640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984102980 0x7fc98410a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 -- 172.21.15.113:0/1379840253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc984101fc0 msgr2=0x7fc984109e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc984101fc0 0x7fc984109e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 -- 172.21.15.113:0/1379840253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984102980 msgr2=0x7fc98410a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984102980 0x7fc98410a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 -- 172.21.15.113:0/1379840253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98410ac20 con 0x7fc984100e70 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc988a8e640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984102980 0x7fc98410a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc983fff640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 0x7fc98410b810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc97800d950 tx=0x7fc97800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978017030 con 0x7fc984100e70 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a2400 con 0x7fc984100e70 2024-04-22T07:12:38.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc978004020 con 0x7fc984100e70 2024-04-22T07:12:38.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780047a0 con 0x7fc984100e70 2024-04-22T07:12:38.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.940+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a2970 con 0x7fc984100e70 2024-04-22T07:12:38.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.941+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc984108250 con 0x7fc984100e70 2024-04-22T07:12:38.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.942+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc9780041d0 con 0x7fc984100e70 2024-04-22T07:12:38.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.943+0000 7fc9817fa640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc96c0a3bb0 0x7fc96c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:38.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.943+0000 7fc9837fe640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc96c0a3bb0 0x7fc96c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:38.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.943+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc9780cfa50 con 0x7fc984100e70 2024-04-22T07:12:38.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.943+0000 7fc9837fe640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc96c0a3bb0 0x7fc96c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc974009230 tx=0x7fc974005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:38.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:38.948+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc97809c8d0 con 0x7fc984100e70 2024-04-22T07:12:39.064 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:12:38 smithi133 systemd[1]: Started Ceph mds.h for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.165+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc98402d5c0 con 0x7fc984100e70 2024-04-22T07:12:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.166+0000 7fc9817fa640 1 -- 172.21.15.113:0/1379840253 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fc97809c1d0 con 0x7fc984100e70 2024-04-22T07:12:39.165 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T07:12:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 9 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc96c0a3bb0 msgr2=0x7fc96c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc96c0a3bb0 0x7fc96c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc974009230 tx=0x7fc974005fd0 comp rx=0 tx=0).stop 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 msgr2=0x7fc98410b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 0x7fc98410b810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc97800d950 tx=0x7fc97800de10 comp rx=0 tx=0).stop 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 shutdown_connections 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc96c0a3bb0 0x7fc96c0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc984102980 0x7fc98410a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc984101fc0 0x7fc984109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 --2- 172.21.15.113:0/1379840253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984100e70 0x7fc98410b810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 >> 172.21.15.113:0/1379840253 conn(0x7fc9840fc3e0 msgr2=0x7fc984105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 shutdown_connections 2024-04-22T07:12:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:39.168+0000 7fc98a4f1640 1 -- 172.21.15.113:0/1379840253 wait complete. 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: Deploying daemon mds.i on smithi154 2024-04-22T07:12:40.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:40.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] up:boot 2024-04-22T07:12:40.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: fsmap 8 up:standby 2024-04-22T07:12:40.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:12:40.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:39 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1379840253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:40.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: Deploying daemon mds.i on smithi154 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] up:boot 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: fsmap 8 up:standby 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:12:40.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:39 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1379840253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:40.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: Deploying daemon mds.i on smithi154 2024-04-22T07:12:40.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:40.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] up:boot 2024-04-22T07:12:40.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: fsmap 8 up:standby 2024-04-22T07:12:40.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:12:40.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:39 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1379840253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:41.066 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:40 smithi154 systemd[1]: Starting Ceph mds.i for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:41.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:41.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 -- 172.21.15.113:0/1390112141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 msgr2=0x7f13341036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:41.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 --2- 172.21.15.113:0/1390112141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f13341036c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1324009930 tx=0x7f132402bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:41.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 -- 172.21.15.113:0/1390112141 shutdown_connections 2024-04-22T07:12:41.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 --2- 172.21.15.113:0/1390112141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1334103c00 0x7f133410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 --2- 172.21.15.113:0/1390112141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f13341036c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 --2- 172.21.15.113:0/1390112141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 0x7f1334100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 -- 172.21.15.113:0/1390112141 >> 172.21.15.113:0/1390112141 conn(0x7f13340fbba0 msgr2=0x7f13340fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 -- 172.21.15.113:0/1390112141 shutdown_connections 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.292+0000 7f133c274640 1 -- 172.21.15.113:0/1390112141 wait complete. 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 Processor -- start 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 -- start start 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 0x7f1334198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f1334198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1334103c00 0x7f133419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f133410dcd0 con 0x7f1334103240 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f133410db50 con 0x7f1334100270 2024-04-22T07:12:41.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133c274640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f133410de50 con 0x7f1334103c00 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f1334198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f1334198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48360/0 (socket says 172.21.15.113:48360) 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 -- 172.21.15.113:0/4269522591 learned_addr learned my addr 172.21.15.113:0/4269522591 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133a811640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1334103c00 0x7f133419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133a010640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 0x7f1334198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 -- 172.21.15.113:0/4269522591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1334103c00 msgr2=0x7f133419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1334103c00 0x7f133419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 -- 172.21.15.113:0/4269522591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 msgr2=0x7f1334198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 0x7f1334198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.293+0000 7f133980f640 1 -- 172.21.15.113:0/4269522591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133419ff40 con 0x7f1334103240 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f133a010640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 0x7f1334198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f133980f640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f1334198860 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1324009430 tx=0x7f1324005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324043020 con 0x7f1334103240 2024-04-22T07:12:41.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13341a01d0 con 0x7f1334103240 2024-04-22T07:12:41.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f13240043d0 con 0x7f1334103240 2024-04-22T07:12:41.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a07a0 con 0x7f1334103240 2024-04-22T07:12:41.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.294+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324003b50 con 0x7f1334103240 2024-04-22T07:12:41.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.295+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1334109370 con 0x7f1334103240 2024-04-22T07:12:41.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.296+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1324046020 con 0x7f1334103240 2024-04-22T07:12:41.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.297+0000 7f132b7fe640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13200a3bb0 0x7f13200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:41.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.297+0000 7f133a010640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13200a3bb0 0x7f13200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:41.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.297+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f13240f1e40 con 0x7f1334103240 2024-04-22T07:12:41.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.297+0000 7f133a010640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13200a3bb0 0x7f13200a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f133406a460 tx=0x7f1330008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:41.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.301+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13240becc0 con 0x7f1334103240 2024-04-22T07:12:41.401 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 podman[63576]: 2024-04-22 07:12:41.049130316 +0000 UTC m=+0.018162665 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:41.401 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 podman[63576]: 2024-04-22 07:12:41.175162932 +0000 UTC m=+0.144195272 container create 2cb0ad707540d628b04dac19a91b06cb60740e4f119678cd55fa01cf4beb7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:12:41.401 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 podman[63576]: 2024-04-22 07:12:41.377671167 +0000 UTC m=+0.346703507 container init 2cb0ad707540d628b04dac19a91b06cb60740e4f119678cd55fa01cf4beb7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-22T07:12:41.401 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 podman[63576]: 2024-04-22 07:12:41.380291679 +0000 UTC m=+0.349324022 container start 2cb0ad707540d628b04dac19a91b06cb60740e4f119678cd55fa01cf4beb7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:12:41.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.522+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f133402d810 con 0x7f1334103240 2024-04-22T07:12:41.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.524+0000 7f132b7fe640 1 -- 172.21.15.113:0/4269522591 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f13240be5c0 con 0x7f1334103240 2024-04-22T07:12:41.523 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T07:12:41.523 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 9 2024-04-22T07:12:41.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13200a3bb0 msgr2=0x7f13200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13200a3bb0 0x7f13200a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f133406a460 tx=0x7f1330008040 comp rx=0 tx=0).stop 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 msgr2=0x7f1334198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f1334198860 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1324009430 tx=0x7f1324005e40 comp rx=0 tx=0).stop 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 shutdown_connections 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13200a3bb0 0x7f13200a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1334103c00 0x7f133419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1334103240 0x7f1334198860 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 --2- 172.21.15.113:0/4269522591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1334100270 0x7f1334198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 >> 172.21.15.113:0/4269522591 conn(0x7f13340fbba0 msgr2=0x7f1334106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 shutdown_connections 2024-04-22T07:12:41.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:41.527+0000 7f133c274640 1 -- 172.21.15.113:0/4269522591 wait complete. 2024-04-22T07:12:41.816 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 ceph-mds[63595]: -- 172.21.15.154:0/4136490931 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609ba8d7a40 con 0x5609ba8aa800 2024-04-22T07:12:41.816 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: starting mds.i at 2024-04-22T07:12:41.816 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 bash[63576]: 2cb0ad707540d628b04dac19a91b06cb60740e4f119678cd55fa01cf4beb7d3d 2024-04-22T07:12:41.816 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:12:41 smithi154 systemd[1]: Started Ceph mds.i for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4269522591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:42.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:42 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4269522591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:42 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4269522591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:42.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:42 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:43.425 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:43 smithi113 ceph-mon[27933]: Deploying daemon mds.g on smithi113 2024-04-22T07:12:43.425 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:43 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] up:boot 2024-04-22T07:12:43.425 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:43 smithi113 ceph-mon[27933]: fsmap 9 up:standby 2024-04-22T07:12:43.426 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:43 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:12:43.537 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:43.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:43 smithi133 ceph-mon[31198]: Deploying daemon mds.g on smithi113 2024-04-22T07:12:43.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:43 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] up:boot 2024-04-22T07:12:43.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:43 smithi133 ceph-mon[31198]: fsmap 9 up:standby 2024-04-22T07:12:43.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:43 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:12:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:43 smithi154 ceph-mon[31862]: Deploying daemon mds.g on smithi113 2024-04-22T07:12:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:43 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] up:boot 2024-04-22T07:12:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:43 smithi154 ceph-mon[31862]: fsmap 9 up:standby 2024-04-22T07:12:43.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:43 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:12:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/3109962557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0101f80 msgr2=0x7f8bd01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/3109962557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0101f80 0x7f8bd01023a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009900 tx=0x7f8bcc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/3109962557 shutdown_connections 2024-04-22T07:12:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/3109962557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd0103b30 0x7f8bd010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/3109962557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bd0103170 0x7f8bd01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/3109962557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0101f80 0x7f8bd01023a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/3109962557 >> 172.21.15.113:0/3109962557 conn(0x7f8bd00fd530 msgr2=0x7f8bd00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/3109962557 shutdown_connections 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.675+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/3109962557 wait complete. 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 Processor -- start 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 -- start start 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bd0101f80 0x7f8bd0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd0103170 0x7f8bd0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 0x7f8bd0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd010dbe0 con 0x7f8bd0103b30 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd010da60 con 0x7f8bd0101f80 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd7fd5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd010dd60 con 0x7f8bd0103170 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 0x7f8bd0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 0x7f8bd0074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48370/0 (socket says 172.21.15.113:48370) 2024-04-22T07:12:43.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 -- 172.21.15.113:0/682899550 learned_addr learned my addr 172.21.15.113:0/682899550 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd5d71640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bd0101f80 0x7f8bd0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd5570640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd0103170 0x7f8bd0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 -- 172.21.15.113:0/682899550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd0103170 msgr2=0x7f8bd0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd0103170 0x7f8bd0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 -- 172.21.15.113:0/682899550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bd0101f80 msgr2=0x7f8bd0075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bd0101f80 0x7f8bd0075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.676+0000 7f8bd6572640 1 -- 172.21.15.113:0/682899550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd0075110 con 0x7f8bd0103b30 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.677+0000 7f8bd6572640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 0x7f8bd0074880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800eaa0 tx=0x7f8bc800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.677+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800cdb0 con 0x7f8bd0103b30 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.677+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd01a23b0 con 0x7f8bd0103b30 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.677+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8bc8005020 con 0x7f8bd0103b30 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.677+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80116d0 con 0x7f8bd0103b30 2024-04-22T07:12:43.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.677+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd01a28b0 con 0x7f8bd0103b30 2024-04-22T07:12:43.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.678+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd01092f0 con 0x7f8bd0103b30 2024-04-22T07:12:43.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.679+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8bc80051d0 con 0x7f8bd0103b30 2024-04-22T07:12:43.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.679+0000 7f8bc6ffd640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8bbc0a3bb0 0x7f8bbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:43.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.680+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8bc80d0200 con 0x7f8bd0103b30 2024-04-22T07:12:43.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.680+0000 7f8bd5d71640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8bbc0a3bb0 0x7f8bbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:43.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.680+0000 7f8bd5d71640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8bbc0a3bb0 0x7f8bbc0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009230 tx=0x7f8bcc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:43.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.685+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bc8016030 con 0x7f8bd0103b30 2024-04-22T07:12:43.834 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:43 smithi113 systemd[1]: Starting Ceph mds.g for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:43.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.918+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd002d770 con 0x7f8bd0103b30 2024-04-22T07:12:43.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.919+0000 7f8bc6ffd640 1 -- 172.21.15.113:0/682899550 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f8bc809d080 con 0x7f8bd0103b30 2024-04-22T07:12:43.917 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T07:12:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 10 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8bbc0a3bb0 msgr2=0x7f8bbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8bbc0a3bb0 0x7f8bbc0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009230 tx=0x7f8bcc005fd0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 msgr2=0x7f8bd0074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 0x7f8bd0074880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800eaa0 tx=0x7f8bc800ef60 comp rx=0 tx=0).stop 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 shutdown_connections 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8bbc0a3bb0 0x7f8bbc0a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8bd0103b30 0x7f8bd0074880 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bd0103170 0x7f8bd0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 --2- 172.21.15.113:0/682899550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bd0101f80 0x7f8bd0075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 >> 172.21.15.113:0/682899550 conn(0x7f8bd00fd530 msgr2=0x7f8bd01064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 shutdown_connections 2024-04-22T07:12:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:43.921+0000 7f8bd7fd5640 1 -- 172.21.15.113:0/682899550 wait complete. 2024-04-22T07:12:44.101 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:43 smithi113 podman[66874]: 2024-04-22 07:12:43.742524815 +0000 UTC m=+0.022604334 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:44.102 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:43 smithi113 podman[66874]: 2024-04-22 07:12:43.841608199 +0000 UTC m=+0.121687718 container create 674baabee99666ae1f6baffdebdb4db4b26135f9e88764271c14ef0772999ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T07:12:44.102 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:44 smithi113 podman[66874]: 2024-04-22 07:12:44.078012198 +0000 UTC m=+0.358091718 container init 674baabee99666ae1f6baffdebdb4db4b26135f9e88764271c14ef0772999ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:12:44.102 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:44 smithi113 podman[66874]: 2024-04-22 07:12:44.080760674 +0000 UTC m=+0.360840200 container start 674baabee99666ae1f6baffdebdb4db4b26135f9e88764271c14ef0772999ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-22T07:12:44.102 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:44 smithi113 ceph-mds[66895]: -- 172.21.15.113:0/375341527 <== mon.0 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a4e9c3a40 con 0x564a4e996800 2024-04-22T07:12:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:44 smithi133 ceph-mon[31198]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:44 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/682899550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:44.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:44 smithi154 ceph-mon[31862]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:44 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/682899550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:44 smithi113 ceph-mon[27933]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:44.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:44 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/682899550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:44.585 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g[66891]: starting mds.g at 2024-04-22T07:12:44.585 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:44 smithi113 bash[66874]: 674baabee99666ae1f6baffdebdb4db4b26135f9e88764271c14ef0772999ec5 2024-04-22T07:12:44.585 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:12:44 smithi113 systemd[1]: Started Ceph mds.g for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: Deploying daemon mds.a on smithi113 2024-04-22T07:12:45.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:45.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.113:6838/1495688995,v1:172.21.15.113:6839/1495688995] up:boot 2024-04-22T07:12:45.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: fsmap 10 up:standby 2024-04-22T07:12:45.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:45 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T07:12:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: Deploying daemon mds.a on smithi113 2024-04-22T07:12:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.113:6838/1495688995,v1:172.21.15.113:6839/1495688995] up:boot 2024-04-22T07:12:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: fsmap 10 up:standby 2024-04-22T07:12:45.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:45 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: Deploying daemon mds.a on smithi113 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:45.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.113:6838/1495688995,v1:172.21.15.113:6839/1495688995] up:boot 2024-04-22T07:12:45.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: fsmap 10 up:standby 2024-04-22T07:12:45.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:45 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T07:12:45.931 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 -- 172.21.15.113:0/774817601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 msgr2=0x7fe9ac10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 --2- 172.21.15.113:0/774817601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac10dd50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009900 tx=0x7fe99c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 -- 172.21.15.113:0/774817601 shutdown_connections 2024-04-22T07:12:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 --2- 172.21.15.113:0/774817601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac10e290 0x7fe9ac10e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 --2- 172.21.15.113:0/774817601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac10dd50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 --2- 172.21.15.113:0/774817601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9ac108f50 0x7fe9ac10b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 -- 172.21.15.113:0/774817601 >> 172.21.15.113:0/774817601 conn(0x7fe9ac06f6d0 msgr2=0x7fe9ac071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 -- 172.21.15.113:0/774817601 shutdown_connections 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.120+0000 7fe9b4470640 1 -- 172.21.15.113:0/774817601 wait complete. 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 Processor -- start 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 -- start start 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac108f50 0x7fe9ac1a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac1a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9ac10e290 0x7fe9ac1ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac11ca10 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac11c890 con 0x7fe9ac10e290 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b4470640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac11cb90 con 0x7fe9ac108f50 2024-04-22T07:12:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b1a0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac1a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b1a0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac1a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52252/0 (socket says 172.21.15.113:52252) 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.121+0000 7fe9b1a0b640 1 -- 172.21.15.113:0/2722925716 learned_addr learned my addr 172.21.15.113:0/2722925716 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b220c640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac108f50 0x7fe9ac1a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b1a0b640 1 -- 172.21.15.113:0/2722925716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac108f50 msgr2=0x7fe9ac1a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b1a0b640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac108f50 0x7fe9ac1a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b1a0b640 1 -- 172.21.15.113:0/2722925716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9ac10e290 msgr2=0x7fe9ac1ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b1a0b640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9ac10e290 0x7fe9ac1ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b1a0b640 1 -- 172.21.15.113:0/2722925716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac1aec80 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b220c640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac108f50 0x7fe9ac1a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b1a0b640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac1a75a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe99c005ab0 tx=0x7fe99c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c041070 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac1aef10 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe99c0052f0 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c003dc0 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.122+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac1af420 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.123+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac0721d0 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.124+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe99c044070 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.125+0000 7fe9a37fe640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe9980a3bb0 0x7fe9980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.125+0000 7fe9b220c640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe9980a3bb0 0x7fe9980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.125+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe99c0f1bd0 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.126+0000 7fe9b220c640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe9980a3bb0 0x7fe9980a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac073d10 tx=0x7fe9a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.129+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe99c0bea50 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.334 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 systemd[1]: Starting Ceph mds.a for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:46.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.369+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9ac02d7e0 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.370+0000 7fe9a37fe640 1 -- 172.21.15.113:0/2722925716 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fe99c0be350 con 0x7fe9ac10b8c0 2024-04-22T07:12:46.368 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:46.368 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T07:12:46.369 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 11 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.372+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe9980a3bb0 msgr2=0x7fe9980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.372+0000 7fe9b4470640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe9980a3bb0 0x7fe9980a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac073d10 tx=0x7fe9a8005f90 comp rx=0 tx=0).stop 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.372+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 msgr2=0x7fe9ac1a75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.372+0000 7fe9b4470640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac1a75a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe99c005ab0 tx=0x7fe99c005e60 comp rx=0 tx=0).stop 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 shutdown_connections 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe9980a3bb0 0x7fe9980a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9ac10e290 0x7fe9ac1ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9ac10b8c0 0x7fe9ac1a75a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 --2- 172.21.15.113:0/2722925716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9ac108f50 0x7fe9ac1a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 >> 172.21.15.113:0/2722925716 conn(0x7fe9ac06f6d0 msgr2=0x7fe9ac10a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 shutdown_connections 2024-04-22T07:12:46.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:46.373+0000 7fe9b4470640 1 -- 172.21.15.113:0/2722925716 wait complete. 2024-04-22T07:12:46.600 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 podman[67235]: 2024-04-22 07:12:46.394359466 +0000 UTC m=+0.018156779 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:46.600 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 podman[67235]: 2024-04-22 07:12:46.513604586 +0000 UTC m=+0.137401886 container create 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=) 2024-04-22T07:12:46.600 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:46 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2722925716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:46.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:46 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2722925716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:46.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:46 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2722925716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:47.084 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 podman[67235]: 2024-04-22 07:12:46.707764676 +0000 UTC m=+0.331561976 container init 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-22T07:12:47.084 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 podman[67235]: 2024-04-22 07:12:46.710498759 +0000 UTC m=+0.334296060 container start 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-22T07:12:47.084 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 ceph-mds[67255]: -- 172.21.15.113:0/3869810208 <== mon.0 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575b8b4da40 con 0x5575b8b21000 2024-04-22T07:12:47.084 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a[67251]: starting mds.a at 2024-04-22T07:12:47.085 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 bash[67235]: 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e 2024-04-22T07:12:47.085 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:12:46 smithi113 systemd[1]: Started Ceph mds.a for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:48.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: Deploying daemon mds.b on smithi133 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.113:6840/1868487444,v1:172.21.15.113:6841/1868487444] up:boot 2024-04-22T07:12:48.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: fsmap 11 up:standby 2024-04-22T07:12:48.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:47 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T07:12:48.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: Deploying daemon mds.b on smithi133 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.113:6840/1868487444,v1:172.21.15.113:6841/1868487444] up:boot 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: fsmap 11 up:standby 2024-04-22T07:12:48.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:47 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T07:12:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: Deploying daemon mds.b on smithi133 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.113:6840/1868487444,v1:172.21.15.113:6841/1868487444] up:boot 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: fsmap 11 up:standby 2024-04-22T07:12:48.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:47 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T07:12:48.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:48.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 -- 172.21.15.113:0/3902150287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80105330 msgr2=0x7f0d80107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:48.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3902150287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80105330 0x7f0d80107720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009900 tx=0x7f0d6c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:48.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 -- 172.21.15.113:0/3902150287 shutdown_connections 2024-04-22T07:12:48.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3902150287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80107c60 0x7f0d8010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3902150287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80105330 0x7f0d80107720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3902150287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80102a00 0x7f0d80104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.497+0000 7f0d871e2640 1 -- 172.21.15.113:0/3902150287 >> 172.21.15.113:0/3902150287 conn(0x7f0d800fc410 msgr2=0x7f0d800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 -- 172.21.15.113:0/3902150287 shutdown_connections 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 -- 172.21.15.113:0/3902150287 wait complete. 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 Processor -- start 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 -- start start 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80102a00 0x7f0d801982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80105330 0x7f0d801987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 0x7f0d8019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010dbf0 con 0x7f0d80107c60 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010da70 con 0x7f0d80105330 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d871e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8010dd70 con 0x7f0d80102a00 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d8577f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 0x7f0d8019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:48.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d8577f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 0x7f0d8019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52290/0 (socket says 172.21.15.113:52290) 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d8577f640 1 -- 172.21.15.113:0/3425795651 learned_addr learned my addr 172.21.15.113:0/3425795651 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.498+0000 7f0d84f7e640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80102a00 0x7f0d801982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d7ffff640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80105330 0x7f0d801987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d8577f640 1 -- 172.21.15.113:0/3425795651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80102a00 msgr2=0x7f0d801982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d8577f640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80102a00 0x7f0d801982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d8577f640 1 -- 172.21.15.113:0/3425795651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80105330 msgr2=0x7f0d801987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d8577f640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80105330 0x7f0d801987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d8577f640 1 -- 172.21.15.113:0/3425795651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8019fec0 con 0x7f0d80107c60 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d7ffff640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80105330 0x7f0d801987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d8577f640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 0x7f0d8019f850 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800eaa0 tx=0x7f0d7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800cdb0 con 0x7f0d80107c60 2024-04-22T07:12:48.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d78005020 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d801a01b0 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d780116d0 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.499+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a0680 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.500+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d800fdf00 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.501+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0d780051d0 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.502+0000 7f0d7dffb640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0d700a3bb0 0x7f0d700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.502+0000 7f0d84f7e640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0d700a3bb0 0x7f0d700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.502+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0d780d0580 con 0x7f0d80107c60 2024-04-22T07:12:48.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.503+0000 7f0d84f7e640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0d700a3bb0 0x7f0d700a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005fd0 tx=0x7f0d7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:48.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.506+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d78016030 con 0x7f0d80107c60 2024-04-22T07:12:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.724+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d8002d5c0 con 0x7f0d80107c60 2024-04-22T07:12:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.724+0000 7f0d7dffb640 1 -- 172.21.15.113:0/3425795651 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f0d7809d400 con 0x7f0d80107c60 2024-04-22T07:12:48.722 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:48.723 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T07:12:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 12 2024-04-22T07:12:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0d700a3bb0 msgr2=0x7f0d700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0d700a3bb0 0x7f0d700a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005fd0 tx=0x7f0d7400b390 comp rx=0 tx=0).stop 2024-04-22T07:12:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 msgr2=0x7f0d8019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 0x7f0d8019f850 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800eaa0 tx=0x7f0d7800ef60 comp rx=0 tx=0).stop 2024-04-22T07:12:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 shutdown_connections 2024-04-22T07:12:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0d700a3bb0 0x7f0d700a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d80107c60 0x7f0d8019f850 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d80105330 0x7f0d801987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 --2- 172.21.15.113:0/3425795651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d80102a00 0x7f0d801982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.726+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 >> 172.21.15.113:0/3425795651 conn(0x7f0d800fc410 msgr2=0x7f0d801032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.727+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 shutdown_connections 2024-04-22T07:12:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:48.727+0000 7f0d871e2640 1 -- 172.21.15.113:0/3425795651 wait complete. 2024-04-22T07:12:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3425795651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:49.064 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:48 smithi133 systemd[1]: Starting Ceph mds.b for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:12:49.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:48 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3425795651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:49.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:48 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3425795651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:49.506 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 podman[61881]: 2024-04-22 07:12:49.143722522 +0000 UTC m=+0.018342660 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:12:49.507 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 podman[61881]: 2024-04-22 07:12:49.246968774 +0000 UTC m=+0.121588903 container create b19aca13ef750ab1f688fcc11759debdcf990ded1d09711f5733bacbcb6786e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:12:49.507 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 podman[61881]: 2024-04-22 07:12:49.482750638 +0000 UTC m=+0.357370764 container init b19aca13ef750ab1f688fcc11759debdcf990ded1d09711f5733bacbcb6786e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-22T07:12:49.507 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 podman[61881]: 2024-04-22 07:12:49.485343686 +0000 UTC m=+0.359963825 container start b19aca13ef750ab1f688fcc11759debdcf990ded1d09711f5733bacbcb6786e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-22T07:12:49.507 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mds[61900]: -- 172.21.15.133:0/3237480040 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a31ff5a40 con 0x558a31fc9000 2024-04-22T07:12:49.814 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: starting mds.b at 2024-04-22T07:12:49.814 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 bash[61881]: b19aca13ef750ab1f688fcc11759debdcf990ded1d09711f5733bacbcb6786e2 2024-04-22T07:12:49.814 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:49 smithi133 systemd[1]: Started Ceph mds.b for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:49 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:50.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:50.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:49 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:49 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:50.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- 172.21.15.113:0/2706167759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118102980 msgr2=0x7f81181092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2706167759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118102980 0x7f81181092a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8110009ca0 tx=0x7f811002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- 172.21.15.113:0/2706167759 shutdown_connections 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2706167759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118102980 0x7f81181092a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2706167759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 0x7f8118102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2706167759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8118100e70 0x7f8118101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- 172.21.15.113:0/2706167759 >> 172.21.15.113:0/2706167759 conn(0x7f81180fc3e0 msgr2=0x7f81180fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- 172.21.15.113:0/2706167759 shutdown_connections 2024-04-22T07:12:50.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- 172.21.15.113:0/2706167759 wait complete. 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 Processor -- start 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- start start 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 0x7f8118198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 0x7f81181987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8118102980 0x7f811819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f811810cb10 con 0x7f8118100e70 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f811810c990 con 0x7f8118102980 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f811dfb6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f811810cc90 con 0x7f8118101fc0 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f81177fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 0x7f8118198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f81177fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 0x7f8118198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52314/0 (socket says 172.21.15.113:52314) 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.839+0000 7f81177fe640 1 -- 172.21.15.113:0/2547323773 learned_addr learned my addr 172.21.15.113:0/2547323773 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f8117fff640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8118102980 0x7f811819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f810ffff640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 0x7f81181987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:50.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81177fe640 1 -- 172.21.15.113:0/2547323773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 msgr2=0x7f81181987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81177fe640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 0x7f81181987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81177fe640 1 -- 172.21.15.113:0/2547323773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8118102980 msgr2=0x7f811819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81177fe640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8118102980 0x7f811819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81177fe640 1 -- 172.21.15.113:0/2547323773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811819fe60 con 0x7f8118100e70 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f810ffff640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 0x7f81181987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81177fe640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 0x7f8118198290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108017030 con 0x7f8118100e70 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8108004020 con 0x7f8118100e70 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81181a0150 con 0x7f8118100e70 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080047a0 con 0x7f8118100e70 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.840+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81181a06a0 con 0x7f8118100e70 2024-04-22T07:12:50.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.841+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81181a0960 con 0x7f8118100e70 2024-04-22T07:12:50.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.842+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f81080041d0 con 0x7f8118100e70 2024-04-22T07:12:50.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.843+0000 7f81157fa640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f81040a3bb0 0x7f81040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:50.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.843+0000 7f810ffff640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f81040a3bb0 0x7f81040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:50.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.843+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81080cf9f0 con 0x7f8118100e70 2024-04-22T07:12:50.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.843+0000 7f810ffff640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f81040a3bb0 0x7f81040a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8118199750 tx=0x7f8100005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:50.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:50.848+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f810809c870 con 0x7f8118100e70 2024-04-22T07:12:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:50 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] up:boot 2024-04-22T07:12:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:50 smithi133 ceph-mon[31198]: fsmap 12 up:standby 2024-04-22T07:12:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:12:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:50 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:51.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:50 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] up:boot 2024-04-22T07:12:51.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:50 smithi154 ceph-mon[31862]: fsmap 12 up:standby 2024-04-22T07:12:51.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:12:51.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:50 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:51.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.077+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81181a0b30 con 0x7f8118100e70 2024-04-22T07:12:51.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.078+0000 7f81157fa640 1 -- 172.21.15.113:0/2547323773 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7f810809c170 con 0x7f8118100e70 2024-04-22T07:12:51.076 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:51.076 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24445,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24446,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-22T07:12:51.076 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f81040a3bb0 msgr2=0x7f81040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f81040a3bb0 0x7f81040a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8118199750 tx=0x7f8100005e50 comp rx=0 tx=0).stop 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 msgr2=0x7f8118198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 0x7f8118198290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800de10 comp rx=0 tx=0).stop 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 shutdown_connections 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f81040a3bb0 0x7f81040a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8118102980 0x7f811819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8118101fc0 0x7f81181987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 --2- 172.21.15.113:0/2547323773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8118100e70 0x7f8118198290 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 >> 172.21.15.113:0/2547323773 conn(0x7f81180fc3e0 msgr2=0x7f81180fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 shutdown_connections 2024-04-22T07:12:51.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.080+0000 7f811dfb6640 1 -- 172.21.15.113:0/2547323773 wait complete. 2024-04-22T07:12:51.090 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:12:51.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 -- 172.21.15.113:0/3910263286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 msgr2=0x7f5de01035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 --2- 172.21.15.113:0/3910263286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01035f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009900 tx=0x7f5dd402bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 -- 172.21.15.113:0/3910263286 shutdown_connections 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 --2- 172.21.15.113:0/3910263286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0103b30 0x7f5de010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 --2- 172.21.15.113:0/3910263286 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01035f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 --2- 172.21.15.113:0/3910263286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de0101f80 0x7f5de01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 -- 172.21.15.113:0/3910263286 >> 172.21.15.113:0/3910263286 conn(0x7f5de00fd530 msgr2=0x7f5de00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 -- 172.21.15.113:0/3910263286 shutdown_connections 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.196+0000 7f5de633d640 1 -- 172.21.15.113:0/3910263286 wait complete. 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 Processor -- start 2024-04-22T07:12:51.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 -- start start 2024-04-22T07:12:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0101f80 0x7f5de0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de0103b30 0x7f5de019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010dbe0 con 0x7f5de0103170 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010da60 con 0x7f5de0101f80 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5de633d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010dd60 con 0x7f5de0103b30 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5ddf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.197+0000 7f5ddf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52322/0 (socket says 172.21.15.113:52322) 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 -- 172.21.15.113:0/87180928 learned_addr learned my addr 172.21.15.113:0/87180928 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddffff640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0101f80 0x7f5de0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5de48da640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de0103b30 0x7f5de019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 -- 172.21.15.113:0/87180928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de0103b30 msgr2=0x7f5de019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de0103b30 0x7f5de019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 -- 172.21.15.113:0/87180928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0101f80 msgr2=0x7f5de0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0101f80 0x7f5de0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 -- 172.21.15.113:0/87180928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de019fea0 con 0x7f5de0103170 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddf7fe640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01987c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005ab0 tx=0x7f5dd4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4041070 con 0x7f5de0103170 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de01a0130 con 0x7f5de0103170 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5dd40052f0 con 0x7f5de0103170 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4003dc0 con 0x7f5de0103170 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.198+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01a0680 con 0x7f5de0103170 2024-04-22T07:12:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.199+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de0109320 con 0x7f5de0103170 2024-04-22T07:12:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.201+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5dd4044070 con 0x7f5de0103170 2024-04-22T07:12:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.202+0000 7f5ddd7fa640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dc80a3bb0 0x7f5dc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.202+0000 7f5ddffff640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dc80a3bb0 0x7f5dc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.202+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5dd40f1d90 con 0x7f5de0103170 2024-04-22T07:12:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.202+0000 7f5ddffff640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dc80a3bb0 0x7f5dc80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5de0102fd0 tx=0x7f5dd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:51.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.205+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dd40be520 con 0x7f5de0103170 2024-04-22T07:12:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:50 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] up:boot 2024-04-22T07:12:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:50 smithi113 ceph-mon[27933]: fsmap 12 up:standby 2024-04-22T07:12:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:12:51.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:50 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:51.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.412+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5de002d810 con 0x7f5de0103170 2024-04-22T07:12:51.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.413+0000 7f5ddd7fa640 1 -- 172.21.15.113:0/87180928 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18504 (secure 0 0 0) 0x7f5dd40c2820 con 0x7f5de0103170 2024-04-22T07:12:51.411 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:51.412 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:06.002736+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:12:51.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.415+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dc80a3bb0 msgr2=0x7f5dc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.415+0000 7f5de633d640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dc80a3bb0 0x7f5dc80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5de0102fd0 tx=0x7f5dd0005f90 comp rx=0 tx=0).stop 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.415+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 msgr2=0x7f5de01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.415+0000 7f5de633d640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01987c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005ab0 tx=0x7f5dd4005e60 comp rx=0 tx=0).stop 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 shutdown_connections 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dc80a3bb0 0x7f5dc80a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de0103b30 0x7f5de019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de0103170 0x7f5de01987c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 --2- 172.21.15.113:0/87180928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de0101f80 0x7f5de0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 >> 172.21.15.113:0/87180928 conn(0x7f5de00fd530 msgr2=0x7f5de01064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:51.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 shutdown_connections 2024-04-22T07:12:51.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.416+0000 7f5de633d640 1 -- 172.21.15.113:0/87180928 wait complete. 2024-04-22T07:12:51.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:12:51.425 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:12:51.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/4118962436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 msgr2=0x7fbfd810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/4118962436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd810a3e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009ca0 tx=0x7fbfd002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/4118962436 shutdown_connections 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/4118962436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd810a3e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/4118962436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8103190 0x7fbfd8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/4118962436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8101fa0 0x7fbfd81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.529+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/4118962436 >> 172.21.15.113:0/4118962436 conn(0x7fbfd80fd530 msgr2=0x7fbfd80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/4118962436 shutdown_connections 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/4118962436 wait complete. 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 Processor -- start 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 -- start start 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8101fa0 0x7fbfd81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8103190 0x7fbfd8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810dc00 con 0x7fbfd8103b50 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810da80 con 0x7fbfd8101fa0 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdf7c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810dd80 con 0x7fbfd8103190 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfddd64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfddd64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52340/0 (socket says 172.21.15.113:52340) 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfddd64640 1 -- 172.21.15.113:0/3308408250 learned_addr learned my addr 172.21.15.113:0/3308408250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:51.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.530+0000 7fbfdd563640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8101fa0 0x7fbfd81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfdcd62640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8103190 0x7fbfd8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfddd64640 1 -- 172.21.15.113:0/3308408250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8103190 msgr2=0x7fbfd8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfddd64640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8103190 0x7fbfd8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfddd64640 1 -- 172.21.15.113:0/3308408250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8101fa0 msgr2=0x7fbfd81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfddd64640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8101fa0 0x7fbfd81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfddd64640 1 -- 172.21.15.113:0/3308408250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd819fe00 con 0x7fbfd8103b50 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfdd563640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8101fa0 0x7fbfd81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:12:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfdcd62640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8103190 0x7fbfd8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfddd64640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd819f790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0005cc0 tx=0x7fbfd0032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0041020 con 0x7fbfd8103b50 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbfd0003e40 con 0x7fbfd8103b50 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd81a0090 con 0x7fbfd8103b50 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0039040 con 0x7fbfd8103b50 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.531+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81a0550 con 0x7fbfd8103b50 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.532+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd81092c0 con 0x7fbfd8103b50 2024-04-22T07:12:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.533+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbfd0044020 con 0x7fbfd8103b50 2024-04-22T07:12:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.533+0000 7fbfce7fc640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfc40a3bb0 0x7fbfc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.534+0000 7fbfdd563640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfc40a3bb0 0x7fbfc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.534+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbfd00f2570 con 0x7fbfd8103b50 2024-04-22T07:12:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.534+0000 7fbfdd563640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfc40a3bb0 0x7fbfc40a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40058c0 tx=0x7fbfd4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.538+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfd004b020 con 0x7fbfd8103b50 2024-04-22T07:12:51.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.717+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfd8062b50 con 0x7fbfd8103b50 2024-04-22T07:12:51.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.717+0000 7fbfce7fc640 1 -- 172.21.15.113:0/3308408250 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fbfd00bf3f0 con 0x7fbfd8103b50 2024-04-22T07:12:51.715 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:12:51.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfc40a3bb0 msgr2=0x7fbfc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfc40a3bb0 0x7fbfc40a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40058c0 tx=0x7fbfd4005850 comp rx=0 tx=0).stop 2024-04-22T07:12:51.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 msgr2=0x7fbfd819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd819f790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0005cc0 tx=0x7fbfd0032950 comp rx=0 tx=0).stop 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 shutdown_connections 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbfc40a3bb0 0x7fbfc40a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfd8103b50 0x7fbfd819f790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd8103190 0x7fbfd8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 --2- 172.21.15.113:0/3308408250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd8101fa0 0x7fbfd81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 >> 172.21.15.113:0/3308408250 conn(0x7fbfd80fd530 msgr2=0x7fbfd8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 shutdown_connections 2024-04-22T07:12:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.719+0000 7fbfdf7c7640 1 -- 172.21.15.113:0/3308408250 wait complete. 2024-04-22T07:12:51.728 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-22T07:12:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.826+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/572361636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 msgr2=0x7f5fe4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.826+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/572361636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe4104e80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/572361636 shutdown_connections 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/572361636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4107cf0 0x7f5fe410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/572361636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe41053c0 0x7f5fe41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/572361636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe4104e80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/572361636 >> 172.21.15.113:0/572361636 conn(0x7f5fe40fc400 msgr2=0x7f5fe40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/572361636 shutdown_connections 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/572361636 wait complete. 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 Processor -- start 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- start start 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe41053c0 0x7f5fe41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4107cf0 0x7f5fe419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410dc20 con 0x7f5fe4102a90 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410daa0 con 0x7f5fe41053c0 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.827+0000 7f5fe8dbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe410dda0 con 0x7f5fe4107cf0 2024-04-22T07:12:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe41982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52366/0 (socket says 172.21.15.113:52366) 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 -- 172.21.15.113:0/892979879 learned_addr learned my addr 172.21.15.113:0/892979879 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe1d9b640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe41053c0 0x7f5fe41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe2d9d640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4107cf0 0x7f5fe419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 -- 172.21.15.113:0/892979879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4107cf0 msgr2=0x7f5fe419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4107cf0 0x7f5fe419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 -- 172.21.15.113:0/892979879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe41053c0 msgr2=0x7f5fe41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe41053c0 0x7f5fe41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 -- 172.21.15.113:0/892979879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe419fec0 con 0x7f5fe4102a90 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe259c640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe41982a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005ab0 tx=0x7f5fdc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc041070 con 0x7f5fe4102a90 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe41a0150 con 0x7f5fe4102a90 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5fdc0052f0 con 0x7f5fe4102a90 2024-04-22T07:12:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc003d20 con 0x7f5fe4102a90 2024-04-22T07:12:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.828+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41a06a0 con 0x7f5fe4102a90 2024-04-22T07:12:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.829+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe40fdef0 con 0x7f5fe4102a90 2024-04-22T07:12:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.830+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5fdc044070 con 0x7f5fe4102a90 2024-04-22T07:12:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.831+0000 7f5fd37fe640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5fcc0a3bb0 0x7f5fcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.831+0000 7f5fe1d9b640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5fcc0a3bb0 0x7f5fcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.831+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5fdc0f19b0 con 0x7f5fe4102a90 2024-04-22T07:12:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.831+0000 7f5fe1d9b640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5fcc0a3bb0 0x7f5fcc0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5fe41997b0 tx=0x7f5fd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:51.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:51.835+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fdc0be1e0 con 0x7f5fe4102a90 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.017+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5fe4062b50 con 0x7f5fe4102a90 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.017+0000 7f5fd37fe640 1 -- 172.21.15.113:0/892979879 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f5fdc0c24e0 con 0x7f5fe4102a90 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-04-22T07:12:52.015 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:12:52.016 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5fcc0a3bb0 msgr2=0x7f5fcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5fcc0a3bb0 0x7f5fcc0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5fe41997b0 tx=0x7f5fd8005f90 comp rx=0 tx=0).stop 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 msgr2=0x7f5fe41982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe41982a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005ab0 tx=0x7f5fdc005dd0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 shutdown_connections 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5fcc0a3bb0 0x7f5fcc0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fe4107cf0 0x7f5fe419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe41053c0 0x7f5fe41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 --2- 172.21.15.113:0/892979879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fe4102a90 0x7f5fe41982a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 >> 172.21.15.113:0/892979879 conn(0x7f5fe40fc400 msgr2=0x7f5fe4103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 shutdown_connections 2024-04-22T07:12:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.019+0000 7f5fe8dbd640 1 -- 172.21.15.113:0/892979879 wait complete. 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2547323773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/87180928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.027 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.028 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:51 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3308408250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:52.028 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-22T07:12:52.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-22T07:12:52.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 -- 172.21.15.113:0/809800296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8101fe0 msgr2=0x7fc2c8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:52.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 --2- 172.21.15.113:0/809800296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8101fe0 0x7fc2c8102460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009900 tx=0x7fc2bc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 -- 172.21.15.113:0/809800296 shutdown_connections 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 --2- 172.21.15.113:0/809800296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c81029a0 0x7fc2c81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 --2- 172.21.15.113:0/809800296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8101fe0 0x7fc2c8102460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 --2- 172.21.15.113:0/809800296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c8100e90 0x7fc2c81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 -- 172.21.15.113:0/809800296 >> 172.21.15.113:0/809800296 conn(0x7fc2c80fc400 msgr2=0x7fc2c80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 -- 172.21.15.113:0/809800296 shutdown_connections 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 -- 172.21.15.113:0/809800296 wait complete. 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 Processor -- start 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.136+0000 7fc2cea26640 1 -- start start 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 0x7fc2c8069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c8101fe0 0x7fc2c819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c81029a0 0x7fc2c819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c810cb60 con 0x7fc2c8100e90 2024-04-22T07:12:52.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c810c9e0 con 0x7fc2c81029a0 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c810cce0 con 0x7fc2c8101fe0 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 0x7fc2c8069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 0x7fc2c8069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52372/0 (socket says 172.21.15.113:52372) 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 -- 172.21.15.113:0/35777202 learned_addr learned my addr 172.21.15.113:0/35777202 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c77fe640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c8101fe0 0x7fc2c819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 -- 172.21.15.113:0/35777202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c8101fe0 msgr2=0x7fc2c819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c8101fe0 0x7fc2c819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 -- 172.21.15.113:0/35777202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c81029a0 msgr2=0x7fc2c819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c81029a0 0x7fc2c819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 -- 172.21.15.113:0/35777202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c819ff50 con 0x7fc2c8100e90 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c7fff640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 0x7fc2c8069c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800d950 tx=0x7fc2b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8017030 con 0x7fc2c8100e90 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c81a01e0 con 0x7fc2c8100e90 2024-04-22T07:12:52.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc2b8004020 con 0x7fc2c8100e90 2024-04-22T07:12:52.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80047a0 con 0x7fc2c8100e90 2024-04-22T07:12:52.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.137+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c81a0750 con 0x7fc2c8100e90 2024-04-22T07:12:52.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.138+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c8068d70 con 0x7fc2c8100e90 2024-04-22T07:12:52.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.139+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc2b80041d0 con 0x7fc2c8100e90 2024-04-22T07:12:52.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.140+0000 7fc2c57fa640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc2b00a3bb0 0x7fc2b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:52.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.140+0000 7fc2c77fe640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc2b00a3bb0 0x7fc2b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:52.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.140+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2b80cf9f0 con 0x7fc2c8100e90 2024-04-22T07:12:52.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.140+0000 7fc2c77fe640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc2b00a3bb0 0x7fc2b00a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc02c240 tx=0x7fc2bc02c850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:52.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.145+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2b809c870 con 0x7fc2c8100e90 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2547323773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/87180928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:51 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3308408250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:52.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:52.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2547323773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:12:52.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/87180928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:51 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3308408250' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:12:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:52.330+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc2c802d660 con 0x7fc2c8100e90 2024-04-22T07:12:53.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.123+0000 7fc2c57fa640 1 -- 172.21.15.113:0/35777202 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fc2c802d660 con 0x7fc2c8100e90 2024-04-22T07:12:53.121 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-04-22T07:12:53.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc2b00a3bb0 msgr2=0x7fc2b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:53.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc2b00a3bb0 0x7fc2b00a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc02c240 tx=0x7fc2bc02c850 comp rx=0 tx=0).stop 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 msgr2=0x7fc2c8069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 0x7fc2c8069c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800d950 tx=0x7fc2b800de10 comp rx=0 tx=0).stop 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 shutdown_connections 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc2b00a3bb0 0x7fc2b00a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c81029a0 0x7fc2c819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2c8101fe0 0x7fc2c819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 --2- 172.21.15.113:0/35777202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c8100e90 0x7fc2c8069c50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 >> 172.21.15.113:0/35777202 conn(0x7fc2c80fc400 msgr2=0x7fc2c8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 shutdown_connections 2024-04-22T07:12:53.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.126+0000 7fc2cea26640 1 -- 172.21.15.113:0/35777202 wait complete. 2024-04-22T07:12:53.138 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-22T07:12:53.160 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/892979879' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T07:12:53.160 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:53.160 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:53.161 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:53.161 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:52 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:53.161 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/35777202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T07:12:53.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.249+0000 7fc54d775640 1 -- 172.21.15.113:0/138657851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 msgr2=0x7fc54810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:53.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.249+0000 7fc54d775640 1 --2- 172.21.15.113:0/138657851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54810a470 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc538009cd0 tx=0x7fc53802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- 172.21.15.113:0/138657851 shutdown_connections 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 --2- 172.21.15.113:0/138657851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54810a470 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 --2- 172.21.15.113:0/138657851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc548103220 0x7fc5481036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 --2- 172.21.15.113:0/138657851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 0x7fc548100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- 172.21.15.113:0/138657851 >> 172.21.15.113:0/138657851 conn(0x7fc5480fbb80 msgr2=0x7fc5480fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- 172.21.15.113:0/138657851 shutdown_connections 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- 172.21.15.113:0/138657851 wait complete. 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 Processor -- start 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- start start 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 0x7fc548198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc548103220 0x7fc5481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:53.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810dd10 con 0x7fc548103be0 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810db90 con 0x7fc548100250 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.250+0000 7fc54d775640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc54810de90 con 0x7fc548103220 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54819fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52390/0 (socket says 172.21.15.113:52390) 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 -- 172.21.15.113:0/2913827627 learned_addr learned my addr 172.21.15.113:0/2913827627 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5467fc640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc548103220 0x7fc5481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc546ffd640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 0x7fc548198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 -- 172.21.15.113:0/2913827627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc548103220 msgr2=0x7fc5481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc548103220 0x7fc5481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 -- 172.21.15.113:0/2913827627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 msgr2=0x7fc548198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 0x7fc548198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 -- 172.21.15.113:0/2913827627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5481a0090 con 0x7fc548103be0 2024-04-22T07:12:53.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc546ffd640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 0x7fc548198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:53.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc5477fe640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54819fa20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc538037040 tx=0x7fc538032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:53.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538041020 con 0x7fc548103be0 2024-04-22T07:12:53.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc538003e40 con 0x7fc548103be0 2024-04-22T07:12:53.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481a0320 con 0x7fc548103be0 2024-04-22T07:12:53.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538039040 con 0x7fc548103be0 2024-04-22T07:12:53.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.251+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a0890 con 0x7fc548103be0 2024-04-22T07:12:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.252+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc548109440 con 0x7fc548103be0 2024-04-22T07:12:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.254+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc538044020 con 0x7fc548103be0 2024-04-22T07:12:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.255+0000 7fc53ffff640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc5340a3bb0 0x7fc5340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.255+0000 7fc546ffd640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc5340a3bb0 0x7fc5340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.255+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fc5380f2aa0 con 0x7fc548103be0 2024-04-22T07:12:53.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.255+0000 7fc546ffd640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc5340a3bb0 0x7fc5340a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:53.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.258+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc53804b020 con 0x7fc548103be0 2024-04-22T07:12:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/892979879' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T07:12:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:52 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/35777202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T07:12:53.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/892979879' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T07:12:53.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:53.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:52 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:12:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/35777202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T07:12:53.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:53.439+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc54802d5c0 con 0x7fc548103be0 2024-04-22T07:12:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.125+0000 7fc53ffff640 1 -- 172.21.15.113:0/2913827627 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fc5380bf7b0 con 0x7fc548103be0 2024-04-22T07:12:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc5340a3bb0 msgr2=0x7fc5340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc5340a3bb0 0x7fc5340a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540008040 comp rx=0 tx=0).stop 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 msgr2=0x7fc54819fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54819fa20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc538037040 tx=0x7fc538032950 comp rx=0 tx=0).stop 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 shutdown_connections 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc5340a3bb0 0x7fc5340a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc548103be0 0x7fc54819fa20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc548103220 0x7fc5481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 --2- 172.21.15.113:0/2913827627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc548100250 0x7fc548198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.128+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 >> 172.21.15.113:0/2913827627 conn(0x7fc5480fbb80 msgr2=0x7fc548106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.129+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 shutdown_connections 2024-04-22T07:12:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.129+0000 7fc54d775640 1 -- 172.21.15.113:0/2913827627 wait complete. 2024-04-22T07:12:54.146 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 -- 172.21.15.113:0/3104256688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c100820 msgr2=0x7fe00c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 --2- 172.21.15.113:0/3104256688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c100820 0x7fe00c100c40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe000009900 tx=0x7fe00002bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 -- 172.21.15.113:0/3104256688 shutdown_connections 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 --2- 172.21.15.113:0/3104256688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c101b20 0x7fe00c101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 --2- 172.21.15.113:0/3104256688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe00c101180 0x7fe00c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 --2- 172.21.15.113:0/3104256688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c100820 0x7fe00c100c40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 -- 172.21.15.113:0/3104256688 >> 172.21.15.113:0/3104256688 conn(0x7fe00c0fc440 msgr2=0x7fe00c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 -- 172.21.15.113:0/3104256688 shutdown_connections 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.290+0000 7fe010a81640 1 -- 172.21.15.113:0/3104256688 wait complete. 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 Processor -- start 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 -- start start 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe00c100820 0x7fe00c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 0x7fe00c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c101b20 0x7fe00c1a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10fe80 con 0x7fe00c101180 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10fd00 con 0x7fe00c100820 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c110000 con 0x7fe00c101b20 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 0x7fe00c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 0x7fe00c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52408/0 (socket says 172.21.15.113:52408) 2024-04-22T07:12:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 -- 172.21.15.113:0/1386223828 learned_addr learned my addr 172.21.15.113:0/1386223828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe00a59c640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe00c100820 0x7fe00c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe00ad9d640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c101b20 0x7fe00c1a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 -- 172.21.15.113:0/1386223828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c101b20 msgr2=0x7fe00c1a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c101b20 0x7fe00c1a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 -- 172.21.15.113:0/1386223828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe00c100820 msgr2=0x7fe00c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe00c100820 0x7fe00c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 -- 172.21.15.113:0/1386223828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c1a2170 con 0x7fe00c101180 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe009d9b640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 0x7fe00c19aa90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b4a0 tx=0x7fdffc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc014070 con 0x7fe00c101180 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdffc010ae0 con 0x7fe00c101180 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c1a2460 con 0x7fe00c101180 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc017ce0 con 0x7fe00c101180 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.291+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1a2a30 con 0x7fe00c101180 2024-04-22T07:12:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.292+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c10b570 con 0x7fe00c101180 2024-04-22T07:12:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.293+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdffc017070 con 0x7fe00c101180 2024-04-22T07:12:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.294+0000 7fdffb7fe640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff40a3bb0 0x7fdff40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.294+0000 7fe00a59c640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff40a3bb0 0x7fdff40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.294+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fdffc0cf940 con 0x7fe00c101180 2024-04-22T07:12:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.294+0000 7fe00a59c640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff40a3bb0 0x7fdff40a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe000006010 tx=0x7fe000005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.298+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdffc09c4d0 con 0x7fe00c101180 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/35777202' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: osdmap e83: 12 total, 12 up, 12 in 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2913827627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.487 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.487 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.487 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:54.487 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:54.487 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:54.513+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe00c02d5c0 con 0x7fe00c101180 2024-04-22T07:12:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/35777202' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T07:12:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: osdmap e83: 12 total, 12 up, 12 in 2024-04-22T07:12:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2913827627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T07:12:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/35777202' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: osdmap e83: 12 total, 12 up, 12 in 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2913827627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:54.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:12:54.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:12:54.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:12:55.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.153+0000 7fdffb7fe640 1 -- 172.21.15.113:0/1386223828 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fdffc09bdd0 con 0x7fe00c101180 2024-04-22T07:12:55.151 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 2 and data pool 3 2024-04-22T07:12:55.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff40a3bb0 msgr2=0x7fdff40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff40a3bb0 0x7fdff40a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe000006010 tx=0x7fe000005b50 comp rx=0 tx=0).stop 2024-04-22T07:12:55.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 msgr2=0x7fe00c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 0x7fe00c19aa90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b4a0 tx=0x7fdffc00b960 comp rx=0 tx=0).stop 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 shutdown_connections 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdff40a3bb0 0x7fdff40a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe00c101b20 0x7fe00c1a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe00c101180 0x7fe00c19aa90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 --2- 172.21.15.113:0/1386223828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe00c100820 0x7fe00c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.158+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 >> 172.21.15.113:0/1386223828 conn(0x7fe00c0fc440 msgr2=0x7fe00c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.159+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 shutdown_connections 2024-04-22T07:12:55.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.159+0000 7fe010a81640 1 -- 172.21.15.113:0/1386223828 wait complete. 2024-04-22T07:12:55.177 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 -- 172.21.15.113:0/1447976936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654100e70 msgr2=0x7fe654101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 --2- 172.21.15.113:0/1447976936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654100e70 0x7fe654101290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009900 tx=0x7fe64c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 -- 172.21.15.113:0/1447976936 shutdown_connections 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 --2- 172.21.15.113:0/1447976936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe654102a10 0x7fe654109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 --2- 172.21.15.113:0/1447976936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe654102050 0x7fe6541024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 --2- 172.21.15.113:0/1447976936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654100e70 0x7fe654101290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 -- 172.21.15.113:0/1447976936 >> 172.21.15.113:0/1447976936 conn(0x7fe6540fc470 msgr2=0x7fe6540fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 -- 172.21.15.113:0/1447976936 shutdown_connections 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.279+0000 7fe658806640 1 -- 172.21.15.113:0/1447976936 wait complete. 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 Processor -- start 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 -- start start 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe654100e70 0x7fe65406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe654102050 0x7fe65419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 0x7fe65419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe65410cb90 con 0x7fe654102a10 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe65410ca10 con 0x7fe654102050 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe658806640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe65410cd10 con 0x7fe654100e70 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 0x7fe65419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 0x7fe65419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52430/0 (socket says 172.21.15.113:52430) 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 -- 172.21.15.113:0/1457674313 learned_addr learned my addr 172.21.15.113:0/1457674313 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe651d9b640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe654102050 0x7fe65419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe65259c640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe654100e70 0x7fe65406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 -- 172.21.15.113:0/1457674313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe654100e70 msgr2=0x7fe65406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe654100e70 0x7fe65406b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 -- 172.21.15.113:0/1457674313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe654102050 msgr2=0x7fe65419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe654102050 0x7fe65419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 -- 172.21.15.113:0/1457674313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65419feb0 con 0x7fe654102a10 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.280+0000 7fe652d9d640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 0x7fe65419f5f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe644009fd0 tx=0x7fe64400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.281+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644017070 con 0x7fe654102a10 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.281+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe644003e40 con 0x7fe654102a10 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.281+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6541a01a0 con 0x7fe654102a10 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.281+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644004820 con 0x7fe654102a10 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.281+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541a06a0 con 0x7fe654102a10 2024-04-22T07:12:55.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.282+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6541081b0 con 0x7fe654102a10 2024-04-22T07:12:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.283+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe644003970 con 0x7fe654102a10 2024-04-22T07:12:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.283+0000 7fe6437fe640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe63c0a3bb0 0x7fe63c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.283+0000 7fe65259c640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe63c0a3bb0 0x7fe63c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.283+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe6440d0070 con 0x7fe654102a10 2024-04-22T07:12:55.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.284+0000 7fe65259c640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe63c0a3bb0 0x7fe63c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe64c02c240 tx=0x7fe64c02c850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:55.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.288+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe644003c60 con 0x7fe654102a10 2024-04-22T07:12:55.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.466+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe65402d7d0 con 0x7fe654102a10 2024-04-22T07:12:55.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.467+0000 7fe6437fe640 1 -- 172.21.15.113:0/1457674313 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22280 (secure 0 0 0) 0x7fe64409c550 con 0x7fe654102a10 2024-04-22T07:12:55.465 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:55.465 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:54.514884+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:12:55.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe63c0a3bb0 msgr2=0x7fe63c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe63c0a3bb0 0x7fe63c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe64c02c240 tx=0x7fe64c02c850 comp rx=0 tx=0).stop 2024-04-22T07:12:55.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 msgr2=0x7fe65419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 0x7fe65419f5f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe644009fd0 tx=0x7fe64400b260 comp rx=0 tx=0).stop 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 shutdown_connections 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe63c0a3bb0 0x7fe63c0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe654102a10 0x7fe65419f5f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe654102050 0x7fe65419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 --2- 172.21.15.113:0/1457674313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe654100e70 0x7fe65406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 >> 172.21.15.113:0/1457674313 conn(0x7fe6540fc470 msgr2=0x7fe654105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 shutdown_connections 2024-04-22T07:12:55.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.469+0000 7fe658806640 1 -- 172.21.15.113:0/1457674313 wait complete. 2024-04-22T07:12:55.477 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:55 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2913827627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T07:12:55.477 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:55 smithi113 ceph-mon[27933]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T07:12:55.477 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:55 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1386223828' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T07:12:55.477 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:55 smithi113 ceph-mon[27933]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T07:12:55.478 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:12:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:55 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2913827627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T07:12:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:55 smithi133 ceph-mon[31198]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T07:12:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:55 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1386223828' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T07:12:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:55 smithi133 ceph-mon[31198]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T07:12:55.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:55 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2913827627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T07:12:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:55 smithi154 ceph-mon[31862]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T07:12:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:55 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1386223828' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T07:12:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:55 smithi154 ceph-mon[31862]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T07:12:55.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 -- 172.21.15.113:0/1960457413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420101f90 msgr2=0x7f84201023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 --2- 172.21.15.113:0/1960457413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420101f90 0x7f84201023b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f841c009900 tx=0x7f841c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 -- 172.21.15.113:0/1960457413 shutdown_connections 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 --2- 172.21.15.113:0/1960457413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420103b40 0x7f842010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 --2- 172.21.15.113:0/1960457413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8420103180 0x7f8420103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 --2- 172.21.15.113:0/1960457413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420101f90 0x7f84201023b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 -- 172.21.15.113:0/1960457413 >> 172.21.15.113:0/1960457413 conn(0x7f84200fd500 msgr2=0x7f84200ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 -- 172.21.15.113:0/1960457413 shutdown_connections 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.578+0000 7f84277ce640 1 -- 172.21.15.113:0/1960457413 wait complete. 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 Processor -- start 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 -- start start 2024-04-22T07:12:55.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420101f90 0x7f8420198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 0x7f8420198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8420103b40 0x7f842019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f842010dc70 con 0x7f8420103180 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f842010daf0 con 0x7f8420101f90 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f84277ce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f842010ddf0 con 0x7f8420103b40 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 0x7f8420198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 0x7f8420198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52462/0 (socket says 172.21.15.113:52462) 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 -- 172.21.15.113:0/1381805017 learned_addr learned my addr 172.21.15.113:0/1381805017 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8425d6b640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8420103b40 0x7f842019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f842556a640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420101f90 0x7f8420198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 -- 172.21.15.113:0/1381805017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8420103b40 msgr2=0x7f842019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8420103b40 0x7f842019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 -- 172.21.15.113:0/1381805017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420101f90 msgr2=0x7f8420198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420101f90 0x7f8420198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f8424d69640 1 -- 172.21.15.113:0/1381805017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842019ff40 con 0x7f8420103180 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.579+0000 7f842556a640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420101f90 0x7f8420198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.580+0000 7f8424d69640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 0x7f8420198860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f841000e9f0 tx=0x7f841000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.580+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841000cd80 con 0x7f8420103180 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.580+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a0230 con 0x7f8420103180 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.580+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f84100054f0 con 0x7f8420103180 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.580+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410010690 con 0x7f8420103180 2024-04-22T07:12:55.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.580+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a07b0 con 0x7f8420103180 2024-04-22T07:12:55.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.581+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84201092b0 con 0x7f8420103180 2024-04-22T07:12:55.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.582+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f84100046c0 con 0x7f8420103180 2024-04-22T07:12:55.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.582+0000 7f84167fc640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f840c0a3bb0 0x7f840c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.583+0000 7f842556a640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f840c0a3bb0 0x7f840c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.583+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f84100d09a0 con 0x7f8420103180 2024-04-22T07:12:55.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.583+0000 7f842556a640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f840c0a3bb0 0x7f840c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f841c0096c0 tx=0x7f841c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:55.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.586+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8410053ff0 con 0x7f8420103180 2024-04-22T07:12:55.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.767+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f842002d810 con 0x7f8420103180 2024-04-22T07:12:55.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.768+0000 7f84167fc640 1 -- 172.21.15.113:0/1381805017 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22280 (secure 0 0 0) 0x7f841001f340 con 0x7f8420103180 2024-04-22T07:12:55.766 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:12:55.766 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:54.514884+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:12:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f840c0a3bb0 msgr2=0x7f840c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f840c0a3bb0 0x7f840c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f841c0096c0 tx=0x7f841c000bc0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 msgr2=0x7f8420198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 0x7f8420198860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f841000e9f0 tx=0x7f841000edf0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 shutdown_connections 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f840c0a3bb0 0x7f840c0a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8420103b40 0x7f842019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8420103180 0x7f8420198860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 --2- 172.21.15.113:0/1381805017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8420101f90 0x7f8420198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 >> 172.21.15.113:0/1381805017 conn(0x7f84200fd500 msgr2=0x7f8420106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.770+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 shutdown_connections 2024-04-22T07:12:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.771+0000 7f84277ce640 1 -- 172.21.15.113:0/1381805017 wait complete. 2024-04-22T07:12:55.779 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-22T07:12:55.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 -- 172.21.15.113:0/2127801424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc107c70 msgr2=0x7f90dc10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/2127801424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc107c70 0x7f90dc10a0c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009ca0 tx=0x7f90d402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 -- 172.21.15.113:0/2127801424 shutdown_connections 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/2127801424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc107c70 0x7f90dc10a0c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/2127801424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90dc105340 0x7f90dc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/2127801424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90dc102a10 0x7f90dc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 -- 172.21.15.113:0/2127801424 >> 172.21.15.113:0/2127801424 conn(0x7f90dc0fc400 msgr2=0x7f90dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 -- 172.21.15.113:0/2127801424 shutdown_connections 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.882+0000 7f90e1ca5640 1 -- 172.21.15.113:0/2127801424 wait complete. 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 Processor -- start 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 -- start start 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 0x7f90dc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90dc105340 0x7f90dc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90dc107c70 0x7f90dc19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc10dcf0 con 0x7f90dc102a10 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc10db70 con 0x7f90dc107c70 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90e1ca5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc10de70 con 0x7f90dc105340 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 0x7f90dc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 0x7f90dc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43010/0 (socket says 172.21.15.113:43010) 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90db7fe640 1 -- 172.21.15.113:0/556913917 learned_addr learned my addr 172.21.15.113:0/556913917 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90dbfff640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90dc107c70 0x7f90dc19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.883+0000 7f90daffd640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90dc105340 0x7f90dc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90db7fe640 1 -- 172.21.15.113:0/556913917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90dc105340 msgr2=0x7f90dc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90db7fe640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90dc105340 0x7f90dc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90db7fe640 1 -- 172.21.15.113:0/556913917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90dc107c70 msgr2=0x7f90dc19f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90db7fe640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90dc107c70 0x7f90dc19f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:55.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90db7fe640 1 -- 172.21.15.113:0/556913917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc1a0010 con 0x7f90dc102a10 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90db7fe640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 0x7f90dc1983f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b4a0 tx=0x7f90cc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc015020 con 0x7f90dc102a10 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f90cc011d00 con 0x7f90dc102a10 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00f2e0 con 0x7f90dc102a10 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc1a0300 con 0x7f90dc102a10 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.884+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc1a08d0 con 0x7f90dc102a10 2024-04-22T07:12:55.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.885+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc0fdef0 con 0x7f90dc102a10 2024-04-22T07:12:55.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.886+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f90cc018020 con 0x7f90dc102a10 2024-04-22T07:12:55.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.887+0000 7f90d8ff9640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f90c40a3bb0 0x7f90c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:55.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.887+0000 7f90daffd640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f90c40a3bb0 0x7f90c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:55.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.887+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f90cc0cfad0 con 0x7f90dc102a10 2024-04-22T07:12:55.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.887+0000 7f90daffd640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f90c40a3bb0 0x7f90c40a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90dc199900 tx=0x7f90d0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:55.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:55.892+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90cc01f020 con 0x7f90dc102a10 2024-04-22T07:12:56.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.107+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f90dc02d5c0 con 0x7f90dc102a10 2024-04-22T07:12:56.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.159+0000 7f90d8ff9640 1 -- 172.21.15.113:0/556913917 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f90cc09c600 con 0x7f90dc102a10 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f90c40a3bb0 msgr2=0x7f90c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f90c40a3bb0 0x7f90c40a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90dc199900 tx=0x7f90d0009110 comp rx=0 tx=0).stop 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 msgr2=0x7f90dc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 0x7f90dc1983f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b4a0 tx=0x7f90cc00b8a0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 shutdown_connections 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f90c40a3bb0 0x7f90c40a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90dc107c70 0x7f90dc19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90dc105340 0x7f90dc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 --2- 172.21.15.113:0/556913917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90dc102a10 0x7f90dc1983f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 >> 172.21.15.113:0/556913917 conn(0x7f90dc0fc400 msgr2=0x7f90dc103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 shutdown_connections 2024-04-22T07:12:56.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.162+0000 7f90e1ca5640 1 -- 172.21.15.113:0/556913917 wait complete. 2024-04-22T07:12:56.171 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-22T07:12:56.171 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-22T07:12:56.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 -- 172.21.15.113:0/3572268740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96201053c0 msgr2=0x7f96201077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:56.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 --2- 172.21.15.113:0/3572268740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96201053c0 0x7f96201077b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9610009900 tx=0x7f961002bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:56.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 -- 172.21.15.113:0/3572268740 shutdown_connections 2024-04-22T07:12:56.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 --2- 172.21.15.113:0/3572268740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620107cf0 0x7f962010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 --2- 172.21.15.113:0/3572268740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96201053c0 0x7f96201077b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 --2- 172.21.15.113:0/3572268740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9620102a90 0x7f9620104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 -- 172.21.15.113:0/3572268740 >> 172.21.15.113:0/3572268740 conn(0x7f96200fc400 msgr2=0x7f96200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 -- 172.21.15.113:0/3572268740 shutdown_connections 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.273+0000 7f9627c8e640 1 -- 172.21.15.113:0/3572268740 wait complete. 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 Processor -- start 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 -- start start 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 0x7f9620102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96201053c0 0x7f9620100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620107cf0 0x7f9620100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f962010dbe0 con 0x7f9620102a90 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f962010da60 con 0x7f96201053c0 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9627c8e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f962010dd60 con 0x7f9620107cf0 2024-04-22T07:12:56.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9625a2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 0x7f9620102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9625a2a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 0x7f9620102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43028/0 (socket says 172.21.15.113:43028) 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9625a2a640 1 -- 172.21.15.113:0/451192337 learned_addr learned my addr 172.21.15.113:0/451192337 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f962622b640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620107cf0 0x7f9620100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.274+0000 7f9625229640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96201053c0 0x7f9620100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625a2a640 1 -- 172.21.15.113:0/451192337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620107cf0 msgr2=0x7f9620100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625a2a640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620107cf0 0x7f9620100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625a2a640 1 -- 172.21.15.113:0/451192337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96201053c0 msgr2=0x7f9620100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625a2a640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96201053c0 0x7f9620100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625a2a640 1 -- 172.21.15.113:0/451192337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9620101810 con 0x7f9620102a90 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625229640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96201053c0 0x7f9620100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:12:56.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9625a2a640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 0x7f9620102400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f961c00d950 tx=0x7f961c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c017070 con 0x7f9620102a90 2024-04-22T07:12:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f961c009c30 con 0x7f9620102a90 2024-04-22T07:12:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96201a23d0 con 0x7f9620102a90 2024-04-22T07:12:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c00f6b0 con 0x7f9620102a90 2024-04-22T07:12:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.275+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96201a28f0 con 0x7f9620102a90 2024-04-22T07:12:56.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.276+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96200fdef0 con 0x7f9620102a90 2024-04-22T07:12:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.277+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f961c0109c0 con 0x7f9620102a90 2024-04-22T07:12:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.277+0000 7f9616ffd640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f960c0a3bb0 0x7f960c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.277+0000 7f9625229640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f960c0a3bb0 0x7f960c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:56.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.278+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f961c0cfef0 con 0x7f9620102a90 2024-04-22T07:12:56.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.278+0000 7f9625229640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f960c0a3bb0 0x7f960c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9610006010 tx=0x7f961002c950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:56.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.281+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f961c009de0 con 0x7f9620102a90 2024-04-22T07:12:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:56.497+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f962002d770 con 0x7f9620102a90 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: pgmap v284: 97 pgs: 10 active+clean, 36 creating+peering, 51 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1386223828' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1457674313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1381805017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/556913917' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T07:12:56.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: pgmap v284: 97 pgs: 10 active+clean, 36 creating+peering, 51 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1386223828' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1457674313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1381805017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/556913917' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: pgmap v284: 97 pgs: 10 active+clean, 36 creating+peering, 51 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1386223828' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1457674313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1381805017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:12:56.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/556913917' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T07:12:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.161+0000 7f9616ffd640 1 -- 172.21.15.113:0/451192337 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f961c09ca20 con 0x7f9620102a90 2024-04-22T07:12:57.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.163+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f960c0a3bb0 msgr2=0x7f960c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:57.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.163+0000 7f9627c8e640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f960c0a3bb0 0x7f960c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9610006010 tx=0x7f961002c950 comp rx=0 tx=0).stop 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.163+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 msgr2=0x7f9620102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.163+0000 7f9627c8e640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 0x7f9620102400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f961c00d950 tx=0x7f961c00de10 comp rx=0 tx=0).stop 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 shutdown_connections 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f960c0a3bb0 0x7f960c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9620107cf0 0x7f9620100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96201053c0 0x7f9620100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 --2- 172.21.15.113:0/451192337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9620102a90 0x7f9620102400 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 >> 172.21.15.113:0/451192337 conn(0x7f96200fc400 msgr2=0x7f96201034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 shutdown_connections 2024-04-22T07:12:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.164+0000 7f9627c8e640 1 -- 172.21.15.113:0/451192337 wait complete. 2024-04-22T07:12:57.177 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-22T07:12:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 -- 172.21.15.113:0/1931848818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1348100880 msgr2=0x7f1348100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 --2- 172.21.15.113:0/1931848818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1348100880 0x7f1348100ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1330009900 tx=0x7f133002bcd0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 -- 172.21.15.113:0/1931848818 shutdown_connections 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 --2- 172.21.15.113:0/1931848818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1348101b80 0x7f1348102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 --2- 172.21.15.113:0/1931848818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13481011e0 0x7f1348101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 --2- 172.21.15.113:0/1931848818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1348100880 0x7f1348100ca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.284+0000 7f134ce8e640 1 -- 172.21.15.113:0/1931848818 >> 172.21.15.113:0/1931848818 conn(0x7f13480fc400 msgr2=0x7f13480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 -- 172.21.15.113:0/1931848818 shutdown_connections 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 -- 172.21.15.113:0/1931848818 wait complete. 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 Processor -- start 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 -- start start 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1348100880 0x7f134819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 0x7f134819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1348101b80 0x7f13481a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f134810fed0 con 0x7f13481011e0 2024-04-22T07:12:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f134810fd50 con 0x7f1348100880 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f134ce8e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1348110050 con 0x7f1348101b80 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.285+0000 7f1345d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 0x7f134819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 0x7f134819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43040/0 (socket says 172.21.15.113:43040) 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 -- 172.21.15.113:0/2626166943 learned_addr learned my addr 172.21.15.113:0/2626166943 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1346d9d640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1348101b80 0x7f13481a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f134659c640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1348100880 0x7f134819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 -- 172.21.15.113:0/2626166943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1348101b80 msgr2=0x7f13481a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1348101b80 0x7f13481a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 -- 172.21.15.113:0/2626166943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1348100880 msgr2=0x7f134819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1348100880 0x7f134819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 -- 172.21.15.113:0/2626166943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13481a2250 con 0x7f13481011e0 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f1345d9b640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 0x7f134819ab70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f133800b4a0 tx=0x7f133800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338014070 con 0x7f13481011e0 2024-04-22T07:12:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13481a2540 con 0x7f13481011e0 2024-04-22T07:12:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1338010ae0 con 0x7f13481011e0 2024-04-22T07:12:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338017ce0 con 0x7f13481011e0 2024-04-22T07:12:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.286+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13481a2b60 con 0x7f13481011e0 2024-04-22T07:12:57.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.287+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134810b860 con 0x7f13481011e0 2024-04-22T07:12:57.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.288+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1338017070 con 0x7f13481011e0 2024-04-22T07:12:57.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.289+0000 7f133f7fe640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13340a3bb0 0x7f13340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:57.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.289+0000 7f134659c640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13340a3bb0 0x7f13340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:57.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.289+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f13380cfa20 con 0x7f13481011e0 2024-04-22T07:12:57.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.289+0000 7f134659c640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13340a3bb0 0x7f13340a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f133002c7b0 tx=0x7f1330000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:57.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.294+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f133809c550 con 0x7f13481011e0 2024-04-22T07:12:57.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:57.509+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f134802d5c0 con 0x7f13481011e0 2024-04-22T07:12:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:57 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/556913917' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T07:12:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:57 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] up:active 2024-04-22T07:12:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:57 smithi133 ceph-mon[31198]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T07:12:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:57 smithi133 ceph-mon[31198]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T07:12:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:57 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/451192337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-22T07:12:57.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:57 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/556913917' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T07:12:57.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:57 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] up:active 2024-04-22T07:12:57.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:57 smithi154 ceph-mon[31862]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T07:12:57.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:57 smithi154 ceph-mon[31862]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T07:12:57.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:57 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/451192337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-22T07:12:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:57 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/556913917' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T07:12:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:57 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] up:active 2024-04-22T07:12:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:57 smithi113 ceph-mon[27933]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T07:12:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:57 smithi113 ceph-mon[27933]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T07:12:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:57 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/451192337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-22T07:12:58.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.168+0000 7f133f7fe640 1 -- 172.21.15.113:0/2626166943 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f133809be50 con 0x7f13481011e0 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13340a3bb0 msgr2=0x7f13340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13340a3bb0 0x7f13340a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f133002c7b0 tx=0x7f1330000bc0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 msgr2=0x7f134819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 0x7f134819ab70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f133800b4a0 tx=0x7f133800b960 comp rx=0 tx=0).stop 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 shutdown_connections 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f13340a3bb0 0x7f13340a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1348101b80 0x7f13481a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13481011e0 0x7f134819ab70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 --2- 172.21.15.113:0/2626166943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1348100880 0x7f134819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 >> 172.21.15.113:0/2626166943 conn(0x7f13480fc400 msgr2=0x7f13480fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:58.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 shutdown_connections 2024-04-22T07:12:58.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.170+0000 7f134ce8e640 1 -- 172.21.15.113:0/2626166943 wait complete. 2024-04-22T07:12:58.184 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-22T07:12:58.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.291+0000 7f2f73255640 1 -- 172.21.15.113:0/114487215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 msgr2=0x7f2f6c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:58.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.291+0000 7f2f73255640 1 --2- 172.21.15.113:0/114487215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c1012b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009900 tx=0x7f2f5c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:58.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.291+0000 7f2f73255640 1 -- 172.21.15.113:0/114487215 shutdown_connections 2024-04-22T07:12:58.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.291+0000 7f2f73255640 1 --2- 172.21.15.113:0/114487215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6c1029a0 0x7f2f6c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.291+0000 7f2f73255640 1 --2- 172.21.15.113:0/114487215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f6c101fe0 0x7f2f6c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.291+0000 7f2f73255640 1 --2- 172.21.15.113:0/114487215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c1012b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- 172.21.15.113:0/114487215 >> 172.21.15.113:0/114487215 conn(0x7f2f6c0fc400 msgr2=0x7f2f6c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:58.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- 172.21.15.113:0/114487215 shutdown_connections 2024-04-22T07:12:58.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- 172.21.15.113:0/114487215 wait complete. 2024-04-22T07:12:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 Processor -- start 2024-04-22T07:12:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- start start 2024-04-22T07:12:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6c101fe0 0x7f2f6c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f6c1029a0 0x7f2f6c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:58.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10cba0 con 0x7f2f6c100e90 2024-04-22T07:12:58.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10ca20 con 0x7f2f6c101fe0 2024-04-22T07:12:58.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f73255640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c10cd20 con 0x7f2f6c1029a0 2024-04-22T07:12:58.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f70ff1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:58.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f70ff1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43050/0 (socket says 172.21.15.113:43050) 2024-04-22T07:12:58.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f70ff1640 1 -- 172.21.15.113:0/1000891446 learned_addr learned my addr 172.21.15.113:0/1000891446 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:58.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.292+0000 7f2f6bfff640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6c101fe0 0x7f2f6c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:58.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f70ff1640 1 -- 172.21.15.113:0/1000891446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f6c1029a0 msgr2=0x7f2f6c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:58.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f70ff1640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f6c1029a0 0x7f2f6c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f70ff1640 1 -- 172.21.15.113:0/1000891446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6c101fe0 msgr2=0x7f2f6c19d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f70ff1640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6c101fe0 0x7f2f6c19d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f70ff1640 1 -- 172.21.15.113:0/1000891446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c1a0000 con 0x7f2f6c100e90 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f70ff1640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c069c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005ab0 tx=0x7f2f5c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c041070 con 0x7f2f6c100e90 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2f5c0052f0 con 0x7f2f6c100e90 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c1a0230 con 0x7f2f6c100e90 2024-04-22T07:12:58.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c003d50 con 0x7f2f6c100e90 2024-04-22T07:12:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.293+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c1a0850 con 0x7f2f6c100e90 2024-04-22T07:12:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.294+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c1081c0 con 0x7f2f6c100e90 2024-04-22T07:12:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.295+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2f5c044070 con 0x7f2f6c100e90 2024-04-22T07:12:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.296+0000 7f2f69ffb640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2f540a3c30 0x7f2f540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.296+0000 7f2f6bfff640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2f540a3c30 0x7f2f540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.296+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2f5c0f1ff0 con 0x7f2f6c100e90 2024-04-22T07:12:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.296+0000 7f2f6bfff640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2f540a3c30 0x7f2f540a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0699c0 tx=0x7f2f60009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.299+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f5c0beb20 con 0x7f2f6c100e90 2024-04-22T07:12:58.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:58.521+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2f6c02d7e0 con 0x7f2f6c100e90 2024-04-22T07:12:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: pgmap v287: 97 pgs: 32 active+clean, 36 creating+peering, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T07:12:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T07:12:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: Cluster is now healthy 2024-04-22T07:12:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/451192337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-22T07:12:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T07:12:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T07:12:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-22T07:12:58.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T07:12:58.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:58 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2626166943' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T07:12:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: pgmap v287: 97 pgs: 32 active+clean, 36 creating+peering, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T07:12:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T07:12:58.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: Cluster is now healthy 2024-04-22T07:12:58.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/451192337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-22T07:12:58.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T07:12:58.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T07:12:58.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-22T07:12:58.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T07:12:58.568 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:58 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2626166943' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T07:12:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: pgmap v287: 97 pgs: 32 active+clean, 36 creating+peering, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:12:58.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T07:12:58.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T07:12:58.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: Cluster is now healthy 2024-04-22T07:12:58.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/451192337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-22T07:12:58.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T07:12:58.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T07:12:58.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-22T07:12:58.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T07:12:58.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:58 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2626166943' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T07:12:59.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.343+0000 7f2f69ffb640 1 -- 172.21.15.113:0/1000891446 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f2f5c0be420 con 0x7f2f6c100e90 2024-04-22T07:12:59.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2f540a3c30 msgr2=0x7f2f540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2f540a3c30 0x7f2f540a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0699c0 tx=0x7f2f60009110 comp rx=0 tx=0).stop 2024-04-22T07:12:59.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 msgr2=0x7f2f6c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c069c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005ab0 tx=0x7f2f5c005e60 comp rx=0 tx=0).stop 2024-04-22T07:12:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 shutdown_connections 2024-04-22T07:12:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2f540a3c30 0x7f2f540a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f6c1029a0 0x7f2f6c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6c101fe0 0x7f2f6c19d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 --2- 172.21.15.113:0/1000891446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f6c100e90 0x7f2f6c069c50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 >> 172.21.15.113:0/1000891446 conn(0x7f2f6c0fc400 msgr2=0x7f2f6c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:59.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 shutdown_connections 2024-04-22T07:12:59.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.348+0000 7f2f73255640 1 -- 172.21.15.113:0/1000891446 wait complete. 2024-04-22T07:12:59.359 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-22T07:12:59.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-22T07:12:59.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 -- 172.21.15.113:0/1074718787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c100e60 msgr2=0x7f6a0c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 --2- 172.21.15.113:0/1074718787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c100e60 0x7f6a0c101280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009900 tx=0x7f69fc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:59.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 -- 172.21.15.113:0/1074718787 shutdown_connections 2024-04-22T07:12:59.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 --2- 172.21.15.113:0/1074718787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0c102a00 0x7f6a0c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 --2- 172.21.15.113:0/1074718787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a0c102040 0x7f6a0c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 --2- 172.21.15.113:0/1074718787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c100e60 0x7f6a0c101280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 -- 172.21.15.113:0/1074718787 >> 172.21.15.113:0/1074718787 conn(0x7f6a0c0fc420 msgr2=0x7f6a0c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 -- 172.21.15.113:0/1074718787 shutdown_connections 2024-04-22T07:12:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.469+0000 7f6a128b3640 1 -- 172.21.15.113:0/1074718787 wait complete. 2024-04-22T07:12:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 Processor -- start 2024-04-22T07:12:59.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 -- start start 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a0c100e60 0x7f6a0c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 0x7f6a0c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0c102a00 0x7f6a0c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10cb20 con 0x7f6a0c102040 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10c9a0 con 0x7f6a0c100e60 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a128b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10cca0 con 0x7f6a0c102a00 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a0b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 0x7f6a0c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a0b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 0x7f6a0c1976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43072/0 (socket says 172.21.15.113:43072) 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a0b7fe640 1 -- 172.21.15.113:0/3788455501 learned_addr learned my addr 172.21.15.113:0/3788455501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a0bfff640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a0c100e60 0x7f6a0c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.470+0000 7f6a10e50640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0c102a00 0x7f6a0c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a0b7fe640 1 -- 172.21.15.113:0/3788455501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0c102a00 msgr2=0x7f6a0c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a0b7fe640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0c102a00 0x7f6a0c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a0b7fe640 1 -- 172.21.15.113:0/3788455501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a0c100e60 msgr2=0x7f6a0c1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a0b7fe640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a0c100e60 0x7f6a0c1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a0b7fe640 1 -- 172.21.15.113:0/3788455501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19edc0 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a0b7fe640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 0x7f6a0c1976e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000e9f0 tx=0x7f6a0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000cd80 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6a00010900 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19f0b0 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00018920 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.471+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c19f600 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.472+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c108200 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.473+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6a00010ab0 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.473+0000 7f6a097fa640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f69f40a3bb0 0x7f69f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.474+0000 7f6a0bfff640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f69f40a3bb0 0x7f69f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.474+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6a000d0b50 con 0x7f6a0c102040 2024-04-22T07:12:59.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.474+0000 7f6a0bfff640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f69f40a3bb0 0x7f69f40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009230 tx=0x7f69fc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:59.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.477+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a0009d680 con 0x7f6a0c102040 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2626166943' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] up:active 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: Cluster is now healthy 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T07:12:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] up:active 2024-04-22T07:12:59.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:12:59.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1000891446' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T07:12:59.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2626166943' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] up:active 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: Cluster is now healthy 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] up:active 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:12:59.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:12:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1000891446' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T07:12:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2626166943' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T07:12:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] up:active 2024-04-22T07:12:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T07:12:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T07:12:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: Cluster is now healthy 2024-04-22T07:12:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-22T07:12:59.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-22T07:12:59.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T07:12:59.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] up:active 2024-04-22T07:12:59.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:12:59.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1000891446' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T07:12:59.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.682+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6a0c062390 con 0x7f69f40a3bb0 2024-04-22T07:12:59.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.701+0000 7f6a097fa640 1 -- 172.21.15.113:0/3788455501 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a0c062390 con 0x7f69f40a3bb0 2024-04-22T07:12:59.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f69f40a3bb0 msgr2=0x7f69f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f69f40a3bb0 0x7f69f40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009230 tx=0x7f69fc005fd0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 msgr2=0x7f6a0c1976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 0x7f6a0c1976e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000e9f0 tx=0x7f6a0000eeb0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 shutdown_connections 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f69f40a3bb0 0x7f69f40a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0c102a00 0x7f6a0c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a0c102040 0x7f6a0c1976e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 --2- 172.21.15.113:0/3788455501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a0c100e60 0x7f6a0c1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 >> 172.21.15.113:0/3788455501 conn(0x7f6a0c0fc420 msgr2=0x7f6a0c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 shutdown_connections 2024-04-22T07:12:59.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.703+0000 7f6a128b3640 1 -- 172.21.15.113:0/3788455501 wait complete. 2024-04-22T07:12:59.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-22T07:12:59.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 -- 172.21.15.113:0/828237900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 msgr2=0x7fdab0104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 --2- 172.21.15.113:0/828237900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0104ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009900 tx=0x7fdaa002bc30 comp rx=0 tx=0).stop 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 -- 172.21.15.113:0/828237900 shutdown_connections 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 --2- 172.21.15.113:0/828237900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdab0107d10 0x7fdab010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 --2- 172.21.15.113:0/828237900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab01053e0 0x7fdab01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 --2- 172.21.15.113:0/828237900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0104ea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 -- 172.21.15.113:0/828237900 >> 172.21.15.113:0/828237900 conn(0x7fdab00fc400 msgr2=0x7fdab00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.816+0000 7fdab616d640 1 -- 172.21.15.113:0/828237900 shutdown_connections 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 -- 172.21.15.113:0/828237900 wait complete. 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 Processor -- start 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 -- start start 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdab01053e0 0x7fdab0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab0107d10 0x7fdab019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010dc10 con 0x7fdab0102ab0 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010da90 con 0x7fdab01053e0 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdab616d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdab010dd90 con 0x7fdab0107d10 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdaaf7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43114/0 (socket says 172.21.15.113:43114) 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.817+0000 7fdaaf7fe640 1 -- 172.21.15.113:0/3383111589 learned_addr learned my addr 172.21.15.113:0/3383111589 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaeffd640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdab01053e0 0x7fdab0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaf7fe640 1 -- 172.21.15.113:0/3383111589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab0107d10 msgr2=0x7fdab019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaf7fe640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab0107d10 0x7fdab019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaf7fe640 1 -- 172.21.15.113:0/3383111589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdab01053e0 msgr2=0x7fdab0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaf7fe640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdab01053e0 0x7fdab0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaf7fe640 1 -- 172.21.15.113:0/3383111589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab019ff40 con 0x7fdab0102ab0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaaf7fe640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0198320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0005ab0 tx=0x7fdaa0005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0041070 con 0x7fdab0102ab0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdaa00052f0 con 0x7fdab0102ab0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01a01d0 con 0x7fdab0102ab0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0003dc0 con 0x7fdab0102ab0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.818+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a0720 con 0x7fdab0102ab0 2024-04-22T07:12:59.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.819+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab00fdef0 con 0x7fdab0102ab0 2024-04-22T07:12:59.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.820+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdaa0044070 con 0x7fdab0102ab0 2024-04-22T07:12:59.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.821+0000 7fdaacff9640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3bb0 0x7fda980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:12:59.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.821+0000 7fdaaeffd640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3bb0 0x7fda980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:12:59.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.821+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdaa00f1ef0 con 0x7fdab0102ab0 2024-04-22T07:12:59.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.821+0000 7fdaaeffd640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3bb0 0x7fda980a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdab0199830 tx=0x7fdaa400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:12:59.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:12:59.825+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdaa00bea20 con 0x7fdab0102ab0 2024-04-22T07:13:00.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.030+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdab0062390 con 0x7fda980a3bb0 2024-04-22T07:13:00.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.054+0000 7fdaacff9640 1 -- 172.21.15.113:0/3383111589 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdab0062390 con 0x7fda980a3bb0 2024-04-22T07:13:00.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3bb0 msgr2=0x7fda980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3bb0 0x7fda980a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdab0199830 tx=0x7fdaa400b390 comp rx=0 tx=0).stop 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 msgr2=0x7fdab0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0198320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0005ab0 tx=0x7fdaa0005e60 comp rx=0 tx=0).stop 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 shutdown_connections 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3bb0 0x7fda980a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab0107d10 0x7fdab019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdab01053e0 0x7fdab0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 --2- 172.21.15.113:0/3383111589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdab0102ab0 0x7fdab0198320 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 >> 172.21.15.113:0/3383111589 conn(0x7fdab00fc400 msgr2=0x7fdab01034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 shutdown_connections 2024-04-22T07:13:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.057+0000 7fdab616d640 1 -- 172.21.15.113:0/3383111589 wait complete. 2024-04-22T07:13:00.064 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:12:59.687+0000 7f1d9dcb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:13:00.064 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:12:59.687+0000 7f1d9dcb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:13:00.064 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:12:59.687+0000 7f1d9dcb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:13:00.064 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:12:59.687+0000 7f1d9dcb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:13:00.064 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:12:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:12:59.687+0000 7f1d9dcb6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:13:00.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-22T07:13:00.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 -- 172.21.15.113:0/650646420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf001029a0 msgr2=0x7fdf001092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 --2- 172.21.15.113:0/650646420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf001029a0 0x7fdf001092c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009ca0 tx=0x7fdef802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 -- 172.21.15.113:0/650646420 shutdown_connections 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 --2- 172.21.15.113:0/650646420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf001029a0 0x7fdf001092c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 --2- 172.21.15.113:0/650646420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf00101fe0 0x7fdf00102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 --2- 172.21.15.113:0/650646420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00100e90 0x7fdf001012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 -- 172.21.15.113:0/650646420 >> 172.21.15.113:0/650646420 conn(0x7fdf000fc400 msgr2=0x7fdf000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 -- 172.21.15.113:0/650646420 shutdown_connections 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.171+0000 7fdf06081640 1 -- 172.21.15.113:0/650646420 wait complete. 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 Processor -- start 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 -- start start 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 0x7fdf00198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00101fe0 0x7fdf00198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf001029a0 0x7fdf0019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0010cb10 con 0x7fdf00100e90 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0010c990 con 0x7fdf00101fe0 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdf06081640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0010cc90 con 0x7fdf001029a0 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 0x7fdf00198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 0x7fdf00198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43122/0 (socket says 172.21.15.113:43122) 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 -- 172.21.15.113:0/661283817 learned_addr learned my addr 172.21.15.113:0/661283817 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdefeffd640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00101fe0 0x7fdf00198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdefffff640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf001029a0 0x7fdf0019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 -- 172.21.15.113:0/661283817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf001029a0 msgr2=0x7fdf0019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf001029a0 0x7fdf0019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 -- 172.21.15.113:0/661283817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00101fe0 msgr2=0x7fdf00198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00101fe0 0x7fdf00198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 -- 172.21.15.113:0/661283817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0019fe70 con 0x7fdf00100e90 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdefeffd640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00101fe0 0x7fdf00198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.172+0000 7fdeff7fe640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 0x7fdf00198250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdef000e9f0 tx=0x7fdef000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:00.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.173+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef000cd80 con 0x7fdf00100e90 2024-04-22T07:13:00.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.173+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001a0160 con 0x7fdf00100e90 2024-04-22T07:13:00.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.173+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdef0010900 con 0x7fdf00100e90 2024-04-22T07:13:00.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.173+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00188d0 con 0x7fdf00100e90 2024-04-22T07:13:00.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.173+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf001a0700 con 0x7fdf00100e90 2024-04-22T07:13:00.174 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:12:59.899+0000 7f5d572bd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T07:13:00.174 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:12:59.899+0000 7f5d572bd700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-22T07:13:00.175 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:12:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:12:59.899+0000 7f5d572bd700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-22T07:13:00.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.174+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf001a0960 con 0x7fdf00100e90 2024-04-22T07:13:00.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.175+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdef0010ab0 con 0x7fdf00100e90 2024-04-22T07:13:00.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.175+0000 7fdefcff9640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdee80a3bb0 0x7fdee80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.176+0000 7fdefeffd640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdee80a3bb0 0x7fdee80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.176+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdef00d0630 con 0x7fdf00100e90 2024-04-22T07:13:00.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.176+0000 7fdefeffd640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdee80a3bb0 0x7fdee80a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf00199760 tx=0x7fdef400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.182+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdef009cb10 con 0x7fdf00100e90 2024-04-22T07:13:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:00 smithi113 ceph-mon[27933]: pgmap v288: 97 pgs: 73 active+clean, 16 creating+peering, 8 unknown; 580 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-04-22T07:13:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1000891446' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T07:13:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:00 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:13:00.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.388+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fdf0006acc0 con 0x7fdee80a3bb0 2024-04-22T07:13:00.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.394+0000 7fdefcff9640 1 -- 172.21.15.113:0/661283817 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdf0006acc0 con 0x7fdee80a3bb0 2024-04-22T07:13:00.392 INFO:teuthology.orchestra.run.smithi113.stdout:/volumes/qa/sv_0/6d87d3bd-6926-418d-bc82-91f2b2da0daa 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdee80a3bb0 msgr2=0x7fdee80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdee80a3bb0 0x7fdee80a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf00199760 tx=0x7fdef400b410 comp rx=0 tx=0).stop 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 msgr2=0x7fdf00198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 0x7fdf00198250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdef000e9f0 tx=0x7fdef000eeb0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 shutdown_connections 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdee80a3bb0 0x7fdee80a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf001029a0 0x7fdf0019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf00101fe0 0x7fdf00198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 --2- 172.21.15.113:0/661283817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf00100e90 0x7fdf00198250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 >> 172.21.15.113:0/661283817 conn(0x7fdf000fc400 msgr2=0x7fdf0006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:00.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 shutdown_connections 2024-04-22T07:13:00.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.396+0000 7fdf06081640 1 -- 172.21.15.113:0/661283817 wait complete. 2024-04-22T07:13:00.405 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 -- 172.21.15.113:0/1306567091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103230 msgr2=0x7f70941036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 --2- 172.21.15.113:0/1306567091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103230 0x7f70941036b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f707c009930 tx=0x7f707c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 -- 172.21.15.113:0/1306567091 shutdown_connections 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 --2- 172.21.15.113:0/1306567091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103bf0 0x7f709410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 --2- 172.21.15.113:0/1306567091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103230 0x7f70941036b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 --2- 172.21.15.113:0/1306567091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7094100260 0x7f7094100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 -- 172.21.15.113:0/1306567091 >> 172.21.15.113:0/1306567091 conn(0x7f70940fbb90 msgr2=0x7f70940fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:00.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 -- 172.21.15.113:0/1306567091 shutdown_connections 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.509+0000 7f70997bd640 1 -- 172.21.15.113:0/1306567091 wait complete. 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 Processor -- start 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 -- start start 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7094100260 0x7f7094198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 0x7f7094198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103bf0 0x7f709419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f709410dc20 con 0x7f7094103bf0 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709410daa0 con 0x7f7094103230 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70997bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f709410dda0 con 0x7f7094100260 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f70937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103bf0 0x7f709419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 0x7f7094198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 0x7f7094198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57180/0 (socket says 172.21.15.113:57180) 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 -- 172.21.15.113:0/229744208 learned_addr learned my addr 172.21.15.113:0/229744208 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f7092ffd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7094100260 0x7f7094198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 -- 172.21.15.113:0/229744208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7094100260 msgr2=0x7f7094198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7094100260 0x7f7094198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 -- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103bf0 msgr2=0x7f709419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103bf0 0x7f709419f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.510+0000 7f708bfff640 1 -- 172.21.15.113:0/229744208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709419ff40 con 0x7f7094103230 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.511+0000 7f708bfff640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 0x7f7094198860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f707c009430 tx=0x7f707c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.511+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c043020 con 0x7f7094103230 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.511+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70941a01d0 con 0x7f7094103230 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.511+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70941a0770 con 0x7f7094103230 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.511+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f707c0043d0 con 0x7f7094103230 2024-04-22T07:13:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.511+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c003b80 con 0x7f7094103230 2024-04-22T07:13:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.512+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70740052c0 con 0x7f7094103230 2024-04-22T07:13:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.513+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f707c046020 con 0x7f7094103230 2024-04-22T07:13:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.513+0000 7f7090ff9640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f70800a3bb0 0x7f70800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.513+0000 7f7092ffd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f70800a3bb0 0x7f70800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.514+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f707c0f1ba0 con 0x7f7094103230 2024-04-22T07:13:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.514+0000 7f7092ffd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f70800a3bb0 0x7f70800a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70840058c0 tx=0x7f708400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:00.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.518+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f707c0be630 con 0x7f7094103230 2024-04-22T07:13:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:00 smithi133 ceph-mon[31198]: pgmap v288: 97 pgs: 73 active+clean, 16 creating+peering, 8 unknown; 580 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-04-22T07:13:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:00 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1000891446' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T07:13:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:00 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:13:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:00 smithi154 ceph-mon[31862]: pgmap v288: 97 pgs: 73 active+clean, 16 creating+peering, 8 unknown; 580 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-04-22T07:13:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:00 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1000891446' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T07:13:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:00 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:13:00.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.720+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7074000bf0 con 0x7f70800a3bb0 2024-04-22T07:13:00.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.745+0000 7f7090ff9640 1 -- 172.21.15.113:0/229744208 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7074000bf0 con 0x7f70800a3bb0 2024-04-22T07:13:00.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f70800a3bb0 msgr2=0x7f70800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f70800a3bb0 0x7f70800a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70840058c0 tx=0x7f708400a2d0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 msgr2=0x7f7094198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 0x7f7094198860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f707c009430 tx=0x7f707c005e40 comp rx=0 tx=0).stop 2024-04-22T07:13:00.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 shutdown_connections 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f70800a3bb0 0x7f70800a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7094103bf0 0x7f709419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7094103230 0x7f7094198860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 --2- 172.21.15.113:0/229744208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7094100260 0x7f7094198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 >> 172.21.15.113:0/229744208 conn(0x7f70940fbb90 msgr2=0x7f7094106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 shutdown_connections 2024-04-22T07:13:00.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.747+0000 7f70997bd640 1 -- 172.21.15.113:0/229744208 wait complete. 2024-04-22T07:13:00.756 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.861+0000 7f12c3da4640 1 -- 172.21.15.113:0/468496435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc100e70 msgr2=0x7f12bc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.861+0000 7f12c3da4640 1 --2- 172.21.15.113:0/468496435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc100e70 0x7f12bc101290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009900 tx=0x7f12b802bc30 comp rx=0 tx=0).stop 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- 172.21.15.113:0/468496435 shutdown_connections 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 --2- 172.21.15.113:0/468496435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc102980 0x7f12bc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 --2- 172.21.15.113:0/468496435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 0x7f12bc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 --2- 172.21.15.113:0/468496435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc100e70 0x7f12bc101290 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- 172.21.15.113:0/468496435 >> 172.21.15.113:0/468496435 conn(0x7f12bc0fc3e0 msgr2=0x7f12bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- 172.21.15.113:0/468496435 shutdown_connections 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- 172.21.15.113:0/468496435 wait complete. 2024-04-22T07:13:00.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 Processor -- start 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- start start 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc100e70 0x7f12bc069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 0x7f12bc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 0x7f12bc19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc10cae0 con 0x7f12bc102980 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc10c960 con 0x7f12bc101fc0 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c3da4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc10cc60 con 0x7f12bc100e70 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c2341640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 0x7f12bc19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c2341640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 0x7f12bc19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43164/0 (socket says 172.21.15.113:43164) 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.862+0000 7f12c2341640 1 -- 172.21.15.113:0/2892371494 learned_addr learned my addr 172.21.15.113:0/2892371494 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c133f640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 0x7f12bc19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c1b40640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc100e70 0x7f12bc069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c2341640 1 -- 172.21.15.113:0/2892371494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc100e70 msgr2=0x7f12bc069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c2341640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc100e70 0x7f12bc069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c2341640 1 -- 172.21.15.113:0/2892371494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 msgr2=0x7f12bc19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c2341640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 0x7f12bc19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c2341640 1 -- 172.21.15.113:0/2892371494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc19fe60 con 0x7f12bc102980 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c1b40640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc100e70 0x7f12bc069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c133f640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 0x7f12bc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c2341640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 0x7f12bc19f5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f12b400eaa0 tx=0x7f12b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b400cdb0 con 0x7f12bc102980 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc1a0150 con 0x7f12bc102980 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f12b4005020 con 0x7f12bc102980 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b40116d0 con 0x7f12bc102980 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.863+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc1a06a0 con 0x7f12bc102980 2024-04-22T07:13:00.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.864+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc108220 con 0x7f12bc102980 2024-04-22T07:13:00.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.865+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f12b40051d0 con 0x7f12bc102980 2024-04-22T07:13:00.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.866+0000 7f12b2ffd640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f12a80a3bb0 0x7f12a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:00.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.866+0000 7f12c1b40640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f12a80a3bb0 0x7f12a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:00.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.866+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12b40d0980 con 0x7f12bc102980 2024-04-22T07:13:00.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.866+0000 7f12c1b40640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f12a80a3bb0 0x7f12a80a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12b80096c0 tx=0x7f12b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:00.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:00.869+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12b4016030 con 0x7f12bc102980 2024-04-22T07:13:01.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.077+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f12bc062390 con 0x7f12a80a3bb0 2024-04-22T07:13:01.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.082+0000 7f12b2ffd640 1 -- 172.21.15.113:0/2892371494 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f12bc062390 con 0x7f12a80a3bb0 2024-04-22T07:13:01.080 INFO:teuthology.orchestra.run.smithi113.stdout:/volumes/qa/sv_1/9a5b95cc-e4a5-4f02-a5dd-c51dfbfd1ae4 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f12a80a3bb0 msgr2=0x7f12a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f12a80a3bb0 0x7f12a80a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12b80096c0 tx=0x7f12b8000bc0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 msgr2=0x7f12bc19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 0x7f12bc19f5a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f12b400eaa0 tx=0x7f12b400ef60 comp rx=0 tx=0).stop 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 shutdown_connections 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f12a80a3bb0 0x7f12a80a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12bc102980 0x7f12bc19f5a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12bc101fc0 0x7f12bc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 --2- 172.21.15.113:0/2892371494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12bc100e70 0x7f12bc069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 >> 172.21.15.113:0/2892371494 conn(0x7f12bc0fc3e0 msgr2=0x7f12bc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:01.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 shutdown_connections 2024-04-22T07:13:01.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.084+0000 7f12c3da4640 1 -- 172.21.15.113:0/2892371494 wait complete. 2024-04-22T07:13:01.093 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 -- 172.21.15.113:0/4220058025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164101f90 msgr2=0x7f01641023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 --2- 172.21.15.113:0/4220058025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164101f90 0x7f01641023b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0160009900 tx=0x7f016002bc30 comp rx=0 tx=0).stop 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 -- 172.21.15.113:0/4220058025 shutdown_connections 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 --2- 172.21.15.113:0/4220058025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0164103b40 0x7f016410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 --2- 172.21.15.113:0/4220058025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0164103180 0x7f0164103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 --2- 172.21.15.113:0/4220058025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164101f90 0x7f01641023b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 -- 172.21.15.113:0/4220058025 >> 172.21.15.113:0/4220058025 conn(0x7f01640fd500 msgr2=0x7f01640ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 -- 172.21.15.113:0/4220058025 shutdown_connections 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.198+0000 7f016c515640 1 -- 172.21.15.113:0/4220058025 wait complete. 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 Processor -- start 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 -- start start 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0164101f90 0x7f0164198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 0x7f0164198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0164103b40 0x7f016419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f016410dc70 con 0x7f0164103180 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f016410daf0 con 0x7f0164103b40 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016c515640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f016410ddf0 con 0x7f0164101f90 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 0x7f0164198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 0x7f0164198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43174/0 (socket says 172.21.15.113:43174) 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 -- 172.21.15.113:0/3520669087 learned_addr learned my addr 172.21.15.113:0/3520669087 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:01.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016aab2640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0164103b40 0x7f016419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f016a2b1640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0164101f90 0x7f0164198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 -- 172.21.15.113:0/3520669087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0164101f90 msgr2=0x7f0164198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0164101f90 0x7f0164198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 -- 172.21.15.113:0/3520669087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0164103b40 msgr2=0x7f016419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0164103b40 0x7f016419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 -- 172.21.15.113:0/3520669087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016419ff30 con 0x7f0164103180 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.199+0000 7f0169ab0640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 0x7f0164198850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f015400b4a0 tx=0x7f015400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.200+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154014070 con 0x7f0164103180 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.200+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0154010ae0 con 0x7f0164103180 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.200+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154017ce0 con 0x7f0164103180 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.200+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01641a0220 con 0x7f0164103180 2024-04-22T07:13:01.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.200+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01641a0840 con 0x7f0164103180 2024-04-22T07:13:01.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.201+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01641092b0 con 0x7f0164103180 2024-04-22T07:13:01.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.202+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0154017070 con 0x7f0164103180 2024-04-22T07:13:01.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.202+0000 7f015b7fe640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f01500a3cc0 0x7f01500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.202+0000 7f016a2b1640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f01500a3cc0 0x7f01500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.203+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f01540cf8f0 con 0x7f0164103180 2024-04-22T07:13:01.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.203+0000 7f016a2b1640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f01500a3cc0 0x7f01500a6180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0160009230 tx=0x7f0160005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:01.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.207+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f015409bd60 con 0x7f0164103180 2024-04-22T07:13:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.424+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f016402d5c0 con 0x7f0164103180 2024-04-22T07:13:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.425+0000 7f015b7fe640 1 -- 172.21.15.113:0/3520669087 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11875 (secure 0 0 0) 0x7f01540a0060 con 0x7f0164103180 2024-04-22T07:13:01.423 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:01.423 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:12:58.522890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24473,24506]},"id":1}]} 2024-04-22T07:13:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f01500a3cc0 msgr2=0x7f01500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f01500a3cc0 0x7f01500a6180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0160009230 tx=0x7f0160005fd0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 msgr2=0x7f0164198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 0x7f0164198850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f015400b4a0 tx=0x7f015400b960 comp rx=0 tx=0).stop 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 shutdown_connections 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f01500a3cc0 0x7f01500a6180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0164103b40 0x7f016419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0164103180 0x7f0164198850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 --2- 172.21.15.113:0/3520669087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0164101f90 0x7f0164198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 >> 172.21.15.113:0/3520669087 conn(0x7f01640fd500 msgr2=0x7f0164106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 shutdown_connections 2024-04-22T07:13:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.427+0000 7f016c515640 1 -- 172.21.15.113:0/3520669087 wait complete. 2024-04-22T07:13:01.434 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:01 smithi113 ceph-mon[27933]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:01.434 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:01 smithi113 ceph-mon[27933]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:01.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:13:01.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 -- 172.21.15.113:0/2092351845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 msgr2=0x7fda9c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 --2- 172.21.15.113:0/2092351845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c104e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda98009900 tx=0x7fda9802bc30 comp rx=0 tx=0).stop 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 -- 172.21.15.113:0/2092351845 shutdown_connections 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 --2- 172.21.15.113:0/2092351845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c107c80 0x7fda9c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 --2- 172.21.15.113:0/2092351845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c105350 0x7fda9c107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 --2- 172.21.15.113:0/2092351845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c104e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 -- 172.21.15.113:0/2092351845 >> 172.21.15.113:0/2092351845 conn(0x7fda9c0fc410 msgr2=0x7fda9c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 -- 172.21.15.113:0/2092351845 shutdown_connections 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.535+0000 7fdaa4099640 1 -- 172.21.15.113:0/2092351845 wait complete. 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 Processor -- start 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 -- start start 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c105350 0x7fda9c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c107c80 0x7fda9c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10dc20 con 0x7fda9c102a20 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10daa0 con 0x7fda9c107c80 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa4099640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10dda0 con 0x7fda9c105350 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43192/0 (socket says 172.21.15.113:43192) 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 -- 172.21.15.113:0/3645052780 learned_addr learned my addr 172.21.15.113:0/3645052780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1634640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c105350 0x7fda9c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa2636640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c107c80 0x7fda9c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 -- 172.21.15.113:0/3645052780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c105350 msgr2=0x7fda9c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c105350 0x7fda9c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 -- 172.21.15.113:0/3645052780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c107c80 msgr2=0x7fda9c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c107c80 0x7fda9c19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 -- 172.21.15.113:0/3645052780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c19fef0 con 0x7fda9c102a20 2024-04-22T07:13:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.536+0000 7fdaa1e35640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c1982d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda98005ab0 tx=0x7fda98005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.537+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98041070 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.537+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fda980052f0 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.537+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1a0180 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.537+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98003dc0 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.537+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a0720 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.537+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c0fdf00 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.539+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fda98044070 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.539+0000 7fda92ffd640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda880a3cc0 0x7fda880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.539+0000 7fdaa1634640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda880a3cc0 0x7fda880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.539+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fda980f2090 con 0x7fda9c102a20 2024-04-22T07:13:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.540+0000 7fdaa1634640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda880a3cc0 0x7fda880a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda9c1997e0 tx=0x7fda8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:01.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.543+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda980bebd0 con 0x7fda9c102a20 2024-04-22T07:13:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:01 smithi133 ceph-mon[31198]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:01 smithi133 ceph-mon[31198]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:01 smithi154 ceph-mon[31862]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:01 smithi154 ceph-mon[31862]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.721+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda9c062b50 con 0x7fda9c102a20 2024-04-22T07:13:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.722+0000 7fda92ffd640 1 -- 172.21.15.113:0/3645052780 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22267 (secure 0 0 0) 0x7fda980be4d0 con 0x7fda9c102a20 2024-04-22T07:13:01.720 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:01.720 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":86,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:56.154646+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda880a3cc0 msgr2=0x7fda880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda880a3cc0 0x7fda880a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda9c1997e0 tx=0x7fda8c005f90 comp rx=0 tx=0).stop 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 msgr2=0x7fda9c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c1982d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda98005ab0 tx=0x7fda98005e60 comp rx=0 tx=0).stop 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 shutdown_connections 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda880a3cc0 0x7fda880a6180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c107c80 0x7fda9c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c105350 0x7fda9c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 --2- 172.21.15.113:0/3645052780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda9c102a20 0x7fda9c1982d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 >> 172.21.15.113:0/3645052780 conn(0x7fda9c0fc410 msgr2=0x7fda9c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:01.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 shutdown_connections 2024-04-22T07:13:01.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.724+0000 7fdaa4099640 1 -- 172.21.15.113:0/3645052780 wait complete. 2024-04-22T07:13:01.733 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-22T07:13:01.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 -- 172.21.15.113:0/168224599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780100e80 msgr2=0x7f37801012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 --2- 172.21.15.113:0/168224599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780100e80 0x7f37801012a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3778009ca0 tx=0x7f377802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 -- 172.21.15.113:0/168224599 shutdown_connections 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 --2- 172.21.15.113:0/168224599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780102990 0x7f37801092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 --2- 172.21.15.113:0/168224599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3780101fd0 0x7f3780102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 --2- 172.21.15.113:0/168224599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780100e80 0x7f37801012a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 -- 172.21.15.113:0/168224599 >> 172.21.15.113:0/168224599 conn(0x7f37800fc410 msgr2=0x7f37800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 -- 172.21.15.113:0/168224599 shutdown_connections 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 -- 172.21.15.113:0/168224599 wait complete. 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 Processor -- start 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 -- start start 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3780100e80 0x7f3780197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.833+0000 7f3787de6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780101fd0 0x7f37801976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3787de6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 0x7f378019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3787de6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f378010caf0 con 0x7f3780102990 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3787de6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f378010c970 con 0x7f3780100e80 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3787de6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f378010cc70 con 0x7f3780101fd0 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 0x7f378019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 0x7f378019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43200/0 (socket says 172.21.15.113:43200) 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 -- 172.21.15.113:0/1230336217 learned_addr learned my addr 172.21.15.113:0/1230336217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:01.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3785381640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780101fd0 0x7f37801976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 -- 172.21.15.113:0/1230336217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780101fd0 msgr2=0x7f37801976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780101fd0 0x7f37801976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 -- 172.21.15.113:0/1230336217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3780100e80 msgr2=0x7f3780197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3785b82640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3780100e80 0x7f3780197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3780100e80 0x7f3780197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 -- 172.21.15.113:0/1230336217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378019ed80 con 0x7f3780102990 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3785381640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780101fd0 0x7f37801976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3786383640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 0x7f378019e710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f377000d750 tx=0x7f377000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770017070 con 0x7f3780102990 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378019f070 con 0x7f3780102990 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f37700054f0 con 0x7f3780102990 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004cd0 con 0x7f3780102990 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.834+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378019f570 con 0x7f3780102990 2024-04-22T07:13:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.835+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3780108190 con 0x7f3780102990 2024-04-22T07:13:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.836+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3770009bd0 con 0x7f3780102990 2024-04-22T07:13:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.837+0000 7f377effd640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f37680a3c40 0x7f37680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.837+0000 7f3785b82640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f37680a3c40 0x7f37680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.837+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f37700cff20 con 0x7f3780102990 2024-04-22T07:13:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.837+0000 7f3785b82640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f37680a3c40 0x7f37680a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3778032460 tx=0x7f37780093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:01.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:01.841+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f377009c950 con 0x7f3780102990 2024-04-22T07:13:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.015+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3780062390 con 0x7f37680a3c40 2024-04-22T07:13:02.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.022+0000 7f377effd640 1 -- 172.21.15.113:0/1230336217 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258755 (secure 0 0 0) 0x7f3780062390 con 0x7f37680a3c40 2024-04-22T07:13:02.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f37680a3c40 msgr2=0x7f37680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f37680a3c40 0x7f37680a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3778032460 tx=0x7f37780093f0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 msgr2=0x7f378019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 0x7f378019e710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f377000d750 tx=0x7f377000dc10 comp rx=0 tx=0).stop 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 shutdown_connections 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f37680a3c40 0x7f37680a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3780102990 0x7f378019e710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3780101fd0 0x7f37801976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 --2- 172.21.15.113:0/1230336217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3780100e80 0x7f3780197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 >> 172.21.15.113:0/1230336217 conn(0x7f37800fc410 msgr2=0x7f3780105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:02.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 shutdown_connections 2024-04-22T07:13:02.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.025+0000 7f3787de6640 1 -- 172.21.15.113:0/1230336217 wait complete. 2024-04-22T07:13:02.041 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-22T07:13:01.073334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":157,"num_read_kb":138,"num_write":305,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":277,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366164,"kb_used_data":12824,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575660,"statfs":{"total":1151940427776,"available":1151565475840,"internally_reserved":0,"allocated":13131776,"data_stored":7131777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":21,"num_read_kb":19,"num_write":124,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945748"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525513+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:12:56.525513+0000","last_peered":"2024-04-22T07:12:56.525513+0000","last_clean":"2024-04-22T07:12:56.525513+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:12:56.525513+0000","last_undegraded":"2024-04-22T07:12:56.525513+0000","last_fullsized":"2024-04-22T07:12:56.525513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482698+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:12:56.482698+0000","last_peered":"2024-04-22T07:12:56.482698+0000","last_clean":"2024-04-22T07:12:56.482698+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:12:56.482698+0000","last_undegraded":"2024-04-22T07:12:56.482698+0000","last_fullsized":"2024-04-22T07:12:56.482698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482619+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:12:56.482619+0000","last_peered":"2024-04-22T07:12:56.482619+0000","last_clean":"2024-04-22T07:12:56.482619+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:12:56.482619+0000","last_undegraded":"2024-04-22T07:12:56.482619+0000","last_fullsized":"2024-04-22T07:12:56.482619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163392+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:12:56.163392+0000","last_peered":"2024-04-22T07:12:56.163392+0000","last_clean":"2024-04-22T07:12:56.163392+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:12:56.163392+0000","last_undegraded":"2024-04-22T07:12:56.163392+0000","last_fullsized":"2024-04-22T07:12:56.163392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.167866+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:12:56.167866+0000","last_peered":"2024-04-22T07:12:56.167866+0000","last_clean":"2024-04-22T07:12:56.167866+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:12:56.167866+0000","last_undegraded":"2024-04-22T07:12:56.167866+0000","last_fullsized":"2024-04-22T07:12:56.167866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482503+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:12:56.482503+0000","last_peered":"2024-04-22T07:12:56.482503+0000","last_clean":"2024-04-22T07:12:56.482503+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:12:56.482503+0000","last_undegraded":"2024-04-22T07:12:56.482503+0000","last_fullsized":"2024-04-22T07:12:56.482503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482511+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:12:56.482511+0000","last_peered":"2024-04-22T07:12:56.482511+0000","last_clean":"2024-04-22T07:12:56.482511+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:12:56.482511+0000","last_undegraded":"2024-04-22T07:12:56.482511+0000","last_fullsized":"2024-04-22T07:12:56.482511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162428+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:12:56.162428+0000","last_peered":"2024-04-22T07:12:56.162428+0000","last_clean":"2024-04-22T07:12:56.162428+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:12:56.162428+0000","last_undegraded":"2024-04-22T07:12:56.162428+0000","last_fullsized":"2024-04-22T07:12:56.162428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483610+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:12:56.483610+0000","last_peered":"2024-04-22T07:12:56.483610+0000","last_clean":"2024-04-22T07:12:56.483610+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:12:56.483610+0000","last_undegraded":"2024-04-22T07:12:56.483610+0000","last_fullsized":"2024-04-22T07:12:56.483610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482915+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:12:56.482915+0000","last_peered":"2024-04-22T07:12:56.482915+0000","last_clean":"2024-04-22T07:12:56.482915+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:12:56.482915+0000","last_undegraded":"2024-04-22T07:12:56.482915+0000","last_fullsized":"2024-04-22T07:12:56.482915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483470+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:12:56.483470+0000","last_peered":"2024-04-22T07:12:56.483470+0000","last_clean":"2024-04-22T07:12:56.483470+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:12:56.483470+0000","last_undegraded":"2024-04-22T07:12:56.483470+0000","last_fullsized":"2024-04-22T07:12:56.483470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163387+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:12:56.163387+0000","last_peered":"2024-04-22T07:12:56.163387+0000","last_clean":"2024-04-22T07:12:56.163387+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:12:56.163387+0000","last_undegraded":"2024-04-22T07:12:56.163387+0000","last_fullsized":"2024-04-22T07:12:56.163387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482118+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:12:56.482118+0000","last_peered":"2024-04-22T07:12:56.482118+0000","last_clean":"2024-04-22T07:12:56.482118+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:12:56.482118+0000","last_undegraded":"2024-04-22T07:12:56.482118+0000","last_fullsized":"2024-04-22T07:12:56.482118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482220+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:12:56.482220+0000","last_peered":"2024-04-22T07:12:56.482220+0000","last_clean":"2024-04-22T07:12:56.482220+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:12:56.482220+0000","last_undegraded":"2024-04-22T07:12:56.482220+0000","last_fullsized":"2024-04-22T07:12:56.482220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482265+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:12:56.482265+0000","last_peered":"2024-04-22T07:12:56.482265+0000","last_clean":"2024-04-22T07:12:56.482265+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:12:56.482265+0000","last_undegraded":"2024-04-22T07:12:56.482265+0000","last_fullsized":"2024-04-22T07:12:56.482265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482136+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:12:56.482136+0000","last_peered":"2024-04-22T07:12:56.482136+0000","last_clean":"2024-04-22T07:12:56.482136+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:12:56.482136+0000","last_undegraded":"2024-04-22T07:12:56.482136+0000","last_fullsized":"2024-04-22T07:12:56.482136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525604+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:12:56.525604+0000","last_peered":"2024-04-22T07:12:56.525604+0000","last_clean":"2024-04-22T07:12:56.525604+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:12:56.525604+0000","last_undegraded":"2024-04-22T07:12:56.525604+0000","last_fullsized":"2024-04-22T07:12:56.525604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.165816+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:12:56.165816+0000","last_peered":"2024-04-22T07:12:56.165816+0000","last_clean":"2024-04-22T07:12:56.165816+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:12:56.165816+0000","last_undegraded":"2024-04-22T07:12:56.165816+0000","last_fullsized":"2024-04-22T07:12:56.165816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482675+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:12:56.482675+0000","last_peered":"2024-04-22T07:12:56.482675+0000","last_clean":"2024-04-22T07:12:56.482675+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:12:56.482675+0000","last_undegraded":"2024-04-22T07:12:56.482675+0000","last_fullsized":"2024-04-22T07:12:56.482675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.185158+0000","last_change":"2024-04-22T07:12:54.140563+0000","last_active":"2024-04-22T07:12:58.185158+0000","last_peered":"2024-04-22T07:12:58.185158+0000","last_clean":"2024-04-22T07:12:58.185158+0000","last_became_active":"2024-04-22T07:12:54.140254+0000","last_became_peered":"2024-04-22T07:12:54.140254+0000","last_unstale":"2024-04-22T07:12:58.185158+0000","last_undegraded":"2024-04-22T07:12:58.185158+0000","last_fullsized":"2024-04-22T07:12:58.185158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:10:41.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162925+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:12:56.162925+0000","last_peered":"2024-04-22T07:12:56.162925+0000","last_clean":"2024-04-22T07:12:56.162925+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:12:56.162925+0000","last_undegraded":"2024-04-22T07:12:56.162925+0000","last_fullsized":"2024-04-22T07:12:56.162925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163070+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:12:56.163070+0000","last_peered":"2024-04-22T07:12:56.163070+0000","last_clean":"2024-04-22T07:12:56.163070+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:12:56.163070+0000","last_undegraded":"2024-04-22T07:12:56.163070+0000","last_fullsized":"2024-04-22T07:12:56.163070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482859+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:12:56.482859+0000","last_peered":"2024-04-22T07:12:56.482859+0000","last_clean":"2024-04-22T07:12:56.482859+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:12:56.482859+0000","last_undegraded":"2024-04-22T07:12:56.482859+0000","last_fullsized":"2024-04-22T07:12:56.482859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.184296+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:12:58.184296+0000","last_peered":"2024-04-22T07:12:58.184296+0000","last_clean":"2024-04-22T07:12:58.184296+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:12:58.184296+0000","last_undegraded":"2024-04-22T07:12:58.184296+0000","last_fullsized":"2024-04-22T07:12:58.184296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483369+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:12:56.483369+0000","last_peered":"2024-04-22T07:12:56.483369+0000","last_clean":"2024-04-22T07:12:56.483369+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:12:56.483369+0000","last_undegraded":"2024-04-22T07:12:56.483369+0000","last_fullsized":"2024-04-22T07:12:56.483369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.168057+0000","last_change":"2024-04-22T07:12:54.142002+0000","last_active":"2024-04-22T07:12:56.168057+0000","last_peered":"2024-04-22T07:12:56.168057+0000","last_clean":"2024-04-22T07:12:56.168057+0000","last_became_active":"2024-04-22T07:12:54.141246+0000","last_became_peered":"2024-04-22T07:12:54.141246+0000","last_unstale":"2024-04-22T07:12:56.168057+0000","last_undegraded":"2024-04-22T07:12:56.168057+0000","last_fullsized":"2024-04-22T07:12:56.168057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:28:10.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163467+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:12:56.163467+0000","last_peered":"2024-04-22T07:12:56.163467+0000","last_clean":"2024-04-22T07:12:56.163467+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:12:56.163467+0000","last_undegraded":"2024-04-22T07:12:56.163467+0000","last_fullsized":"2024-04-22T07:12:56.163467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.179142+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:12:57.179142+0000","last_peered":"2024-04-22T07:12:57.179142+0000","last_clean":"2024-04-22T07:12:57.179142+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:12:57.179142+0000","last_undegraded":"2024-04-22T07:12:57.179142+0000","last_fullsized":"2024-04-22T07:12:57.179142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.167780+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:12:56.167780+0000","last_peered":"2024-04-22T07:12:56.167780+0000","last_clean":"2024-04-22T07:12:56.167780+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:12:56.167780+0000","last_undegraded":"2024-04-22T07:12:56.167780+0000","last_fullsized":"2024-04-22T07:12:56.167780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.185841+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:12:58.185841+0000","last_peered":"2024-04-22T07:12:58.185841+0000","last_clean":"2024-04-22T07:12:58.185841+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:12:58.185841+0000","last_undegraded":"2024-04-22T07:12:58.185841+0000","last_fullsized":"2024-04-22T07:12:58.185841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163290+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:12:56.163290+0000","last_peered":"2024-04-22T07:12:56.163290+0000","last_clean":"2024-04-22T07:12:56.163290+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:12:56.163290+0000","last_undegraded":"2024-04-22T07:12:56.163290+0000","last_fullsized":"2024-04-22T07:12:56.163290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.185202+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:12:58.185202+0000","last_peered":"2024-04-22T07:12:58.185202+0000","last_clean":"2024-04-22T07:12:58.185202+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:12:58.185202+0000","last_undegraded":"2024-04-22T07:12:58.185202+0000","last_fullsized":"2024-04-22T07:12:58.185202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162480+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:12:56.162480+0000","last_peered":"2024-04-22T07:12:56.162480+0000","last_clean":"2024-04-22T07:12:56.162480+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:12:56.162480+0000","last_undegraded":"2024-04-22T07:12:56.162480+0000","last_fullsized":"2024-04-22T07:12:56.162480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162674+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:12:56.162674+0000","last_peered":"2024-04-22T07:12:56.162674+0000","last_clean":"2024-04-22T07:12:56.162674+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:12:56.162674+0000","last_undegraded":"2024-04-22T07:12:56.162674+0000","last_fullsized":"2024-04-22T07:12:56.162674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.165682+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:12:56.165682+0000","last_peered":"2024-04-22T07:12:56.165682+0000","last_clean":"2024-04-22T07:12:56.165682+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:12:56.165682+0000","last_undegraded":"2024-04-22T07:12:56.165682+0000","last_fullsized":"2024-04-22T07:12:56.165682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.180960+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:12:57.180960+0000","last_peered":"2024-04-22T07:12:57.180960+0000","last_clean":"2024-04-22T07:12:57.180960+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:12:57.180960+0000","last_undegraded":"2024-04-22T07:12:57.180960+0000","last_fullsized":"2024-04-22T07:12:57.180960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483096+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:12:56.483096+0000","last_peered":"2024-04-22T07:12:56.483096+0000","last_clean":"2024-04-22T07:12:56.483096+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:12:56.483096+0000","last_undegraded":"2024-04-22T07:12:56.483096+0000","last_fullsized":"2024-04-22T07:12:56.483096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178382+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:12:57.178382+0000","last_peered":"2024-04-22T07:12:57.178382+0000","last_clean":"2024-04-22T07:12:57.178382+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:12:57.178382+0000","last_undegraded":"2024-04-22T07:12:57.178382+0000","last_fullsized":"2024-04-22T07:12:57.178382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482817+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:12:56.482817+0000","last_peered":"2024-04-22T07:12:56.482817+0000","last_clean":"2024-04-22T07:12:56.482817+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:12:56.482817+0000","last_undegraded":"2024-04-22T07:12:56.482817+0000","last_fullsized":"2024-04-22T07:12:56.482817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163502+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:12:56.163502+0000","last_peered":"2024-04-22T07:12:56.163502+0000","last_clean":"2024-04-22T07:12:56.163502+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:12:56.163502+0000","last_undegraded":"2024-04-22T07:12:56.163502+0000","last_fullsized":"2024-04-22T07:12:56.163502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483521+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:12:56.483521+0000","last_peered":"2024-04-22T07:12:56.483521+0000","last_clean":"2024-04-22T07:12:56.483521+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:12:56.483521+0000","last_undegraded":"2024-04-22T07:12:56.483521+0000","last_fullsized":"2024-04-22T07:12:56.483521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.340493+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:12:58.340493+0000","last_peered":"2024-04-22T07:12:58.340493+0000","last_clean":"2024-04-22T07:12:58.340493+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:12:58.340493+0000","last_undegraded":"2024-04-22T07:12:58.340493+0000","last_fullsized":"2024-04-22T07:12:58.340493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.168040+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:12:56.168040+0000","last_peered":"2024-04-22T07:12:56.168040+0000","last_clean":"2024-04-22T07:12:56.168040+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:12:56.168040+0000","last_undegraded":"2024-04-22T07:12:56.168040+0000","last_fullsized":"2024-04-22T07:12:56.168040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.184548+0000","last_change":"2024-04-22T07:12:54.140783+0000","last_active":"2024-04-22T07:12:58.184548+0000","last_peered":"2024-04-22T07:12:58.184548+0000","last_clean":"2024-04-22T07:12:58.184548+0000","last_became_active":"2024-04-22T07:12:54.140402+0000","last_became_peered":"2024-04-22T07:12:54.140402+0000","last_unstale":"2024-04-22T07:12:58.184548+0000","last_undegraded":"2024-04-22T07:12:58.184548+0000","last_fullsized":"2024-04-22T07:12:58.184548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:22.838953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.481973+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:12:56.481973+0000","last_peered":"2024-04-22T07:12:56.481973+0000","last_clean":"2024-04-22T07:12:56.481973+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:12:56.481973+0000","last_undegraded":"2024-04-22T07:12:56.481973+0000","last_fullsized":"2024-04-22T07:12:56.481973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178963+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:12:57.178963+0000","last_peered":"2024-04-22T07:12:57.178963+0000","last_clean":"2024-04-22T07:12:57.178963+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:12:57.178963+0000","last_undegraded":"2024-04-22T07:12:57.178963+0000","last_fullsized":"2024-04-22T07:12:57.178963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482524+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:12:56.482524+0000","last_peered":"2024-04-22T07:12:56.482524+0000","last_clean":"2024-04-22T07:12:56.482524+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:12:56.482524+0000","last_undegraded":"2024-04-22T07:12:56.482524+0000","last_fullsized":"2024-04-22T07:12:56.482524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.341973+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:12:58.341973+0000","last_peered":"2024-04-22T07:12:58.341973+0000","last_clean":"2024-04-22T07:12:58.341973+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:12:58.341973+0000","last_undegraded":"2024-04-22T07:12:58.341973+0000","last_fullsized":"2024-04-22T07:12:58.341973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482092+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:12:56.482092+0000","last_peered":"2024-04-22T07:12:56.482092+0000","last_clean":"2024-04-22T07:12:56.482092+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:12:56.482092+0000","last_undegraded":"2024-04-22T07:12:56.482092+0000","last_fullsized":"2024-04-22T07:12:56.482092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.186446+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:12:58.186446+0000","last_peered":"2024-04-22T07:12:58.186446+0000","last_clean":"2024-04-22T07:12:58.186446+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:12:58.186446+0000","last_undegraded":"2024-04-22T07:12:58.186446+0000","last_fullsized":"2024-04-22T07:12:58.186446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162895+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:12:56.162895+0000","last_peered":"2024-04-22T07:12:56.162895+0000","last_clean":"2024-04-22T07:12:56.162895+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:12:56.162895+0000","last_undegraded":"2024-04-22T07:12:56.162895+0000","last_fullsized":"2024-04-22T07:12:56.162895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.168175+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:12:56.168175+0000","last_peered":"2024-04-22T07:12:56.168175+0000","last_clean":"2024-04-22T07:12:56.168175+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:12:56.168175+0000","last_undegraded":"2024-04-22T07:12:56.168175+0000","last_fullsized":"2024-04-22T07:12:56.168175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525369+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:12:56.525369+0000","last_peered":"2024-04-22T07:12:56.525369+0000","last_clean":"2024-04-22T07:12:56.525369+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:12:56.525369+0000","last_undegraded":"2024-04-22T07:12:56.525369+0000","last_fullsized":"2024-04-22T07:12:56.525369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.165705+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:12:56.165705+0000","last_peered":"2024-04-22T07:12:56.165705+0000","last_clean":"2024-04-22T07:12:56.165705+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:12:56.165705+0000","last_undegraded":"2024-04-22T07:12:56.165705+0000","last_fullsized":"2024-04-22T07:12:56.165705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482735+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:12:56.482735+0000","last_peered":"2024-04-22T07:12:56.482735+0000","last_clean":"2024-04-22T07:12:56.482735+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:12:56.482735+0000","last_undegraded":"2024-04-22T07:12:56.482735+0000","last_fullsized":"2024-04-22T07:12:56.482735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483168+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:12:56.483168+0000","last_peered":"2024-04-22T07:12:56.483168+0000","last_clean":"2024-04-22T07:12:56.483168+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:12:56.483168+0000","last_undegraded":"2024-04-22T07:12:56.483168+0000","last_fullsized":"2024-04-22T07:12:56.483168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483778+0000","last_change":"2024-04-22T07:12:55.175229+0000","last_active":"2024-04-22T07:12:56.483778+0000","last_peered":"2024-04-22T07:12:56.483778+0000","last_clean":"2024-04-22T07:12:56.483778+0000","last_became_active":"2024-04-22T07:12:55.174799+0000","last_became_peered":"2024-04-22T07:12:55.174799+0000","last_unstale":"2024-04-22T07:12:56.483778+0000","last_undegraded":"2024-04-22T07:12:56.483778+0000","last_fullsized":"2024-04-22T07:12:56.483778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:18:26.650199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163041+0000","last_change":"2024-04-22T07:12:55.174700+0000","last_active":"2024-04-22T07:12:56.163041+0000","last_peered":"2024-04-22T07:12:56.163041+0000","last_clean":"2024-04-22T07:12:56.163041+0000","last_became_active":"2024-04-22T07:12:55.173133+0000","last_became_peered":"2024-04-22T07:12:55.173133+0000","last_unstale":"2024-04-22T07:12:56.163041+0000","last_undegraded":"2024-04-22T07:12:56.163041+0000","last_fullsized":"2024-04-22T07:12:56.163041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:34:54.914711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.186460+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:12:58.186460+0000","last_peered":"2024-04-22T07:12:58.186460+0000","last_clean":"2024-04-22T07:12:58.186460+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:12:58.186460+0000","last_undegraded":"2024-04-22T07:12:58.186460+0000","last_fullsized":"2024-04-22T07:12:58.186460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162650+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:12:56.162650+0000","last_peered":"2024-04-22T07:12:56.162650+0000","last_clean":"2024-04-22T07:12:56.162650+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:12:56.162650+0000","last_undegraded":"2024-04-22T07:12:56.162650+0000","last_fullsized":"2024-04-22T07:12:56.162650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.338900+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:12:58.338900+0000","last_peered":"2024-04-22T07:12:58.338900+0000","last_clean":"2024-04-22T07:12:58.338900+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:12:58.338900+0000","last_undegraded":"2024-04-22T07:12:58.338900+0000","last_fullsized":"2024-04-22T07:12:58.338900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482108+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:12:56.482108+0000","last_peered":"2024-04-22T07:12:56.482108+0000","last_clean":"2024-04-22T07:12:56.482108+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:12:56.482108+0000","last_undegraded":"2024-04-22T07:12:56.482108+0000","last_fullsized":"2024-04-22T07:12:56.482108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.481795+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:12:56.481795+0000","last_peered":"2024-04-22T07:12:56.481795+0000","last_clean":"2024-04-22T07:12:56.481795+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:12:56.481795+0000","last_undegraded":"2024-04-22T07:12:56.481795+0000","last_fullsized":"2024-04-22T07:12:56.481795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.183826+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:12:58.183826+0000","last_peered":"2024-04-22T07:12:58.183826+0000","last_clean":"2024-04-22T07:12:58.183826+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:12:58.183826+0000","last_undegraded":"2024-04-22T07:12:58.183826+0000","last_fullsized":"2024-04-22T07:12:58.183826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178533+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:12:57.178533+0000","last_peered":"2024-04-22T07:12:57.178533+0000","last_clean":"2024-04-22T07:12:57.178533+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:12:57.178533+0000","last_undegraded":"2024-04-22T07:12:57.178533+0000","last_fullsized":"2024-04-22T07:12:57.178533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482079+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:12:56.482079+0000","last_peered":"2024-04-22T07:12:56.482079+0000","last_clean":"2024-04-22T07:12:56.482079+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:12:56.482079+0000","last_undegraded":"2024-04-22T07:12:56.482079+0000","last_fullsized":"2024-04-22T07:12:56.482079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.183492+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:12:58.183492+0000","last_peered":"2024-04-22T07:12:58.183492+0000","last_clean":"2024-04-22T07:12:58.183492+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:12:58.183492+0000","last_undegraded":"2024-04-22T07:12:58.183492+0000","last_fullsized":"2024-04-22T07:12:58.183492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163114+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:12:56.163114+0000","last_peered":"2024-04-22T07:12:56.163114+0000","last_clean":"2024-04-22T07:12:56.163114+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:12:56.163114+0000","last_undegraded":"2024-04-22T07:12:56.163114+0000","last_fullsized":"2024-04-22T07:12:56.163114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.177639+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:12:57.177639+0000","last_peered":"2024-04-22T07:12:57.177639+0000","last_clean":"2024-04-22T07:12:57.177639+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:12:57.177639+0000","last_undegraded":"2024-04-22T07:12:57.177639+0000","last_fullsized":"2024-04-22T07:12:57.177639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482683+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:12:56.482683+0000","last_peered":"2024-04-22T07:12:56.482683+0000","last_clean":"2024-04-22T07:12:56.482683+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:12:56.482683+0000","last_undegraded":"2024-04-22T07:12:56.482683+0000","last_fullsized":"2024-04-22T07:12:56.482683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.177785+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:12:57.177785+0000","last_peered":"2024-04-22T07:12:57.177785+0000","last_clean":"2024-04-22T07:12:57.177785+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:12:57.177785+0000","last_undegraded":"2024-04-22T07:12:57.177785+0000","last_fullsized":"2024-04-22T07:12:57.177785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.167986+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:12:56.167986+0000","last_peered":"2024-04-22T07:12:56.167986+0000","last_clean":"2024-04-22T07:12:56.167986+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:12:56.167986+0000","last_undegraded":"2024-04-22T07:12:56.167986+0000","last_fullsized":"2024-04-22T07:12:56.167986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.180554+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:12:57.180554+0000","last_peered":"2024-04-22T07:12:57.180554+0000","last_clean":"2024-04-22T07:12:57.180554+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:12:57.180554+0000","last_undegraded":"2024-04-22T07:12:57.180554+0000","last_fullsized":"2024-04-22T07:12:57.180554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525128+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:12:56.525128+0000","last_peered":"2024-04-22T07:12:56.525128+0000","last_clean":"2024-04-22T07:12:56.525128+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:12:56.525128+0000","last_undegraded":"2024-04-22T07:12:56.525128+0000","last_fullsized":"2024-04-22T07:12:56.525128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482369+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:12:56.482369+0000","last_peered":"2024-04-22T07:12:56.482369+0000","last_clean":"2024-04-22T07:12:56.482369+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:12:56.482369+0000","last_undegraded":"2024-04-22T07:12:56.482369+0000","last_fullsized":"2024-04-22T07:12:56.482369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162743+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:12:56.162743+0000","last_peered":"2024-04-22T07:12:56.162743+0000","last_clean":"2024-04-22T07:12:56.162743+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:12:56.162743+0000","last_undegraded":"2024-04-22T07:12:56.162743+0000","last_fullsized":"2024-04-22T07:12:56.162743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178775+0000","last_change":"2024-04-22T07:12:54.147007+0000","last_active":"2024-04-22T07:12:57.178775+0000","last_peered":"2024-04-22T07:12:57.178775+0000","last_clean":"2024-04-22T07:12:57.178775+0000","last_became_active":"2024-04-22T07:12:54.146668+0000","last_became_peered":"2024-04-22T07:12:54.146668+0000","last_unstale":"2024-04-22T07:12:57.178775+0000","last_undegraded":"2024-04-22T07:12:57.178775+0000","last_fullsized":"2024-04-22T07:12:57.178775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:28.076768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483277+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:12:56.483277+0000","last_peered":"2024-04-22T07:12:56.483277+0000","last_clean":"2024-04-22T07:12:56.483277+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:12:56.483277+0000","last_undegraded":"2024-04-22T07:12:56.483277+0000","last_fullsized":"2024-04-22T07:12:56.483277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.186320+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:12:58.186320+0000","last_peered":"2024-04-22T07:12:58.186320+0000","last_clean":"2024-04-22T07:12:58.186320+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:12:58.186320+0000","last_undegraded":"2024-04-22T07:12:58.186320+0000","last_fullsized":"2024-04-22T07:12:58.186320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483670+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:12:56.483670+0000","last_peered":"2024-04-22T07:12:56.483670+0000","last_clean":"2024-04-22T07:12:56.483670+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:12:56.483670+0000","last_undegraded":"2024-04-22T07:12:56.483670+0000","last_fullsized":"2024-04-22T07:12:56.483670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163299+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:12:56.163299+0000","last_peered":"2024-04-22T07:12:56.163299+0000","last_clean":"2024-04-22T07:12:56.163299+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:12:56.163299+0000","last_undegraded":"2024-04-22T07:12:56.163299+0000","last_fullsized":"2024-04-22T07:12:56.163299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482714+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:12:56.482714+0000","last_peered":"2024-04-22T07:12:56.482714+0000","last_clean":"2024-04-22T07:12:56.482714+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:12:56.482714+0000","last_undegraded":"2024-04-22T07:12:56.482714+0000","last_fullsized":"2024-04-22T07:12:56.482714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.481974+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:12:56.481974+0000","last_peered":"2024-04-22T07:12:56.481974+0000","last_clean":"2024-04-22T07:12:56.481974+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:12:56.481974+0000","last_undegraded":"2024-04-22T07:12:56.481974+0000","last_fullsized":"2024-04-22T07:12:56.481974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482735+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:12:56.482735+0000","last_peered":"2024-04-22T07:12:56.482735+0000","last_clean":"2024-04-22T07:12:56.482735+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:12:56.482735+0000","last_undegraded":"2024-04-22T07:12:56.482735+0000","last_fullsized":"2024-04-22T07:12:56.482735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525239+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:12:56.525239+0000","last_peered":"2024-04-22T07:12:56.525239+0000","last_clean":"2024-04-22T07:12:56.525239+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:12:56.525239+0000","last_undegraded":"2024-04-22T07:12:56.525239+0000","last_fullsized":"2024-04-22T07:12:56.525239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482542+0000","last_change":"2024-04-22T07:12:55.169456+0000","last_active":"2024-04-22T07:12:56.482542+0000","last_peered":"2024-04-22T07:12:56.482542+0000","last_clean":"2024-04-22T07:12:56.482542+0000","last_became_active":"2024-04-22T07:12:55.169000+0000","last_became_peered":"2024-04-22T07:12:55.169000+0000","last_unstale":"2024-04-22T07:12:56.482542+0000","last_undegraded":"2024-04-22T07:12:56.482542+0000","last_fullsized":"2024-04-22T07:12:56.482542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:06.709915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482419+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:12:56.482419+0000","last_peered":"2024-04-22T07:12:56.482419+0000","last_clean":"2024-04-22T07:12:56.482419+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:12:56.482419+0000","last_undegraded":"2024-04-22T07:12:56.482419+0000","last_fullsized":"2024-04-22T07:12:56.482419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482537+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:12:56.482537+0000","last_peered":"2024-04-22T07:12:56.482537+0000","last_clean":"2024-04-22T07:12:56.482537+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:12:56.482537+0000","last_undegraded":"2024-04-22T07:12:56.482537+0000","last_fullsized":"2024-04-22T07:12:56.482537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482051+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:12:56.482051+0000","last_peered":"2024-04-22T07:12:56.482051+0000","last_clean":"2024-04-22T07:12:56.482051+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:12:56.482051+0000","last_undegraded":"2024-04-22T07:12:56.482051+0000","last_fullsized":"2024-04-22T07:12:56.482051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525380+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:12:56.525380+0000","last_peered":"2024-04-22T07:12:56.525380+0000","last_clean":"2024-04-22T07:12:56.525380+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:12:56.525380+0000","last_undegraded":"2024-04-22T07:12:56.525380+0000","last_fullsized":"2024-04-22T07:12:56.525380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482760+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:12:56.482760+0000","last_peered":"2024-04-22T07:12:56.482760+0000","last_clean":"2024-04-22T07:12:56.482760+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:12:56.482760+0000","last_undegraded":"2024-04-22T07:12:56.482760+0000","last_fullsized":"2024-04-22T07:12:56.482760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163089+0000","last_change":"2024-04-22T07:12:55.169344+0000","last_active":"2024-04-22T07:12:56.163089+0000","last_peered":"2024-04-22T07:12:56.163089+0000","last_clean":"2024-04-22T07:12:56.163089+0000","last_became_active":"2024-04-22T07:12:55.168995+0000","last_became_peered":"2024-04-22T07:12:55.168995+0000","last_unstale":"2024-04-22T07:12:56.163089+0000","last_undegraded":"2024-04-22T07:12:56.163089+0000","last_fullsized":"2024-04-22T07:12:56.163089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:47.870749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163292+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:12:56.163292+0000","last_peered":"2024-04-22T07:12:56.163292+0000","last_clean":"2024-04-22T07:12:56.163292+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:12:56.163292+0000","last_undegraded":"2024-04-22T07:12:56.163292+0000","last_fullsized":"2024-04-22T07:12:56.163292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162446+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:12:56.162446+0000","last_peered":"2024-04-22T07:12:56.162446+0000","last_clean":"2024-04-22T07:12:56.162446+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:12:56.162446+0000","last_undegraded":"2024-04-22T07:12:56.162446+0000","last_fullsized":"2024-04-22T07:12:56.162446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483238+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:12:56.483238+0000","last_peered":"2024-04-22T07:12:56.483238+0000","last_clean":"2024-04-22T07:12:56.483238+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:12:56.483238+0000","last_undegraded":"2024-04-22T07:12:56.483238+0000","last_fullsized":"2024-04-22T07:12:56.483238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482108+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:12:56.482108+0000","last_peered":"2024-04-22T07:12:56.482108+0000","last_clean":"2024-04-22T07:12:56.482108+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:12:56.482108+0000","last_undegraded":"2024-04-22T07:12:56.482108+0000","last_fullsized":"2024-04-22T07:12:56.482108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525296+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:12:56.525296+0000","last_peered":"2024-04-22T07:12:56.525296+0000","last_clean":"2024-04-22T07:12:56.525296+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:12:56.525296+0000","last_undegraded":"2024-04-22T07:12:56.525296+0000","last_fullsized":"2024-04-22T07:12:56.525296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":21,"num_read_kb":19,"num_write":124,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":44110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":444461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1037437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":444505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":464774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":446841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":444163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1035723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":30056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:13:02.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 -- 172.21.15.113:0/3164271591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac103180 msgr2=0x7fdaac103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 --2- 172.21.15.113:0/3164271591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac103180 0x7fdaac103600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009900 tx=0x7fda9c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 -- 172.21.15.113:0/3164271591 shutdown_connections 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 --2- 172.21.15.113:0/3164271591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103b40 0x7fdaac10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 --2- 172.21.15.113:0/3164271591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac103180 0x7fdaac103600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 --2- 172.21.15.113:0/3164271591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaac101f90 0x7fdaac1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 -- 172.21.15.113:0/3164271591 >> 172.21.15.113:0/3164271591 conn(0x7fdaac0fd500 msgr2=0x7fdaac0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 -- 172.21.15.113:0/3164271591 shutdown_connections 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.156+0000 7fdab366a640 1 -- 172.21.15.113:0/3164271591 wait complete. 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 Processor -- start 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 -- start start 2024-04-22T07:13:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 0x7fdaac198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103180 0x7fdaac198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaac103b40 0x7fdaac19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10dbf0 con 0x7fdaac101f90 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10da70 con 0x7fdaac103180 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab366a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10dd70 con 0x7fdaac103b40 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 0x7fdaac198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 0x7fdaac198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43210/0 (socket says 172.21.15.113:43210) 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 -- 172.21.15.113:0/2164071068 learned_addr learned my addr 172.21.15.113:0/2164071068 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1c07640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaac103b40 0x7fdaac19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 -- 172.21.15.113:0/2164071068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaac103b40 msgr2=0x7fdaac19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaac103b40 0x7fdaac19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab0c05640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103180 0x7fdaac198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 -- 172.21.15.113:0/2164071068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103180 msgr2=0x7fdaac198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103180 0x7fdaac198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 -- 172.21.15.113:0/2164071068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac19fd70 con 0x7fdaac101f90 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab0c05640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103180 0x7fdaac198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdab1406640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 0x7fdaac198150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c970 tx=0x7fdaa800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.157+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8007c10 con 0x7fdaac101f90 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.158+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdaa8007db0 con 0x7fdaac101f90 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.158+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a0060 con 0x7fdaac101f90 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.158+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800f910 con 0x7fdaac101f90 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.158+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a05b0 con 0x7fdaac101f90 2024-04-22T07:13:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.159+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac109250 con 0x7fdaac101f90 2024-04-22T07:13:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.160+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdaa800fab0 con 0x7fdaac101f90 2024-04-22T07:13:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.160+0000 7fdaa27fc640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3c40 0x7fda980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.160+0000 7fdab0c05640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3c40 0x7fda980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.160+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdaa80d0830 con 0x7fdaac101f90 2024-04-22T07:13:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.161+0000 7fdab0c05640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3c40 0x7fda980a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0096c0 tx=0x7fda9c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:02.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.165+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdaa809d2f0 con 0x7fdaac101f90 2024-04-22T07:13:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.5 KiB/s wr, 20 op/s 2024-04-22T07:13:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T07:13:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3520669087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:13:02.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3645052780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:02.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.342+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdaac062390 con 0x7fda980a3c40 2024-04-22T07:13:02.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.349+0000 7fdaa27fc640 1 -- 172.21.15.113:0/2164071068 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258755 (secure 0 0 0) 0x7fdaac062390 con 0x7fda980a3c40 2024-04-22T07:13:02.347 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:02.356 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:13:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3c40 msgr2=0x7fda980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3c40 0x7fda980a6100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0096c0 tx=0x7fda9c000bc0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 msgr2=0x7fdaac198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 0x7fdaac198150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800c970 tx=0x7fdaa800ce30 comp rx=0 tx=0).stop 2024-04-22T07:13:02.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 shutdown_connections 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fda980a3c40 0x7fda980a6100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaac103b40 0x7fdaac19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac103180 0x7fdaac198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 --2- 172.21.15.113:0/2164071068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdaac101f90 0x7fdaac198150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 >> 172.21.15.113:0/2164071068 conn(0x7fdaac0fd500 msgr2=0x7fdaac106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 shutdown_connections 2024-04-22T07:13:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:02.352+0000 7fdab366a640 1 -- 172.21.15.113:0/2164071068 wait complete. 2024-04-22T07:13:02.360 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-22T07:13:01.073334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":157,"num_read_kb":138,"num_write":305,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":277,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366164,"kb_used_data":12824,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575660,"statfs":{"total":1151940427776,"available":1151565475840,"internally_reserved":0,"allocated":13131776,"data_stored":7131777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":21,"num_read_kb":19,"num_write":124,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945748"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525513+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:12:56.525513+0000","last_peered":"2024-04-22T07:12:56.525513+0000","last_clean":"2024-04-22T07:12:56.525513+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:12:56.525513+0000","last_undegraded":"2024-04-22T07:12:56.525513+0000","last_fullsized":"2024-04-22T07:12:56.525513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482698+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:12:56.482698+0000","last_peered":"2024-04-22T07:12:56.482698+0000","last_clean":"2024-04-22T07:12:56.482698+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:12:56.482698+0000","last_undegraded":"2024-04-22T07:12:56.482698+0000","last_fullsized":"2024-04-22T07:12:56.482698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482619+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:12:56.482619+0000","last_peered":"2024-04-22T07:12:56.482619+0000","last_clean":"2024-04-22T07:12:56.482619+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:12:56.482619+0000","last_undegraded":"2024-04-22T07:12:56.482619+0000","last_fullsized":"2024-04-22T07:12:56.482619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163392+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:12:56.163392+0000","last_peered":"2024-04-22T07:12:56.163392+0000","last_clean":"2024-04-22T07:12:56.163392+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:12:56.163392+0000","last_undegraded":"2024-04-22T07:12:56.163392+0000","last_fullsized":"2024-04-22T07:12:56.163392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.167866+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:12:56.167866+0000","last_peered":"2024-04-22T07:12:56.167866+0000","last_clean":"2024-04-22T07:12:56.167866+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:12:56.167866+0000","last_undegraded":"2024-04-22T07:12:56.167866+0000","last_fullsized":"2024-04-22T07:12:56.167866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482503+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:12:56.482503+0000","last_peered":"2024-04-22T07:12:56.482503+0000","last_clean":"2024-04-22T07:12:56.482503+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:12:56.482503+0000","last_undegraded":"2024-04-22T07:12:56.482503+0000","last_fullsized":"2024-04-22T07:12:56.482503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482511+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:12:56.482511+0000","last_peered":"2024-04-22T07:12:56.482511+0000","last_clean":"2024-04-22T07:12:56.482511+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:12:56.482511+0000","last_undegraded":"2024-04-22T07:12:56.482511+0000","last_fullsized":"2024-04-22T07:12:56.482511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162428+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:12:56.162428+0000","last_peered":"2024-04-22T07:12:56.162428+0000","last_clean":"2024-04-22T07:12:56.162428+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:12:56.162428+0000","last_undegraded":"2024-04-22T07:12:56.162428+0000","last_fullsized":"2024-04-22T07:12:56.162428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483610+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:12:56.483610+0000","last_peered":"2024-04-22T07:12:56.483610+0000","last_clean":"2024-04-22T07:12:56.483610+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:12:56.483610+0000","last_undegraded":"2024-04-22T07:12:56.483610+0000","last_fullsized":"2024-04-22T07:12:56.483610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482915+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:12:56.482915+0000","last_peered":"2024-04-22T07:12:56.482915+0000","last_clean":"2024-04-22T07:12:56.482915+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:12:56.482915+0000","last_undegraded":"2024-04-22T07:12:56.482915+0000","last_fullsized":"2024-04-22T07:12:56.482915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483470+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:12:56.483470+0000","last_peered":"2024-04-22T07:12:56.483470+0000","last_clean":"2024-04-22T07:12:56.483470+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:12:56.483470+0000","last_undegraded":"2024-04-22T07:12:56.483470+0000","last_fullsized":"2024-04-22T07:12:56.483470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163387+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:12:56.163387+0000","last_peered":"2024-04-22T07:12:56.163387+0000","last_clean":"2024-04-22T07:12:56.163387+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:12:56.163387+0000","last_undegraded":"2024-04-22T07:12:56.163387+0000","last_fullsized":"2024-04-22T07:12:56.163387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482118+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:12:56.482118+0000","last_peered":"2024-04-22T07:12:56.482118+0000","last_clean":"2024-04-22T07:12:56.482118+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:12:56.482118+0000","last_undegraded":"2024-04-22T07:12:56.482118+0000","last_fullsized":"2024-04-22T07:12:56.482118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482220+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:12:56.482220+0000","last_peered":"2024-04-22T07:12:56.482220+0000","last_clean":"2024-04-22T07:12:56.482220+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:12:56.482220+0000","last_undegraded":"2024-04-22T07:12:56.482220+0000","last_fullsized":"2024-04-22T07:12:56.482220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482265+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:12:56.482265+0000","last_peered":"2024-04-22T07:12:56.482265+0000","last_clean":"2024-04-22T07:12:56.482265+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:12:56.482265+0000","last_undegraded":"2024-04-22T07:12:56.482265+0000","last_fullsized":"2024-04-22T07:12:56.482265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482136+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:12:56.482136+0000","last_peered":"2024-04-22T07:12:56.482136+0000","last_clean":"2024-04-22T07:12:56.482136+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:12:56.482136+0000","last_undegraded":"2024-04-22T07:12:56.482136+0000","last_fullsized":"2024-04-22T07:12:56.482136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525604+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:12:56.525604+0000","last_peered":"2024-04-22T07:12:56.525604+0000","last_clean":"2024-04-22T07:12:56.525604+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:12:56.525604+0000","last_undegraded":"2024-04-22T07:12:56.525604+0000","last_fullsized":"2024-04-22T07:12:56.525604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.165816+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:12:56.165816+0000","last_peered":"2024-04-22T07:12:56.165816+0000","last_clean":"2024-04-22T07:12:56.165816+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:12:56.165816+0000","last_undegraded":"2024-04-22T07:12:56.165816+0000","last_fullsized":"2024-04-22T07:12:56.165816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482675+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:12:56.482675+0000","last_peered":"2024-04-22T07:12:56.482675+0000","last_clean":"2024-04-22T07:12:56.482675+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:12:56.482675+0000","last_undegraded":"2024-04-22T07:12:56.482675+0000","last_fullsized":"2024-04-22T07:12:56.482675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.185158+0000","last_change":"2024-04-22T07:12:54.140563+0000","last_active":"2024-04-22T07:12:58.185158+0000","last_peered":"2024-04-22T07:12:58.185158+0000","last_clean":"2024-04-22T07:12:58.185158+0000","last_became_active":"2024-04-22T07:12:54.140254+0000","last_became_peered":"2024-04-22T07:12:54.140254+0000","last_unstale":"2024-04-22T07:12:58.185158+0000","last_undegraded":"2024-04-22T07:12:58.185158+0000","last_fullsized":"2024-04-22T07:12:58.185158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:10:41.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162925+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:12:56.162925+0000","last_peered":"2024-04-22T07:12:56.162925+0000","last_clean":"2024-04-22T07:12:56.162925+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:12:56.162925+0000","last_undegraded":"2024-04-22T07:12:56.162925+0000","last_fullsized":"2024-04-22T07:12:56.162925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163070+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:12:56.163070+0000","last_peered":"2024-04-22T07:12:56.163070+0000","last_clean":"2024-04-22T07:12:56.163070+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:12:56.163070+0000","last_undegraded":"2024-04-22T07:12:56.163070+0000","last_fullsized":"2024-04-22T07:12:56.163070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482859+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:12:56.482859+0000","last_peered":"2024-04-22T07:12:56.482859+0000","last_clean":"2024-04-22T07:12:56.482859+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:12:56.482859+0000","last_undegraded":"2024-04-22T07:12:56.482859+0000","last_fullsized":"2024-04-22T07:12:56.482859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.184296+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:12:58.184296+0000","last_peered":"2024-04-22T07:12:58.184296+0000","last_clean":"2024-04-22T07:12:58.184296+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:12:58.184296+0000","last_undegraded":"2024-04-22T07:12:58.184296+0000","last_fullsized":"2024-04-22T07:12:58.184296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483369+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:12:56.483369+0000","last_peered":"2024-04-22T07:12:56.483369+0000","last_clean":"2024-04-22T07:12:56.483369+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:12:56.483369+0000","last_undegraded":"2024-04-22T07:12:56.483369+0000","last_fullsized":"2024-04-22T07:12:56.483369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.168057+0000","last_change":"2024-04-22T07:12:54.142002+0000","last_active":"2024-04-22T07:12:56.168057+0000","last_peered":"2024-04-22T07:12:56.168057+0000","last_clean":"2024-04-22T07:12:56.168057+0000","last_became_active":"2024-04-22T07:12:54.141246+0000","last_became_peered":"2024-04-22T07:12:54.141246+0000","last_unstale":"2024-04-22T07:12:56.168057+0000","last_undegraded":"2024-04-22T07:12:56.168057+0000","last_fullsized":"2024-04-22T07:12:56.168057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:28:10.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163467+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:12:56.163467+0000","last_peered":"2024-04-22T07:12:56.163467+0000","last_clean":"2024-04-22T07:12:56.163467+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:12:56.163467+0000","last_undegraded":"2024-04-22T07:12:56.163467+0000","last_fullsized":"2024-04-22T07:12:56.163467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.179142+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:12:57.179142+0000","last_peered":"2024-04-22T07:12:57.179142+0000","last_clean":"2024-04-22T07:12:57.179142+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:12:57.179142+0000","last_undegraded":"2024-04-22T07:12:57.179142+0000","last_fullsized":"2024-04-22T07:12:57.179142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.167780+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:12:56.167780+0000","last_peered":"2024-04-22T07:12:56.167780+0000","last_clean":"2024-04-22T07:12:56.167780+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:12:56.167780+0000","last_undegraded":"2024-04-22T07:12:56.167780+0000","last_fullsized":"2024-04-22T07:12:56.167780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.185841+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:12:58.185841+0000","last_peered":"2024-04-22T07:12:58.185841+0000","last_clean":"2024-04-22T07:12:58.185841+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:12:58.185841+0000","last_undegraded":"2024-04-22T07:12:58.185841+0000","last_fullsized":"2024-04-22T07:12:58.185841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163290+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:12:56.163290+0000","last_peered":"2024-04-22T07:12:56.163290+0000","last_clean":"2024-04-22T07:12:56.163290+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:12:56.163290+0000","last_undegraded":"2024-04-22T07:12:56.163290+0000","last_fullsized":"2024-04-22T07:12:56.163290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.185202+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:12:58.185202+0000","last_peered":"2024-04-22T07:12:58.185202+0000","last_clean":"2024-04-22T07:12:58.185202+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:12:58.185202+0000","last_undegraded":"2024-04-22T07:12:58.185202+0000","last_fullsized":"2024-04-22T07:12:58.185202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162480+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:12:56.162480+0000","last_peered":"2024-04-22T07:12:56.162480+0000","last_clean":"2024-04-22T07:12:56.162480+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:12:56.162480+0000","last_undegraded":"2024-04-22T07:12:56.162480+0000","last_fullsized":"2024-04-22T07:12:56.162480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162674+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:12:56.162674+0000","last_peered":"2024-04-22T07:12:56.162674+0000","last_clean":"2024-04-22T07:12:56.162674+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:12:56.162674+0000","last_undegraded":"2024-04-22T07:12:56.162674+0000","last_fullsized":"2024-04-22T07:12:56.162674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.165682+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:12:56.165682+0000","last_peered":"2024-04-22T07:12:56.165682+0000","last_clean":"2024-04-22T07:12:56.165682+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:12:56.165682+0000","last_undegraded":"2024-04-22T07:12:56.165682+0000","last_fullsized":"2024-04-22T07:12:56.165682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.180960+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:12:57.180960+0000","last_peered":"2024-04-22T07:12:57.180960+0000","last_clean":"2024-04-22T07:12:57.180960+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:12:57.180960+0000","last_undegraded":"2024-04-22T07:12:57.180960+0000","last_fullsized":"2024-04-22T07:12:57.180960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483096+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:12:56.483096+0000","last_peered":"2024-04-22T07:12:56.483096+0000","last_clean":"2024-04-22T07:12:56.483096+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:12:56.483096+0000","last_undegraded":"2024-04-22T07:12:56.483096+0000","last_fullsized":"2024-04-22T07:12:56.483096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178382+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:12:57.178382+0000","last_peered":"2024-04-22T07:12:57.178382+0000","last_clean":"2024-04-22T07:12:57.178382+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:12:57.178382+0000","last_undegraded":"2024-04-22T07:12:57.178382+0000","last_fullsized":"2024-04-22T07:12:57.178382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482817+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:12:56.482817+0000","last_peered":"2024-04-22T07:12:56.482817+0000","last_clean":"2024-04-22T07:12:56.482817+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:12:56.482817+0000","last_undegraded":"2024-04-22T07:12:56.482817+0000","last_fullsized":"2024-04-22T07:12:56.482817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163502+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:12:56.163502+0000","last_peered":"2024-04-22T07:12:56.163502+0000","last_clean":"2024-04-22T07:12:56.163502+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:12:56.163502+0000","last_undegraded":"2024-04-22T07:12:56.163502+0000","last_fullsized":"2024-04-22T07:12:56.163502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483521+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:12:56.483521+0000","last_peered":"2024-04-22T07:12:56.483521+0000","last_clean":"2024-04-22T07:12:56.483521+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:12:56.483521+0000","last_undegraded":"2024-04-22T07:12:56.483521+0000","last_fullsized":"2024-04-22T07:12:56.483521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.340493+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:12:58.340493+0000","last_peered":"2024-04-22T07:12:58.340493+0000","last_clean":"2024-04-22T07:12:58.340493+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:12:58.340493+0000","last_undegraded":"2024-04-22T07:12:58.340493+0000","last_fullsized":"2024-04-22T07:12:58.340493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.168040+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:12:56.168040+0000","last_peered":"2024-04-22T07:12:56.168040+0000","last_clean":"2024-04-22T07:12:56.168040+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:12:56.168040+0000","last_undegraded":"2024-04-22T07:12:56.168040+0000","last_fullsized":"2024-04-22T07:12:56.168040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.184548+0000","last_change":"2024-04-22T07:12:54.140783+0000","last_active":"2024-04-22T07:12:58.184548+0000","last_peered":"2024-04-22T07:12:58.184548+0000","last_clean":"2024-04-22T07:12:58.184548+0000","last_became_active":"2024-04-22T07:12:54.140402+0000","last_became_peered":"2024-04-22T07:12:54.140402+0000","last_unstale":"2024-04-22T07:12:58.184548+0000","last_undegraded":"2024-04-22T07:12:58.184548+0000","last_fullsized":"2024-04-22T07:12:58.184548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:26:22.838953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.481973+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:12:56.481973+0000","last_peered":"2024-04-22T07:12:56.481973+0000","last_clean":"2024-04-22T07:12:56.481973+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:12:56.481973+0000","last_undegraded":"2024-04-22T07:12:56.481973+0000","last_fullsized":"2024-04-22T07:12:56.481973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178963+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:12:57.178963+0000","last_peered":"2024-04-22T07:12:57.178963+0000","last_clean":"2024-04-22T07:12:57.178963+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:12:57.178963+0000","last_undegraded":"2024-04-22T07:12:57.178963+0000","last_fullsized":"2024-04-22T07:12:57.178963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482524+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:12:56.482524+0000","last_peered":"2024-04-22T07:12:56.482524+0000","last_clean":"2024-04-22T07:12:56.482524+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:12:56.482524+0000","last_undegraded":"2024-04-22T07:12:56.482524+0000","last_fullsized":"2024-04-22T07:12:56.482524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.341973+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:12:58.341973+0000","last_peered":"2024-04-22T07:12:58.341973+0000","last_clean":"2024-04-22T07:12:58.341973+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:12:58.341973+0000","last_undegraded":"2024-04-22T07:12:58.341973+0000","last_fullsized":"2024-04-22T07:12:58.341973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482092+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:12:56.482092+0000","last_peered":"2024-04-22T07:12:56.482092+0000","last_clean":"2024-04-22T07:12:56.482092+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:12:56.482092+0000","last_undegraded":"2024-04-22T07:12:56.482092+0000","last_fullsized":"2024-04-22T07:12:56.482092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.186446+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:12:58.186446+0000","last_peered":"2024-04-22T07:12:58.186446+0000","last_clean":"2024-04-22T07:12:58.186446+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:12:58.186446+0000","last_undegraded":"2024-04-22T07:12:58.186446+0000","last_fullsized":"2024-04-22T07:12:58.186446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162895+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:12:56.162895+0000","last_peered":"2024-04-22T07:12:56.162895+0000","last_clean":"2024-04-22T07:12:56.162895+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:12:56.162895+0000","last_undegraded":"2024-04-22T07:12:56.162895+0000","last_fullsized":"2024-04-22T07:12:56.162895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.168175+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:12:56.168175+0000","last_peered":"2024-04-22T07:12:56.168175+0000","last_clean":"2024-04-22T07:12:56.168175+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:12:56.168175+0000","last_undegraded":"2024-04-22T07:12:56.168175+0000","last_fullsized":"2024-04-22T07:12:56.168175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525369+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:12:56.525369+0000","last_peered":"2024-04-22T07:12:56.525369+0000","last_clean":"2024-04-22T07:12:56.525369+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:12:56.525369+0000","last_undegraded":"2024-04-22T07:12:56.525369+0000","last_fullsized":"2024-04-22T07:12:56.525369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.165705+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:12:56.165705+0000","last_peered":"2024-04-22T07:12:56.165705+0000","last_clean":"2024-04-22T07:12:56.165705+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:12:56.165705+0000","last_undegraded":"2024-04-22T07:12:56.165705+0000","last_fullsized":"2024-04-22T07:12:56.165705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482735+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:12:56.482735+0000","last_peered":"2024-04-22T07:12:56.482735+0000","last_clean":"2024-04-22T07:12:56.482735+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:12:56.482735+0000","last_undegraded":"2024-04-22T07:12:56.482735+0000","last_fullsized":"2024-04-22T07:12:56.482735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483168+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:12:56.483168+0000","last_peered":"2024-04-22T07:12:56.483168+0000","last_clean":"2024-04-22T07:12:56.483168+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:12:56.483168+0000","last_undegraded":"2024-04-22T07:12:56.483168+0000","last_fullsized":"2024-04-22T07:12:56.483168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483778+0000","last_change":"2024-04-22T07:12:55.175229+0000","last_active":"2024-04-22T07:12:56.483778+0000","last_peered":"2024-04-22T07:12:56.483778+0000","last_clean":"2024-04-22T07:12:56.483778+0000","last_became_active":"2024-04-22T07:12:55.174799+0000","last_became_peered":"2024-04-22T07:12:55.174799+0000","last_unstale":"2024-04-22T07:12:56.483778+0000","last_undegraded":"2024-04-22T07:12:56.483778+0000","last_fullsized":"2024-04-22T07:12:56.483778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:18:26.650199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163041+0000","last_change":"2024-04-22T07:12:55.174700+0000","last_active":"2024-04-22T07:12:56.163041+0000","last_peered":"2024-04-22T07:12:56.163041+0000","last_clean":"2024-04-22T07:12:56.163041+0000","last_became_active":"2024-04-22T07:12:55.173133+0000","last_became_peered":"2024-04-22T07:12:55.173133+0000","last_unstale":"2024-04-22T07:12:56.163041+0000","last_undegraded":"2024-04-22T07:12:56.163041+0000","last_fullsized":"2024-04-22T07:12:56.163041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:34:54.914711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.186460+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:12:58.186460+0000","last_peered":"2024-04-22T07:12:58.186460+0000","last_clean":"2024-04-22T07:12:58.186460+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:12:58.186460+0000","last_undegraded":"2024-04-22T07:12:58.186460+0000","last_fullsized":"2024-04-22T07:12:58.186460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162650+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:12:56.162650+0000","last_peered":"2024-04-22T07:12:56.162650+0000","last_clean":"2024-04-22T07:12:56.162650+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:12:56.162650+0000","last_undegraded":"2024-04-22T07:12:56.162650+0000","last_fullsized":"2024-04-22T07:12:56.162650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.338900+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:12:58.338900+0000","last_peered":"2024-04-22T07:12:58.338900+0000","last_clean":"2024-04-22T07:12:58.338900+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:12:58.338900+0000","last_undegraded":"2024-04-22T07:12:58.338900+0000","last_fullsized":"2024-04-22T07:12:58.338900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482108+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:12:56.482108+0000","last_peered":"2024-04-22T07:12:56.482108+0000","last_clean":"2024-04-22T07:12:56.482108+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:12:56.482108+0000","last_undegraded":"2024-04-22T07:12:56.482108+0000","last_fullsized":"2024-04-22T07:12:56.482108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.481795+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:12:56.481795+0000","last_peered":"2024-04-22T07:12:56.481795+0000","last_clean":"2024-04-22T07:12:56.481795+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:12:56.481795+0000","last_undegraded":"2024-04-22T07:12:56.481795+0000","last_fullsized":"2024-04-22T07:12:56.481795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.183826+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:12:58.183826+0000","last_peered":"2024-04-22T07:12:58.183826+0000","last_clean":"2024-04-22T07:12:58.183826+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:12:58.183826+0000","last_undegraded":"2024-04-22T07:12:58.183826+0000","last_fullsized":"2024-04-22T07:12:58.183826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178533+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:12:57.178533+0000","last_peered":"2024-04-22T07:12:57.178533+0000","last_clean":"2024-04-22T07:12:57.178533+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:12:57.178533+0000","last_undegraded":"2024-04-22T07:12:57.178533+0000","last_fullsized":"2024-04-22T07:12:57.178533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482079+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:12:56.482079+0000","last_peered":"2024-04-22T07:12:56.482079+0000","last_clean":"2024-04-22T07:12:56.482079+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:12:56.482079+0000","last_undegraded":"2024-04-22T07:12:56.482079+0000","last_fullsized":"2024-04-22T07:12:56.482079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.183492+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:12:58.183492+0000","last_peered":"2024-04-22T07:12:58.183492+0000","last_clean":"2024-04-22T07:12:58.183492+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:12:58.183492+0000","last_undegraded":"2024-04-22T07:12:58.183492+0000","last_fullsized":"2024-04-22T07:12:58.183492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163114+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:12:56.163114+0000","last_peered":"2024-04-22T07:12:56.163114+0000","last_clean":"2024-04-22T07:12:56.163114+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:12:56.163114+0000","last_undegraded":"2024-04-22T07:12:56.163114+0000","last_fullsized":"2024-04-22T07:12:56.163114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.177639+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:12:57.177639+0000","last_peered":"2024-04-22T07:12:57.177639+0000","last_clean":"2024-04-22T07:12:57.177639+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:12:57.177639+0000","last_undegraded":"2024-04-22T07:12:57.177639+0000","last_fullsized":"2024-04-22T07:12:57.177639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482683+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:12:56.482683+0000","last_peered":"2024-04-22T07:12:56.482683+0000","last_clean":"2024-04-22T07:12:56.482683+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:12:56.482683+0000","last_undegraded":"2024-04-22T07:12:56.482683+0000","last_fullsized":"2024-04-22T07:12:56.482683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.177785+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:12:57.177785+0000","last_peered":"2024-04-22T07:12:57.177785+0000","last_clean":"2024-04-22T07:12:57.177785+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:12:57.177785+0000","last_undegraded":"2024-04-22T07:12:57.177785+0000","last_fullsized":"2024-04-22T07:12:57.177785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.167986+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:12:56.167986+0000","last_peered":"2024-04-22T07:12:56.167986+0000","last_clean":"2024-04-22T07:12:56.167986+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:12:56.167986+0000","last_undegraded":"2024-04-22T07:12:56.167986+0000","last_fullsized":"2024-04-22T07:12:56.167986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.180554+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:12:57.180554+0000","last_peered":"2024-04-22T07:12:57.180554+0000","last_clean":"2024-04-22T07:12:57.180554+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:12:57.180554+0000","last_undegraded":"2024-04-22T07:12:57.180554+0000","last_fullsized":"2024-04-22T07:12:57.180554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525128+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:12:56.525128+0000","last_peered":"2024-04-22T07:12:56.525128+0000","last_clean":"2024-04-22T07:12:56.525128+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:12:56.525128+0000","last_undegraded":"2024-04-22T07:12:56.525128+0000","last_fullsized":"2024-04-22T07:12:56.525128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482369+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:12:56.482369+0000","last_peered":"2024-04-22T07:12:56.482369+0000","last_clean":"2024-04-22T07:12:56.482369+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:12:56.482369+0000","last_undegraded":"2024-04-22T07:12:56.482369+0000","last_fullsized":"2024-04-22T07:12:56.482369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162743+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:12:56.162743+0000","last_peered":"2024-04-22T07:12:56.162743+0000","last_clean":"2024-04-22T07:12:56.162743+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:12:56.162743+0000","last_undegraded":"2024-04-22T07:12:56.162743+0000","last_fullsized":"2024-04-22T07:12:56.162743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:57.178775+0000","last_change":"2024-04-22T07:12:54.147007+0000","last_active":"2024-04-22T07:12:57.178775+0000","last_peered":"2024-04-22T07:12:57.178775+0000","last_clean":"2024-04-22T07:12:57.178775+0000","last_became_active":"2024-04-22T07:12:54.146668+0000","last_became_peered":"2024-04-22T07:12:54.146668+0000","last_unstale":"2024-04-22T07:12:57.178775+0000","last_undegraded":"2024-04-22T07:12:57.178775+0000","last_fullsized":"2024-04-22T07:12:57.178775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:28.076768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483277+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:12:56.483277+0000","last_peered":"2024-04-22T07:12:56.483277+0000","last_clean":"2024-04-22T07:12:56.483277+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:12:56.483277+0000","last_undegraded":"2024-04-22T07:12:56.483277+0000","last_fullsized":"2024-04-22T07:12:56.483277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:58.186320+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:12:58.186320+0000","last_peered":"2024-04-22T07:12:58.186320+0000","last_clean":"2024-04-22T07:12:58.186320+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:12:58.186320+0000","last_undegraded":"2024-04-22T07:12:58.186320+0000","last_fullsized":"2024-04-22T07:12:58.186320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483670+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:12:56.483670+0000","last_peered":"2024-04-22T07:12:56.483670+0000","last_clean":"2024-04-22T07:12:56.483670+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:12:56.483670+0000","last_undegraded":"2024-04-22T07:12:56.483670+0000","last_fullsized":"2024-04-22T07:12:56.483670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163299+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:12:56.163299+0000","last_peered":"2024-04-22T07:12:56.163299+0000","last_clean":"2024-04-22T07:12:56.163299+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:12:56.163299+0000","last_undegraded":"2024-04-22T07:12:56.163299+0000","last_fullsized":"2024-04-22T07:12:56.163299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482714+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:12:56.482714+0000","last_peered":"2024-04-22T07:12:56.482714+0000","last_clean":"2024-04-22T07:12:56.482714+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:12:56.482714+0000","last_undegraded":"2024-04-22T07:12:56.482714+0000","last_fullsized":"2024-04-22T07:12:56.482714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.481974+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:12:56.481974+0000","last_peered":"2024-04-22T07:12:56.481974+0000","last_clean":"2024-04-22T07:12:56.481974+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:12:56.481974+0000","last_undegraded":"2024-04-22T07:12:56.481974+0000","last_fullsized":"2024-04-22T07:12:56.481974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482735+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:12:56.482735+0000","last_peered":"2024-04-22T07:12:56.482735+0000","last_clean":"2024-04-22T07:12:56.482735+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:12:56.482735+0000","last_undegraded":"2024-04-22T07:12:56.482735+0000","last_fullsized":"2024-04-22T07:12:56.482735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525239+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:12:56.525239+0000","last_peered":"2024-04-22T07:12:56.525239+0000","last_clean":"2024-04-22T07:12:56.525239+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:12:56.525239+0000","last_undegraded":"2024-04-22T07:12:56.525239+0000","last_fullsized":"2024-04-22T07:12:56.525239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482542+0000","last_change":"2024-04-22T07:12:55.169456+0000","last_active":"2024-04-22T07:12:56.482542+0000","last_peered":"2024-04-22T07:12:56.482542+0000","last_clean":"2024-04-22T07:12:56.482542+0000","last_became_active":"2024-04-22T07:12:55.169000+0000","last_became_peered":"2024-04-22T07:12:55.169000+0000","last_unstale":"2024-04-22T07:12:56.482542+0000","last_undegraded":"2024-04-22T07:12:56.482542+0000","last_fullsized":"2024-04-22T07:12:56.482542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:48:06.709915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482419+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:12:56.482419+0000","last_peered":"2024-04-22T07:12:56.482419+0000","last_clean":"2024-04-22T07:12:56.482419+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:12:56.482419+0000","last_undegraded":"2024-04-22T07:12:56.482419+0000","last_fullsized":"2024-04-22T07:12:56.482419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482537+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:12:56.482537+0000","last_peered":"2024-04-22T07:12:56.482537+0000","last_clean":"2024-04-22T07:12:56.482537+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:12:56.482537+0000","last_undegraded":"2024-04-22T07:12:56.482537+0000","last_fullsized":"2024-04-22T07:12:56.482537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482051+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:12:56.482051+0000","last_peered":"2024-04-22T07:12:56.482051+0000","last_clean":"2024-04-22T07:12:56.482051+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:12:56.482051+0000","last_undegraded":"2024-04-22T07:12:56.482051+0000","last_fullsized":"2024-04-22T07:12:56.482051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525380+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:12:56.525380+0000","last_peered":"2024-04-22T07:12:56.525380+0000","last_clean":"2024-04-22T07:12:56.525380+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:12:56.525380+0000","last_undegraded":"2024-04-22T07:12:56.525380+0000","last_fullsized":"2024-04-22T07:12:56.525380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482760+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:12:56.482760+0000","last_peered":"2024-04-22T07:12:56.482760+0000","last_clean":"2024-04-22T07:12:56.482760+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:12:56.482760+0000","last_undegraded":"2024-04-22T07:12:56.482760+0000","last_fullsized":"2024-04-22T07:12:56.482760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163089+0000","last_change":"2024-04-22T07:12:55.169344+0000","last_active":"2024-04-22T07:12:56.163089+0000","last_peered":"2024-04-22T07:12:56.163089+0000","last_clean":"2024-04-22T07:12:56.163089+0000","last_became_active":"2024-04-22T07:12:55.168995+0000","last_became_peered":"2024-04-22T07:12:55.168995+0000","last_unstale":"2024-04-22T07:12:56.163089+0000","last_undegraded":"2024-04-22T07:12:56.163089+0000","last_fullsized":"2024-04-22T07:12:56.163089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:59:47.870749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.163292+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:12:56.163292+0000","last_peered":"2024-04-22T07:12:56.163292+0000","last_clean":"2024-04-22T07:12:56.163292+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:12:56.163292+0000","last_undegraded":"2024-04-22T07:12:56.163292+0000","last_fullsized":"2024-04-22T07:12:56.163292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.162446+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:12:56.162446+0000","last_peered":"2024-04-22T07:12:56.162446+0000","last_clean":"2024-04-22T07:12:56.162446+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:12:56.162446+0000","last_undegraded":"2024-04-22T07:12:56.162446+0000","last_fullsized":"2024-04-22T07:12:56.162446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.483238+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:12:56.483238+0000","last_peered":"2024-04-22T07:12:56.483238+0000","last_clean":"2024-04-22T07:12:56.483238+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:12:56.483238+0000","last_undegraded":"2024-04-22T07:12:56.483238+0000","last_fullsized":"2024-04-22T07:12:56.483238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.482108+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:12:56.482108+0000","last_peered":"2024-04-22T07:12:56.482108+0000","last_clean":"2024-04-22T07:12:56.482108+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:12:56.482108+0000","last_undegraded":"2024-04-22T07:12:56.482108+0000","last_fullsized":"2024-04-22T07:12:56.482108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T07:12:56.525296+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:12:56.525296+0000","last_peered":"2024-04-22T07:12:56.525296+0000","last_clean":"2024-04-22T07:12:56.525296+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:12:56.525296+0000","last_undegraded":"2024-04-22T07:12:56.525296+0000","last_fullsized":"2024-04-22T07:12:56.525296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":21,"num_read_kb":19,"num_write":124,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":44110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":444461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1037437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":444505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712472,"statfs":{"total":95995035648,"available":95961571328,"internally_reserved":0,"allocated":1536000,"data_stored":1036039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":464774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":446841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":444163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1035723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":30056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:13:02.365 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-22T07:13:02.365 INFO:tasks.cephadm:Setting up client nodes... 2024-04-22T07:13:02.366 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.5 KiB/s wr, 20 op/s 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3520669087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:13:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3645052780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.5 KiB/s wr, 20 op/s 2024-04-22T07:13:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:13:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T07:13:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3520669087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:13:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3645052780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:03.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:03 smithi113 ceph-mon[27933]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:13:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:03 smithi133 ceph-mon[31198]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:13:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:03 smithi154 ceph-mon[31862]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 -- 172.21.15.113:0/2696736602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 msgr2=0x7fd91c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 --2- 172.21.15.113:0/2696736602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c103020 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0099c0 tx=0x7fd90c009cd0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 -- 172.21.15.113:0/2696736602 shutdown_connections 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 --2- 172.21.15.113:0/2696736602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c103560 0x7fd91c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 --2- 172.21.15.113:0/2696736602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c1019c0 0x7fd91c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 --2- 172.21.15.113:0/2696736602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c103020 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 -- 172.21.15.113:0/2696736602 >> 172.21.15.113:0/2696736602 conn(0x7fd91c0fcfa0 msgr2=0x7fd91c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 -- 172.21.15.113:0/2696736602 shutdown_connections 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.146+0000 7fd923f8f700 1 -- 172.21.15.113:0/2696736602 wait complete. 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 Processor -- start 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 -- start start 2024-04-22T07:13:04.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c1019c0 0x7fd91c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c103560 0x7fd91c1a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d690 con 0x7fd91c102bc0 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d510 con 0x7fd91c103560 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd923f8f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10d810 con 0x7fd91c1019c0 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd92152a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd92152a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43226/0 (socket says 172.21.15.113:43226) 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd92152a700 1 -- 172.21.15.113:0/3115729440 learned_addr learned my addr 172.21.15.113:0/3115729440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.147+0000 7fd921d2b700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c1019c0 0x7fd91c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92252c700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c103560 0x7fd91c1a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92152a700 1 -- 172.21.15.113:0/3115729440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c1019c0 msgr2=0x7fd91c06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92152a700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c1019c0 0x7fd91c06bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92152a700 1 -- 172.21.15.113:0/3115729440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c103560 msgr2=0x7fd91c1a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:04.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92152a700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c103560 0x7fd91c1a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92152a700 1 -- 172.21.15.113:0/3115729440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c1a0920 con 0x7fd91c102bc0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd921d2b700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c1019c0 0x7fd91c06bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd92152a700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c19da00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009f50 tx=0x7fd90c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c027070 con 0x7fd91c102bc0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd90c022470 con 0x7fd91c102bc0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c1a0bb0 con 0x7fd91c102bc0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00f6e0 con 0x7fd91c102bc0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.148+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c1a1000 con 0x7fd91c102bc0 2024-04-22T07:13:04.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.149+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c02df80 con 0x7fd91c102bc0 2024-04-22T07:13:04.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.150+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd90c02a070 con 0x7fd91c102bc0 2024-04-22T07:13:04.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.151+0000 7fd912ffd700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd9080a3d80 0x7fd9080a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:04.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.151+0000 7fd921d2b700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd9080a3d80 0x7fd9080a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:04.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.151+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd90c0d12e0 con 0x7fd91c102bc0 2024-04-22T07:13:04.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.151+0000 7fd921d2b700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd9080a3d80 0x7fd9080a6240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd91c102a20 tx=0x7fd918008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:04.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.155+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd90c013070 con 0x7fd91c102bc0 2024-04-22T07:13:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.440+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd91c02e1d0 con 0x7fd91c102bc0 2024-04-22T07:13:04.438 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:04 smithi113 ceph-mon[27933]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:13:04.438 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:04 smithi113 ceph-mon[27933]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-04-22T07:13:04.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.442+0000 7fd912ffd700 1 -- 172.21.15.113:0/3115729440 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd90c09dda0 con 0x7fd91c102bc0 2024-04-22T07:13:04.440 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-22T07:13:04.440 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAADiZmnKhQGhAAZWqTNLif23IEuy5Iwhpjkg== 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd9080a3d80 msgr2=0x7fd9080a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd9080a3d80 0x7fd9080a6240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd91c102a20 tx=0x7fd918008040 comp rx=0 tx=0).stop 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 msgr2=0x7fd91c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c19da00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009f50 tx=0x7fd90c0036f0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 shutdown_connections 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd9080a3d80 0x7fd9080a6240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c103560 0x7fd91c1a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd91c1019c0 0x7fd91c06bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 --2- 172.21.15.113:0/3115729440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd91c102bc0 0x7fd91c19da00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 >> 172.21.15.113:0/3115729440 conn(0x7fd91c0fcfa0 msgr2=0x7fd91c105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 shutdown_connections 2024-04-22T07:13:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:04.445+0000 7fd923f8f700 1 -- 172.21.15.113:0/3115729440 wait complete. 2024-04-22T07:13:04.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:04 smithi133 ceph-mon[31198]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:13:04.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:04 smithi133 ceph-mon[31198]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-04-22T07:13:04.467 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:04 smithi154 ceph-mon[31862]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:13:04.467 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:04 smithi154 ceph-mon[31862]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-04-22T07:13:05.085 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:13:05.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-22T07:13:05.086 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T07:13:05.121 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T07:13:05.121 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-22T07:13:05.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-22T07:13:05.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:05 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3115729440' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T07:13:05.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:05 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3115729440' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T07:13:05.417 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:05.489 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:05 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3115729440' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T07:13:05.489 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:05 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3115729440' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T07:13:05.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:05 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3115729440' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T07:13:05.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:05 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3115729440' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T07:13:06.521 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:06 smithi113 ceph-mon[27933]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.1 KiB/s wr, 20 op/s 2024-04-22T07:13:06.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:06 smithi133 ceph-mon[31198]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.1 KiB/s wr, 20 op/s 2024-04-22T07:13:06.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:06 smithi154 ceph-mon[31862]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.1 KiB/s wr, 20 op/s 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 -- 172.21.15.113:0/3480948824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c102a90 msgr2=0x7ff93c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 --2- 172.21.15.113:0/3480948824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c102a90 0x7ff93c102f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff930009990 tx=0x7ff930009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 -- 172.21.15.113:0/3480948824 shutdown_connections 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 --2- 172.21.15.113:0/3480948824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93c103450 0x7ff93c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 --2- 172.21.15.113:0/3480948824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff93c108a50 0x7ff93c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 --2- 172.21.15.113:0/3480948824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c102a90 0x7ff93c102f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 -- 172.21.15.113:0/3480948824 >> 172.21.15.113:0/3480948824 conn(0x7ff93c079ba0 msgr2=0x7ff93c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 -- 172.21.15.113:0/3480948824 shutdown_connections 2024-04-22T07:13:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 -- 172.21.15.113:0/3480948824 wait complete. 2024-04-22T07:13:07.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 Processor -- start 2024-04-22T07:13:07.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.002+0000 7ff94296d700 1 -- start start 2024-04-22T07:13:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff94296d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff93c102a90 0x7ff93c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff94296d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 0x7ff93c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff94296d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93c108a50 0x7ff93c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff94296d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c112da0 con 0x7ff93c103450 2024-04-22T07:13:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff94296d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c112c20 con 0x7ff93c102a90 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff94296d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c112f20 con 0x7ff93c108a50 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 0x7ff93c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 0x7ff93c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43804/0 (socket says 172.21.15.113:43804) 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 -- 172.21.15.113:0/792125508 learned_addr learned my addr 172.21.15.113:0/792125508 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93bfff700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff93c102a90 0x7ff93c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff940f0a700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93c108a50 0x7ff93c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 -- 172.21.15.113:0/792125508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93c108a50 msgr2=0x7ff93c19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93c108a50 0x7ff93c19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 -- 172.21.15.113:0/792125508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff93c102a90 msgr2=0x7ff93c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff93c102a90 0x7ff93c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 -- 172.21.15.113:0/792125508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c19e2a0 con 0x7ff93c103450 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff93b7fe700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 0x7ff93c1997d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff930009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.003+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930027070 con 0x7ff93c103450 2024-04-22T07:13:07.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.004+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff930003fb0 con 0x7ff93c103450 2024-04-22T07:13:07.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.004+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930022e10 con 0x7ff93c103450 2024-04-22T07:13:07.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.004+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c19e530 con 0x7ff93c103450 2024-04-22T07:13:07.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.004+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c1a5e10 con 0x7ff93c103450 2024-04-22T07:13:07.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.004+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93c104180 con 0x7ff93c103450 2024-04-22T07:13:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.006+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff93002a070 con 0x7ff93c103450 2024-04-22T07:13:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.006+0000 7ff9397fa700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9240a3cd0 0x7ff9240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.006+0000 7ff93bfff700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9240a3cd0 0x7ff9240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.006+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff9300d1610 con 0x7ff93c103450 2024-04-22T07:13:07.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.007+0000 7ff93bfff700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9240a3cd0 0x7ff9240a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007be0 tx=0x7ff92c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.010+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff930031070 con 0x7ff93c103450 2024-04-22T07:13:07.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.272+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff93c02df80 con 0x7ff93c103450 2024-04-22T07:13:07.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.276+0000 7ff9397fa700 1 -- 172.21.15.113:0/792125508 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265004 (secure 0 0 0) 0x7ff930013070 con 0x7ff93c103450 2024-04-22T07:13:07.278 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:07.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9240a3cd0 msgr2=0x7ff9240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:07.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9240a3cd0 0x7ff9240a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007be0 tx=0x7ff92c0073d0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 msgr2=0x7ff93c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:07.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 0x7ff93c1997d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff930009f50 comp rx=0 tx=0).stop 2024-04-22T07:13:07.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 shutdown_connections 2024-04-22T07:13:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9240a3cd0 0x7ff9240a6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff93c108a50 0x7ff93c19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff93c102a90 0x7ff93c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 --2- 172.21.15.113:0/792125508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c103450 0x7ff93c1997d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 >> 172.21.15.113:0/792125508 conn(0x7ff93c079ba0 msgr2=0x7ff93c1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 shutdown_connections 2024-04-22T07:13:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:07.279+0000 7ff94296d700 1 -- 172.21.15.113:0/792125508 wait complete. 2024-04-22T07:13:07.890 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":610623186},{"type":"v1","addr":"172.21.15.113:6801","nonce":610623186}]},"active_addr":"172.21.15.113:6801/610623186","active_change":"2024-04-22T07:05:54.991272+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.113:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1824685298}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3098383517}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3223226441}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3347479031}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":2834116794}]}]} 2024-04-22T07:13:07.896 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-22T07:13:07.896 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-22T07:13:07.897 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T07:13:08.136 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:08.465 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:08 smithi113 ceph-mon[27933]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.3 KiB/s wr, 18 op/s 2024-04-22T07:13:08.465 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:08 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/792125508' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T07:13:08.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:08 smithi133 ceph-mon[31198]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.3 KiB/s wr, 18 op/s 2024-04-22T07:13:08.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:08 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/792125508' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T07:13:08.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:08 smithi154 ceph-mon[31862]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.3 KiB/s wr, 18 op/s 2024-04-22T07:13:08.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:08 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/792125508' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T07:13:09.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 -- 172.21.15.113:0/1706923564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c101470 msgr2=0x7fd10c1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:09.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 --2- 172.21.15.113:0/1706923564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c101470 0x7fd10c1018f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009990 tx=0x7fd0fc009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 -- 172.21.15.113:0/1706923564 shutdown_connections 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 --2- 172.21.15.113:0/1706923564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd10c101e30 0x7fd10c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 --2- 172.21.15.113:0/1706923564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd10c108c20 0x7fd10c109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 --2- 172.21.15.113:0/1706923564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c101470 0x7fd10c1018f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 -- 172.21.15.113:0/1706923564 >> 172.21.15.113:0/1706923564 conn(0x7fd10c076bb0 msgr2=0x7fd10c076fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 -- 172.21.15.113:0/1706923564 shutdown_connections 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.658+0000 7fd113d39700 1 -- 172.21.15.113:0/1706923564 wait complete. 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 Processor -- start 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 -- start start 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd10c101470 0x7fd10c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd10c101e30 0x7fd10c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 0x7fd10c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c112d70 con 0x7fd10c108c20 2024-04-22T07:13:09.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c112bf0 con 0x7fd10c101470 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd113d39700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c112ef0 con 0x7fd10c101e30 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 0x7fd10c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 0x7fd10c1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43822/0 (socket says 172.21.15.113:43822) 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 -- 172.21.15.113:0/1870692094 learned_addr learned my addr 172.21.15.113:0/1870692094 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1112d4700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd10c101e30 0x7fd10c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd111ad5700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd10c101470 0x7fd10c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 -- 172.21.15.113:0/1870692094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd10c101e30 msgr2=0x7fd10c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd10c101e30 0x7fd10c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 -- 172.21.15.113:0/1870692094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd10c101470 msgr2=0x7fd10c102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd10c101470 0x7fd10c102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 -- 172.21.15.113:0/1870692094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c1a76b0 con 0x7fd10c108c20 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.659+0000 7fd1122d6700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 0x7fd10c1030d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd10400ada0 tx=0x7fd10400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.660+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104017020 con 0x7fd10c108c20 2024-04-22T07:13:09.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.660+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd10400cda0 con 0x7fd10c108c20 2024-04-22T07:13:09.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.660+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10c1a79a0 con 0x7fd10c108c20 2024-04-22T07:13:09.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.660+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104005310 con 0x7fd10c108c20 2024-04-22T07:13:09.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.660+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10c1a7eb0 con 0x7fd10c108c20 2024-04-22T07:13:09.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.661+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10c069730 con 0x7fd10c108c20 2024-04-22T07:13:09.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.662+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd10400c8d0 con 0x7fd10c108c20 2024-04-22T07:13:09.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.662+0000 7fd102ffd700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd0f80a3cd0 0x7fd0f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:09.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.663+0000 7fd111ad5700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd0f80a3cd0 0x7fd0f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:09.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.663+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd1040cfb20 con 0x7fd10c108c20 2024-04-22T07:13:09.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.663+0000 7fd111ad5700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd0f80a3cd0 0x7fd0f80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd108005950 tx=0x7fd10800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:09.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.667+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd104021020 con 0x7fd10c108c20 2024-04-22T07:13:09.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.867+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd10c02e140 con 0x7fd10c108c20 2024-04-22T07:13:09.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.869+0000 7fd102ffd700 1 -- 172.21.15.113:0/1870692094 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22267 (secure 0 0 0) 0x7fd10400cbc0 con 0x7fd10c108c20 2024-04-22T07:13:09.867 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:09.867 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":86,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:56.154646+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd0f80a3cd0 msgr2=0x7fd0f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd0f80a3cd0 0x7fd0f80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd108005950 tx=0x7fd10800b410 comp rx=0 tx=0).stop 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 msgr2=0x7fd10c1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 0x7fd10c1030d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd10400ada0 tx=0x7fd10400c330 comp rx=0 tx=0).stop 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 shutdown_connections 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd0f80a3cd0 0x7fd0f80a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd10c108c20 0x7fd10c1030d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd10c101470 0x7fd10c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 --2- 172.21.15.113:0/1870692094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd10c101e30 0x7fd10c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 >> 172.21.15.113:0/1870692094 conn(0x7fd10c076bb0 msgr2=0x7fd10c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:09.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 shutdown_connections 2024-04-22T07:13:09.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:09.871+0000 7fd113d39700 1 -- 172.21.15.113:0/1870692094 wait complete. 2024-04-22T07:13:10.468 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:10 smithi113 ceph-mon[27933]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 19 op/s 2024-04-22T07:13:10.468 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:10 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1870692094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:10.496 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-22T07:13:10.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T07:13:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:10 smithi133 ceph-mon[31198]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 19 op/s 2024-04-22T07:13:10.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:10 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1870692094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:10.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:10 smithi154 ceph-mon[31862]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 19 op/s 2024-04-22T07:13:10.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:10 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1870692094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:10.736 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 -- 172.21.15.113:0/3768231861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288106990 msgr2=0x7fd288106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 --2- 172.21.15.113:0/3768231861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288106990 0x7fd288106d70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd278009990 tx=0x7fd278009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 -- 172.21.15.113:0/3768231861 shutdown_connections 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 --2- 172.21.15.113:0/3768231861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2881026e0 0x7fd28810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 --2- 172.21.15.113:0/3768231861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288106990 0x7fd288106d70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 --2- 172.21.15.113:0/3768231861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288101d20 0x7fd2881021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 -- 172.21.15.113:0/3768231861 >> 172.21.15.113:0/3768231861 conn(0x7fd2880788f0 msgr2=0x7fd288078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 -- 172.21.15.113:0/3768231861 shutdown_connections 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 -- 172.21.15.113:0/3768231861 wait complete. 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.172+0000 7fd28d668700 1 Processor -- start 2024-04-22T07:13:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 -- start start 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 0x7fd28819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2881026e0 0x7fd28819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288106990 0x7fd2881a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111cf0 con 0x7fd288101d20 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111b70 con 0x7fd288106990 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd28d668700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd288111e70 con 0x7fd2881026e0 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 0x7fd28819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 0x7fd28819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43836/0 (socket says 172.21.15.113:43836) 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 -- 172.21.15.113:0/2463716524 learned_addr learned my addr 172.21.15.113:0/2463716524 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd2867fc700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2881026e0 0x7fd28819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd2877fe700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288106990 0x7fd2881a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 -- 172.21.15.113:0/2463716524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2881026e0 msgr2=0x7fd28819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2881026e0 0x7fd28819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 -- 172.21.15.113:0/2463716524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288106990 msgr2=0x7fd2881a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288106990 0x7fd2881a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 -- 172.21.15.113:0/2463716524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2881a5730 con 0x7fd288101d20 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd2877fe700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288106990 0x7fd2881a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.173+0000 7fd286ffd700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 0x7fd28819a170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd2780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.174+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278027070 con 0x7fd288101d20 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.174+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2881a5900 con 0x7fd288101d20 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.174+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd2780040f0 con 0x7fd288101d20 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.174+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27800f460 con 0x7fd288101d20 2024-04-22T07:13:12.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.174+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a5d50 con 0x7fd288101d20 2024-04-22T07:13:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.175+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd288103080 con 0x7fd288101d20 2024-04-22T07:13:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.176+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd27802a070 con 0x7fd288101d20 2024-04-22T07:13:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.176+0000 7fd26ffff700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd2700a3cd0 0x7fd2700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.176+0000 7fd2867fc700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd2700a3cd0 0x7fd2700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.176+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd2780d12d0 con 0x7fd288101d20 2024-04-22T07:13:12.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.177+0000 7fd2867fc700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd2700a3cd0 0x7fd2700a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd288079110 tx=0x7fd27c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:12.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.181+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd27805eff0 con 0x7fd288101d20 2024-04-22T07:13:12.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:12 smithi113 ceph-mon[27933]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-04-22T07:13:12.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.383+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28802e140 con 0x7fd288101d20 2024-04-22T07:13:12.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.384+0000 7fd26ffff700 1 -- 172.21.15.113:0/2463716524 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22267 (secure 0 0 0) 0x7fd278013070 con 0x7fd288101d20 2024-04-22T07:13:12.383 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:13:12.386 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":86,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:12:56.154646+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:13:12.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.386+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd2700a3cd0 msgr2=0x7fd2700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.386+0000 7fd28d668700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd2700a3cd0 0x7fd2700a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd288079110 tx=0x7fd27c009190 comp rx=0 tx=0).stop 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.386+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 msgr2=0x7fd28819a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.386+0000 7fd28d668700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 0x7fd28819a170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd278000c00 tx=0x7fd2780047b0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.386+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 shutdown_connections 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.386+0000 7fd28d668700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd2700a3cd0 0x7fd2700a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.387+0000 7fd28d668700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd288106990 0x7fd2881a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.387+0000 7fd28d668700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd288101d20 0x7fd28819a170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.387+0000 7fd28d668700 1 --2- 172.21.15.113:0/2463716524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2881026e0 0x7fd28819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.387+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 >> 172.21.15.113:0/2463716524 conn(0x7fd2880788f0 msgr2=0x7fd288100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:12.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.387+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 shutdown_connections 2024-04-22T07:13:12.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:12.387+0000 7fd28d668700 1 -- 172.21.15.113:0/2463716524 wait complete. 2024-04-22T07:13:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:12 smithi133 ceph-mon[31198]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-04-22T07:13:12.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:12 smithi154 ceph-mon[31862]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2024-04-22T07:13:13.020 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-22T07:13:13.020 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-22T07:13:13.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-22T07:13:13.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-22T07:13:13.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-22T07:13:13.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-22T07:13:13.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-22T07:13:13.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-22T07:13:13.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-22T07:13:13.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-22T07:13:13.024 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-22T07:13:13.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-22T07:13:13.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:13 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2463716524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:13 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2463716524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:13.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:13 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2463716524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:13:13.745 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.745 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.748 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.753 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.753 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.757 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.762 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.762 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.763 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.766 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.772 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:14.462 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:14 smithi113 ceph-mon[27933]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-22T07:13:14.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:14 smithi133 ceph-mon[31198]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-22T07:13:14.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:14 smithi154 ceph-mon[31862]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-22T07:13:16.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:16 smithi113 ceph-mon[27933]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-22T07:13:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:16 smithi133 ceph-mon[31198]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-22T07:13:16.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:16 smithi154 ceph-mon[31862]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-22T07:13:18.468 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:18 smithi113 ceph-mon[27933]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-22T07:13:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:18 smithi133 ceph-mon[31198]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-22T07:13:18.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:18 smithi154 ceph-mon[31862]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-22T07:13:20.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:20 smithi113 ceph-mon[27933]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-22T07:13:20.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:20 smithi133 ceph-mon[31198]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-22T07:13:20.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:20 smithi154 ceph-mon[31862]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-22T07:13:22.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:22 smithi133 ceph-mon[31198]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-22T07:13:22.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:22 smithi154 ceph-mon[31862]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-22T07:13:22.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:22 smithi113 ceph-mon[27933]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 -- 172.21.15.113:0/1273895265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55681034d0 msgr2=0x7f5568103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 --2- 172.21.15.113:0/1273895265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55681034d0 0x7f5568103950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f555c009990 tx=0x7f555c009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 -- 172.21.15.113:0/1273895265 shutdown_connections 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 --2- 172.21.15.113:0/1273895265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568103e90 0x7f556810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 --2- 172.21.15.113:0/1273895265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5568101d90 0x7f5568102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 --2- 172.21.15.113:0/1273895265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55681034d0 0x7f5568103950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 -- 172.21.15.113:0/1273895265 >> 172.21.15.113:0/1273895265 conn(0x7f55680779e0 msgr2=0x7f5568077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:24.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 -- 172.21.15.113:0/1273895265 shutdown_connections 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 -- 172.21.15.113:0/1273895265 wait complete. 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.884+0000 7f556de29700 1 Processor -- start 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 -- start start 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5568101d90 0x7f5568074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55681034d0 0x7f556806c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5568103e90 0x7f556806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111dd0 con 0x7f5568103e90 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111c50 con 0x7f55681034d0 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f556de29700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111f50 con 0x7f5568101d90 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5568103e90 0x7f556806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5568103e90 0x7f556806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57662/0 (socket says 172.21.15.113:57662) 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 -- 172.21.15.113:0/2952845055 learned_addr learned my addr 172.21.15.113:0/2952845055 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f55677fe700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5568101d90 0x7f5568074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5566ffd700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55681034d0 0x7f556806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 -- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5568101d90 msgr2=0x7f5568074510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5568101d90 0x7f5568074510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 -- 172.21.15.113:0/2952845055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55681034d0 msgr2=0x7f556806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55681034d0 0x7f556806c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f5567fff700 1 -- 172.21.15.113:0/2952845055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556806d2b0 con 0x7f5568103e90 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.885+0000 7f55677fe700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5568101d90 0x7f5568074510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.886+0000 7f5567fff700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5568103e90 0x7f556806ca70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5560007f10 tx=0x7f556000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.886+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560016030 con 0x7f5568103e90 2024-04-22T07:13:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.886+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556806b230 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.886+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5560004bb0 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.886+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560005020 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.886+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556806b770 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.887+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f556806bca0 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.888+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5560012460 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.888+0000 7f5564ff9700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f55500a3cd0 0x7f55500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f55677fe700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f55500a3cd0 0x7f55500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55600d2ca0 con 0x7f5568103e90 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f5564ff9700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] conn(0x7f55500ad4b0 0x7f55500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 --> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5560019e70 con 0x7f55500ad4b0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f55600d3100 con 0x7f5568103e90 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f55677fe700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f55500a3cd0 0x7f55500a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.889+0000 7f5566ffd700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] conn(0x7f55500ad4b0 0x7f55500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.890+0000 7f5566ffd700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] conn(0x7f55500ad4b0 0x7f55500af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.891+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== osd.5 v2:172.21.15.154:6808/3828843548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5560019e70 con 0x7f55500ad4b0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.908+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 --> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f556806c0a0 con 0x7f55500ad4b0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f5564ff9700 1 -- 172.21.15.113:0/2952845055 <== osd.5 v2:172.21.15.154:6808/3828843548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f556806c0a0 con 0x7f55500ad4b0 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] conn(0x7f55500ad4b0 msgr2=0x7f55500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] conn(0x7f55500ad4b0 0x7f55500af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f55500a3cd0 msgr2=0x7f55500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f55500a3cd0 0x7f55500a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558005e20 comp rx=0 tx=0).stop 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5568103e90 msgr2=0x7f556806ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:24.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 --2- 172.21.15.113:0/2952845055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5568103e90 0x7f556806ca70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5560007f10 tx=0x7f556000f5c0 comp rx=0 tx=0).stop 2024-04-22T07:13:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f5567fff700 1 -- 172.21.15.113:0/2952845055 reap_dead start 2024-04-22T07:13:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 shutdown_connections 2024-04-22T07:13:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 >> 172.21.15.113:0/2952845055 conn(0x7f55680779e0 msgr2=0x7f5568106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 shutdown_connections 2024-04-22T07:13:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:21.909+0000 7f556de29700 1 -- 172.21.15.113:0/2952845055 wait complete. 2024-04-22T07:13:24.487 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:24 smithi113 ceph-mon[27933]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-22T07:13:24.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:24 smithi133 ceph-mon[31198]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-22T07:13:24.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:24 smithi154 ceph-mon[31862]: pgmap v300: 97 pgs: 97 active+clean; 635 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-22T07:13:24.817 INFO:teuthology.orchestra.run.smithi113.stdout:163208757296 2024-04-22T07:13:24.817 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 -- 172.21.15.113:0/4161420452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c102300 msgr2=0x7f584c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 --2- 172.21.15.113:0/4161420452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c102300 0x7f584c10e7f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f584400b0a0 tx=0x7f584400b3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 -- 172.21.15.113:0/4161420452 shutdown_connections 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 --2- 172.21.15.113:0/4161420452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c102300 0x7f584c10e7f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 --2- 172.21.15.113:0/4161420452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f584c107990 0x7f584c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 --2- 172.21.15.113:0/4161420452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f584c101940 0x7f584c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 -- 172.21.15.113:0/4161420452 >> 172.21.15.113:0/4161420452 conn(0x7f584c0788f0 msgr2=0x7f584c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 -- 172.21.15.113:0/4161420452 shutdown_connections 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.418+0000 7f58546a7700 1 -- 172.21.15.113:0/4161420452 wait complete. 2024-04-22T07:13:25.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 Processor -- start 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 -- start start 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f584c101940 0x7f584c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f584c102300 0x7f584c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c107990 0x7f584c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f584c111c90 con 0x7f584c107990 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f584c111b10 con 0x7f584c102300 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f58546a7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f584c111e10 con 0x7f584c101940 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.419+0000 7f5852c44700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c107990 0x7f584c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c107990 0x7f584c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57706/0 (socket says 172.21.15.113:57706) 2024-04-22T07:13:25.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 -- 172.21.15.113:0/1999375978 learned_addr learned my addr 172.21.15.113:0/1999375978 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:25.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5851c42700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f584c102300 0x7f584c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:25.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852443700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f584c101940 0x7f584c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:25.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 -- 172.21.15.113:0/1999375978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f584c101940 msgr2=0x7f584c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:25.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f584c101940 0x7f584c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 -- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f584c102300 msgr2=0x7f584c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:25.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f584c102300 0x7f584c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 -- 172.21.15.113:0/1999375978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c19d0b0 con 0x7f584c107990 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852443700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f584c101940 0x7f584c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5851c42700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f584c102300 0x7f584c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f5852c44700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c107990 0x7f584c19c970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f584400b830 tx=0x7f5844007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844021020 con 0x7f584c107990 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f584401db40 con 0x7f584c107990 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c19d340 con 0x7f584c107990 2024-04-22T07:13:25.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584401bcf0 con 0x7f584c107990 2024-04-22T07:13:25.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.420+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c1a4c20 con 0x7f584c107990 2024-04-22T07:13:25.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.421+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f584c0631d0 con 0x7f584c107990 2024-04-22T07:13:25.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.422+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5844024020 con 0x7f584c107990 2024-04-22T07:13:25.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f58437fe700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58380a3cd0 0x7f58380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f5852443700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58380a3cd0 0x7f58380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f584400f020 con 0x7f584c107990 2024-04-22T07:13:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f58437fe700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] conn(0x7f58380ad4b0 0x7f58380af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 --> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f584400aea0 con 0x7f58380ad4b0 2024-04-22T07:13:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f58440d21c0 con 0x7f584c107990 2024-04-22T07:13:25.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f5852443700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58380a3cd0 0x7f58380a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f583c007900 tx=0x7f583c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:25.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.423+0000 7f5851c42700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] conn(0x7f58380ad4b0 0x7f58380af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:25.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.424+0000 7f5851c42700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] conn(0x7f58380ad4b0 0x7f58380af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:25.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.425+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== osd.4 v2:172.21.15.133:6808/2027989696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f584400aea0 con 0x7f58380ad4b0 2024-04-22T07:13:25.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.444+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 --> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f584c063650 con 0x7f58380ad4b0 2024-04-22T07:13:25.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58437fe700 1 -- 172.21.15.113:0/1999375978 <== osd.4 v2:172.21.15.133:6808/2027989696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f584c063650 con 0x7f58380ad4b0 2024-04-22T07:13:25.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] conn(0x7f58380ad4b0 msgr2=0x7f58380af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] conn(0x7f58380ad4b0 0x7f58380af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58380a3cd0 msgr2=0x7f58380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58380a3cd0 0x7f58380a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f583c007900 tx=0x7f583c008040 comp rx=0 tx=0).stop 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c107990 msgr2=0x7f584c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 --2- 172.21.15.113:0/1999375978 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c107990 0x7f584c19c970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f584400b830 tx=0x7f5844007b40 comp rx=0 tx=0).stop 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f5852c44700 1 -- 172.21.15.113:0/1999375978 reap_dead start 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 shutdown_connections 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 >> 172.21.15.113:0/1999375978 conn(0x7f584c0788f0 msgr2=0x7f584c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 shutdown_connections 2024-04-22T07:13:25.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.445+0000 7f58546a7700 1 -- 172.21.15.113:0/1999375978 wait complete. 2024-04-22T07:13:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 -- 172.21.15.113:0/3462363382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc101940 msgr2=0x7fcdfc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 --2- 172.21.15.113:0/3462363382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc101940 0x7fcdfc101dc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009990 tx=0x7fcdf0009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 -- 172.21.15.113:0/3462363382 shutdown_connections 2024-04-22T07:13:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 --2- 172.21.15.113:0/3462363382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdfc102300 0x7fcdfc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 --2- 172.21.15.113:0/3462363382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdfc107990 0x7fcdfc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 --2- 172.21.15.113:0/3462363382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc101940 0x7fcdfc101dc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 -- 172.21.15.113:0/3462363382 >> 172.21.15.113:0/3462363382 conn(0x7fcdfc0788f0 msgr2=0x7fcdfc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:26.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 -- 172.21.15.113:0/3462363382 shutdown_connections 2024-04-22T07:13:26.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.007+0000 7fce00f19700 1 -- 172.21.15.113:0/3462363382 wait complete. 2024-04-22T07:13:26.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 Processor -- start 2024-04-22T07:13:26.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 -- start start 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdfc101940 0x7fcdfc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc102300 0x7fcdfc1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdfc107990 0x7fcdfc19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc111c60 con 0x7fcdfc102300 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc111ae0 con 0x7fcdfc101940 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fce00f19700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc111de0 con 0x7fcdfc107990 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fcdf9d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc102300 0x7fcdfc1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fcdf9d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc102300 0x7fcdfc1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38482/0 (socket says 172.21.15.113:38482) 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.008+0000 7fcdf9d9b700 1 -- 172.21.15.113:0/3174657464 learned_addr learned my addr 172.21.15.113:0/3174657464 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdfa59c700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdfc101940 0x7fcdfc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdf9d9b700 1 -- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdfc107990 msgr2=0x7fcdfc19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdf9d9b700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdfc107990 0x7fcdfc19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdf9d9b700 1 -- 172.21.15.113:0/3174657464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdfc101940 msgr2=0x7fcdfc198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdf9d9b700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdfc101940 0x7fcdfc198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdf9d9b700 1 -- 172.21.15.113:0/3174657464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc19d0a0 con 0x7fcdfc102300 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdfa59c700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdfc101940 0x7fcdfc198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdf9d9b700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc102300 0x7fcdfc1985d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf002a070 con 0x7fcdfc102300 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc19d330 con 0x7fcdfc102300 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcdf000f460 con 0x7fcdfc102300 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0025680 con 0x7fcdfc102300 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.009+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc06a720 con 0x7fcdfc102300 2024-04-22T07:13:26.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.010+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcdfc06ac70 con 0x7fcdfc102300 2024-04-22T07:13:26.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.011+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcdf002d070 con 0x7fcdfc102300 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdeb7fe700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcde40a3cd0 0x7fcde40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcdf00d3870 con 0x7fcdfc102300 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdfa59c700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcde40a3cd0 0x7fcde40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdeb7fe700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] conn(0x7fcde40ad4b0 0x7fcde40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 --> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcdf0001d30 con 0x7fcde40ad4b0 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fcdf00d3d50 con 0x7fcdfc102300 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.012+0000 7fcdfa59c700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcde40a3cd0 0x7fcde40a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00baa0 tx=0x7fcdec005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.013+0000 7fcdfad9d700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] conn(0x7fcde40ad4b0 0x7fcde40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.013+0000 7fcdfad9d700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] conn(0x7fcde40ad4b0 0x7fcde40af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:26.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.014+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== osd.11 v2:172.21.15.154:6824/3813220428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcdf0001d30 con 0x7fcde40ad4b0 2024-04-22T07:13:26.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.030+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 --> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcdfc06b070 con 0x7fcde40ad4b0 2024-04-22T07:13:26.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.030+0000 7fcdeb7fe700 1 -- 172.21.15.113:0/3174657464 <== osd.11 v2:172.21.15.154:6824/3813220428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcdfc06b070 con 0x7fcde40ad4b0 2024-04-22T07:13:26.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] conn(0x7fcde40ad4b0 msgr2=0x7fcde40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] conn(0x7fcde40ad4b0 0x7fcde40af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcde40a3cd0 msgr2=0x7fcde40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcde40a3cd0 0x7fcde40a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00baa0 tx=0x7fcdec005db0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc102300 msgr2=0x7fcdfc1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 --2- 172.21.15.113:0/3174657464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdfc102300 0x7fcdfc1985d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf000b610 comp rx=0 tx=0).stop 2024-04-22T07:13:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fcdfad9d700 1 -- 172.21.15.113:0/3174657464 reap_dead start 2024-04-22T07:13:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 shutdown_connections 2024-04-22T07:13:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 >> 172.21.15.113:0/3174657464 conn(0x7fcdfc0788f0 msgr2=0x7fcdfc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 shutdown_connections 2024-04-22T07:13:26.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:26.031+0000 7fce00f19700 1 -- 172.21.15.113:0/3174657464 wait complete. 2024-04-22T07:13:26.438 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:26 smithi113 ceph-mon[27933]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-22T07:13:26.461 INFO:teuthology.orchestra.run.smithi113.stdout:141733920823 2024-04-22T07:13:26.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-22T07:13:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:26 smithi133 ceph-mon[31198]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-22T07:13:26.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:26 smithi154 ceph-mon[31862]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-22T07:13:26.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 -- 172.21.15.113:0/1911507023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 msgr2=0x7fd610108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 --2- 172.21.15.113:0/1911507023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd610108e00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd600009990 tx=0x7fd600009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 -- 172.21.15.113:0/1911507023 shutdown_connections 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 --2- 172.21.15.113:0/1911507023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 0x7fd61010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 --2- 172.21.15.113:0/1911507023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd610108e00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 --2- 172.21.15.113:0/1911507023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61006afe0 0x7fd61006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 -- 172.21.15.113:0/1911507023 >> 172.21.15.113:0/1911507023 conn(0x7fd6100788b0 msgr2=0x7fd610078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 -- 172.21.15.113:0/1911507023 shutdown_connections 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.228+0000 7fd6156b3700 1 -- 172.21.15.113:0/1911507023 wait complete. 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.229+0000 7fd6156b3700 1 Processor -- start 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.229+0000 7fd6156b3700 1 -- start start 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd6156b3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61006afe0 0x7fd6101992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd6156b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 0x7fd610199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd6156b3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd61019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd6156b3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd610112d80 con 0x7fd610108a20 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd6156b3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd610112c00 con 0x7fd61006afe0 2024-04-22T07:13:26.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd6156b3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd610112f00 con 0x7fd61006b980 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd61019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60e7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 0x7fd610199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60e7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 0x7fd610199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:43868/0 (socket says 172.21.15.113:43868) 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd61019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57684/0 (socket says 172.21.15.113:57684) 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60e7fc700 1 -- 172.21.15.113:0/3704769704 learned_addr learned my addr 172.21.15.113:0/3704769704 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60effd700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61006afe0 0x7fd6101992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 -- 172.21.15.113:0/3704769704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 msgr2=0x7fd610199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 0x7fd610199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 -- 172.21.15.113:0/3704769704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61006afe0 msgr2=0x7fd6101992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61006afe0 0x7fd6101992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 -- 172.21.15.113:0/3704769704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61019e300 con 0x7fd610108a20 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60e7fc700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd61006b980 0x7fd610199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd60f7fe700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd61019dbc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd60800be20 tx=0x7fd608009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.230+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608019070 con 0x7fd610108a20 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.231+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61019e5f0 con 0x7fd610108a20 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.231+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd60800cb90 con 0x7fd610108a20 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.231+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6101a5e70 con 0x7fd610108a20 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.231+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080125a0 con 0x7fd610108a20 2024-04-22T07:13:26.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.231+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6100630d0 con 0x7fd610108a20 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.233+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd60801c070 con 0x7fd610108a20 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.233+0000 7fd5f7fff700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd5f80a3d50 0x7fd5f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.233+0000 7fd60effd700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd5f80a3d50 0x7fd5f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd6080d02e0 con 0x7fd610108a20 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd5f7fff700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] conn(0x7fd5f80ad530 0x7fd5f80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 --> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd608007e50 con 0x7fd5f80ad530 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd6080d07c0 con 0x7fd610108a20 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd60e7fc700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] conn(0x7fd5f80ad530 0x7fd5f80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd60effd700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd5f80a3d50 0x7fd5f80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b2e0 tx=0x7fd600005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.234+0000 7fd60e7fc700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] conn(0x7fd5f80ad530 0x7fd5f80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.235+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== osd.3 v2:172.21.15.113:6810/1273649373 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd608007e50 con 0x7fd5f80ad530 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.254+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 --> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6100634d0 con 0x7fd5f80ad530 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.254+0000 7fd5f7fff700 1 -- 172.21.15.113:0/3704769704 <== osd.3 v2:172.21.15.113:6810/1273649373 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6100634d0 con 0x7fd5f80ad530 2024-04-22T07:13:26.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] conn(0x7fd5f80ad530 msgr2=0x7fd5f80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] conn(0x7fd5f80ad530 0x7fd5f80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd5f80a3d50 msgr2=0x7fd5f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd5f80a3d50 0x7fd5f80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b2e0 tx=0x7fd600005960 comp rx=0 tx=0).stop 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 msgr2=0x7fd61019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 --2- 172.21.15.113:0/3704769704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd610108a20 0x7fd61019dbc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd60800be20 tx=0x7fd608009ed0 comp rx=0 tx=0).stop 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd60f7fe700 1 -- 172.21.15.113:0/3704769704 reap_dead start 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 shutdown_connections 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 >> 172.21.15.113:0/3704769704 conn(0x7fd6100788b0 msgr2=0x7fd610100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 shutdown_connections 2024-04-22T07:13:26.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:25.255+0000 7fd6156b3700 1 -- 172.21.15.113:0/3704769704 wait complete. 2024-04-22T07:13:28.473 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:28 smithi113 ceph-mon[27933]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:28 smithi133 ceph-mon[31198]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:28.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:28 smithi154 ceph-mon[31862]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:29.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 -- 172.21.15.113:0/2136087692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 msgr2=0x7f49bc1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 --2- 172.21.15.113:0/2136087692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1021a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009990 tx=0x7f49ac009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 -- 172.21.15.113:0/2136087692 shutdown_connections 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 --2- 172.21.15.113:0/2136087692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49bc1026e0 0x7f49bc10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 --2- 172.21.15.113:0/2136087692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49bc106990 0x7f49bc106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 --2- 172.21.15.113:0/2136087692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1021a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 -- 172.21.15.113:0/2136087692 >> 172.21.15.113:0/2136087692 conn(0x7f49bc0788f0 msgr2=0x7f49bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 -- 172.21.15.113:0/2136087692 shutdown_connections 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 -- 172.21.15.113:0/2136087692 wait complete. 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 Processor -- start 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.159+0000 7f49c4692700 1 -- start start 2024-04-22T07:13:29.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c4692700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c4692700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49bc1026e0 0x7f49bc103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c4692700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49bc106990 0x7f49bc104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c4692700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111c90 con 0x7f49bc101d20 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c4692700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111b10 con 0x7f49bc106990 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c4692700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111e10 con 0x7f49bc1026e0 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38502/0 (socket says 172.21.15.113:38502) 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 -- 172.21.15.113:0/990564726 learned_addr learned my addr 172.21.15.113:0/990564726 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c1c2d700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49bc1026e0 0x7f49bc103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c2c2f700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49bc106990 0x7f49bc104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 -- 172.21.15.113:0/990564726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49bc1026e0 msgr2=0x7f49bc103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49bc1026e0 0x7f49bc103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 -- 172.21.15.113:0/990564726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49bc106990 msgr2=0x7f49bc104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49bc106990 0x7f49bc104060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 -- 172.21.15.113:0/990564726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc1a7770 con 0x7f49bc101d20 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c2c2f700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49bc106990 0x7f49bc104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49c242e700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1035e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f49b800c920 tx=0x7f49b800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.160+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8007b00 con 0x7f49bc101d20 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.161+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f49b8004d00 con 0x7f49bc101d20 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.161+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8005650 con 0x7f49bc101d20 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.161+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc1a7a60 con 0x7f49bc101d20 2024-04-22T07:13:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.161+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc1a7fa0 con 0x7f49bc101d20 2024-04-22T07:13:29.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.161+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49bc0630d0 con 0x7f49bc101d20 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.163+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f49b8007ca0 con 0x7f49bc101d20 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.163+0000 7f49b37fe700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f49a80a3d50 0x7f49a80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.163+0000 7f49c1c2d700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f49a80a3d50 0x7f49a80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f49b80d03b0 con 0x7f49bc101d20 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49b37fe700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] conn(0x7f49a80ad530 0x7f49a80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 --> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49a80b0000 con 0x7f49a80ad530 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f49b80d0890 con 0x7f49bc101d20 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49c2c2f700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] conn(0x7f49a80ad530 0x7f49a80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49c1c2d700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f49a80a3d50 0x7f49a80a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0790a0 tx=0x7f49ac00b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.164+0000 7f49c2c2f700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] conn(0x7f49a80ad530 0x7f49a80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:29.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.165+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== osd.9 v2:172.21.15.113:6826/744209950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49a80b0000 con 0x7f49a80ad530 2024-04-22T07:13:29.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 --> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49bc063550 con 0x7f49a80ad530 2024-04-22T07:13:29.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49b37fe700 1 -- 172.21.15.113:0/990564726 <== osd.9 v2:172.21.15.113:6826/744209950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49bc063550 con 0x7f49a80ad530 2024-04-22T07:13:29.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] conn(0x7f49a80ad530 msgr2=0x7f49a80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] conn(0x7f49a80ad530 0x7f49a80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f49a80a3d50 msgr2=0x7f49a80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f49a80a3d50 0x7f49a80a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0790a0 tx=0x7f49ac00b560 comp rx=0 tx=0).stop 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 msgr2=0x7f49bc1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c4692700 1 --2- 172.21.15.113:0/990564726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49bc101d20 0x7f49bc1035e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f49b800c920 tx=0x7f49b800cc30 comp rx=0 tx=0).stop 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.181+0000 7f49c2c2f700 1 -- 172.21.15.113:0/990564726 reap_dead start 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.182+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 shutdown_connections 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.182+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 >> 172.21.15.113:0/990564726 conn(0x7f49bc0788f0 msgr2=0x7f49bc06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.182+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 shutdown_connections 2024-04-22T07:13:29.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:29.182+0000 7f49c4692700 1 -- 172.21.15.113:0/990564726 wait complete. 2024-04-22T07:13:30.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:30 smithi113 ceph-mon[27933]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-22T07:13:30.451 INFO:teuthology.orchestra.run.smithi113.stdout:292057776158 2024-04-22T07:13:30.452 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-22T07:13:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:30 smithi133 ceph-mon[31198]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-22T07:13:30.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:30 smithi154 ceph-mon[31862]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 -- 172.21.15.113:0/1620004628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 msgr2=0x7f568410f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 --2- 172.21.15.113:0/1620004628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f568410f980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f567c00b0a0 tx=0x7f567c00b3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 -- 172.21.15.113:0/1620004628 shutdown_connections 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 --2- 172.21.15.113:0/1620004628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f568410f980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 --2- 172.21.15.113:0/1620004628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5684108ab0 0x7f5684108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 --2- 172.21.15.113:0/1620004628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5684102ac0 0x7f5684102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 -- 172.21.15.113:0/1620004628 >> 172.21.15.113:0/1620004628 conn(0x7f5684079bf0 msgr2=0x7f568407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 -- 172.21.15.113:0/1620004628 shutdown_connections 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 -- 172.21.15.113:0/1620004628 wait complete. 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 Processor -- start 2024-04-22T07:13:30.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.726+0000 7f568be81700 1 -- start start 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5684102ac0 0x7f5684074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f5684075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5684108ab0 0x7f56840757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5684112dd0 con 0x7f5684103480 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5684112c50 con 0x7f5684108ab0 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5684112f50 con 0x7f5684102ac0 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f5684075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f5684075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38516/0 (socket says 172.21.15.113:38516) 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 -- 172.21.15.113:0/3357172178 learned_addr learned my addr 172.21.15.113:0/3357172178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f5689c1d700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5684102ac0 0x7f5684074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568a41e700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5684108ab0 0x7f56840757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 -- 172.21.15.113:0/3357172178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5684102ac0 msgr2=0x7f5684074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:30.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5684102ac0 0x7f5684074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 -- 172.21.15.113:0/3357172178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5684108ab0 msgr2=0x7f56840757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5684108ab0 0x7f56840757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 -- 172.21.15.113:0/3357172178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56841a76f0 con 0x7f5684103480 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568a41e700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5684108ab0 0x7f56840757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568941c700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f5684075280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f567400d980 tx=0x7f567400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674017070 con 0x7f5684103480 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.727+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56841a79e0 con 0x7f5684103480 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.728+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841a7f20 con 0x7f5684103480 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.728+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5674009c00 con 0x7f5684103480 2024-04-22T07:13:30.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.728+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674010da0 con 0x7f5684103480 2024-04-22T07:13:30.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.728+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5668001030 con 0x7f5684103480 2024-04-22T07:13:30.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.730+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5674010460 con 0x7f5684103480 2024-04-22T07:13:30.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.730+0000 7f567affd700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56700a3d50 0x7f56700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f5689c1d700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56700a3d50 0x7f56700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f56740d0710 con 0x7f5684103480 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f567affd700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] conn(0x7f56700ad530 0x7f56700af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 --> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56700b0000 con 0x7f56700ad530 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f56740d0bf0 con 0x7f5684103480 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f5689c1d700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56700a3d50 0x7f56700a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5680005950 tx=0x7f568000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f568a41e700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] conn(0x7f56700ad530 0x7f56700af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.731+0000 7f568a41e700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] conn(0x7f56700ad530 0x7f56700af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.732+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== osd.6 v2:172.21.15.113:6818/499388993 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56700b0000 con 0x7f56700ad530 2024-04-22T07:13:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 --> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5668002e60 con 0x7f56700ad530 2024-04-22T07:13:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f567affd700 1 -- 172.21.15.113:0/3357172178 <== osd.6 v2:172.21.15.113:6818/499388993 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5668002e60 con 0x7f56700ad530 2024-04-22T07:13:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] conn(0x7f56700ad530 msgr2=0x7f56700af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] conn(0x7f56700ad530 0x7f56700af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:30.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56700a3d50 msgr2=0x7f56700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56700a3d50 0x7f56700a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5680005950 tx=0x7f568000a300 comp rx=0 tx=0).stop 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 msgr2=0x7f5684075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 --2- 172.21.15.113:0/3357172178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5684103480 0x7f5684075280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f567400d980 tx=0x7f567400dc90 comp rx=0 tx=0).stop 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568a41e700 1 -- 172.21.15.113:0/3357172178 reap_dead start 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 shutdown_connections 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.749+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 >> 172.21.15.113:0/3357172178 conn(0x7f5684079bf0 msgr2=0x7f5684100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.750+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 shutdown_connections 2024-04-22T07:13:30.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:30.750+0000 7f568be81700 1 -- 172.21.15.113:0/3357172178 wait complete. 2024-04-22T07:13:31.142 INFO:teuthology.orchestra.run.smithi113.stdout:107374182460 2024-04-22T07:13:31.142 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-22T07:13:31.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 -- 172.21.15.113:0/1926269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 msgr2=0x7f14dc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:31.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 --2- 172.21.15.113:0/1926269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc108e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009990 tx=0x7f14d8009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 -- 172.21.15.113:0/1926269440 shutdown_connections 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 --2- 172.21.15.113:0/1926269440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc06b980 0x7f14dc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 --2- 172.21.15.113:0/1926269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc108e00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 --2- 172.21.15.113:0/1926269440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06afe0 0x7f14dc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 -- 172.21.15.113:0/1926269440 >> 172.21.15.113:0/1926269440 conn(0x7f14dc0788b0 msgr2=0x7f14dc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 -- 172.21.15.113:0/1926269440 shutdown_connections 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.340+0000 7f14e377c700 1 -- 172.21.15.113:0/1926269440 wait complete. 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 Processor -- start 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 -- start start 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06afe0 0x7f14dc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc06b980 0x7f14dc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc112d40 con 0x7f14dc108a20 2024-04-22T07:13:31.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc112bc0 con 0x7f14dc06b980 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e377c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc112ec0 con 0x7f14dc06afe0 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e1d19700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e1d19700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38526/0 (socket says 172.21.15.113:38526) 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e1d19700 1 -- 172.21.15.113:0/2817529248 learned_addr learned my addr 172.21.15.113:0/2817529248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e1518700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06afe0 0x7f14dc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.341+0000 7f14e0d17700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc06b980 0x7f14dc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1d19700 1 -- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06afe0 msgr2=0x7f14dc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1d19700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06afe0 0x7f14dc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1d19700 1 -- 172.21.15.113:0/2817529248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc06b980 msgr2=0x7f14dc199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1d19700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc06b980 0x7f14dc199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1d19700 1 -- 172.21.15.113:0/2817529248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc19e1e0 con 0x7f14dc108a20 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1518700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14dc06afe0 0x7f14dc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:13:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e1d19700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc19daa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007ee0 tx=0x7f14d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d400dbd0 con 0x7f14dc108a20 2024-04-22T07:13:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc19e4d0 con 0x7f14dc108a20 2024-04-22T07:13:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1a5d50 con 0x7f14dc108a20 2024-04-22T07:13:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f14d400dd70 con 0x7f14dc108a20 2024-04-22T07:13:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.342+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4005a50 con 0x7f14dc108a20 2024-04-22T07:13:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.343+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14c0001030 con 0x7f14dc108a20 2024-04-22T07:13:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.345+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f14d401a020 con 0x7f14dc108a20 2024-04-22T07:13:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.345+0000 7f14d27fc700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f14c80a3ef0 0x7f14c80a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.345+0000 7f14e1518700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f14c80a3ef0 0x7f14c80a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:31.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.345+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f14d40d00f0 con 0x7f14dc108a20 2024-04-22T07:13:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.346+0000 7f14d27fc700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] conn(0x7f14c80ad6d0 0x7f14c80afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.346+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 --> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14d4005a50 con 0x7f14c80ad6d0 2024-04-22T07:13:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.346+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f14d40d0550 con 0x7f14dc108a20 2024-04-22T07:13:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.346+0000 7f14e1518700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f14c80a3ef0 0x7f14c80a63b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b5c0 tx=0x7f14d8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.346+0000 7f14e0d17700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] conn(0x7f14c80ad6d0 0x7f14c80afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.346+0000 7f14e0d17700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] conn(0x7f14c80ad6d0 0x7f14c80afaf0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:31.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.347+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== osd.8 v2:172.21.15.154:6816/3748797627 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14d4005a50 con 0x7f14c80ad6d0 2024-04-22T07:13:31.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.363+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 --> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14c0002e60 con 0x7f14c80ad6d0 2024-04-22T07:13:31.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14d27fc700 1 -- 172.21.15.113:0/2817529248 <== osd.8 v2:172.21.15.154:6816/3748797627 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f14c0002e60 con 0x7f14c80ad6d0 2024-04-22T07:13:31.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] conn(0x7f14c80ad6d0 msgr2=0x7f14c80afaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] conn(0x7f14c80ad6d0 0x7f14c80afaf0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f14c80a3ef0 msgr2=0x7f14c80a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f14c80a3ef0 0x7f14c80a63b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b5c0 tx=0x7f14d8005960 comp rx=0 tx=0).stop 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 msgr2=0x7f14dc19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 --2- 172.21.15.113:0/2817529248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc108a20 0x7f14dc19daa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007ee0 tx=0x7f14d400d3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e1d19700 1 -- 172.21.15.113:0/2817529248 reap_dead start 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 shutdown_connections 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 >> 172.21.15.113:0/2817529248 conn(0x7f14dc0788b0 msgr2=0x7f14dc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 shutdown_connections 2024-04-22T07:13:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:31.364+0000 7f14e377c700 1 -- 172.21.15.113:0/2817529248 wait complete. 2024-04-22T07:13:31.796 INFO:teuthology.orchestra.run.smithi113.stdout:347892350997 2024-04-22T07:13:31.796 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-22T07:13:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:32 smithi133 ceph-mon[31198]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:32.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:32 smithi154 ceph-mon[31862]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:32 smithi113 ceph-mon[27933]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:32.743 INFO:teuthology.orchestra.run.smithi113.stdout:197568495661 2024-04-22T07:13:32.743 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi113.stdout:257698037796 2024-04-22T07:13:33.415 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-22T07:13:33.548 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:33 smithi113 ceph-mon[27933]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:33 smithi133 ceph-mon[31198]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:33.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:33 smithi154 ceph-mon[31862]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-22T07:13:34.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 -- 172.21.15.113:0/336622315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c06a750 msgr2=0x7f896c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:34.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 --2- 172.21.15.113:0/336622315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c06a750 0x7f896c06abb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8958009990 tx=0x7f8958009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 -- 172.21.15.113:0/336622315 shutdown_connections 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 --2- 172.21.15.113:0/336622315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f896c06b0f0 0x7f896c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 --2- 172.21.15.113:0/336622315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c108a30 0x7f896c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 --2- 172.21.15.113:0/336622315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c06a750 0x7f896c06abb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 -- 172.21.15.113:0/336622315 >> 172.21.15.113:0/336622315 conn(0x7f896c0788f0 msgr2=0x7f896c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 -- 172.21.15.113:0/336622315 shutdown_connections 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.926+0000 7f8972b20700 1 -- 172.21.15.113:0/336622315 wait complete. 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 Processor -- start 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 -- start start 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f896c06a750 0x7f896c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:34.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c06b0f0 0x7f896c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c108a30 0x7f896c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f896c112d90 con 0x7f896c108a30 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f896c112c10 con 0x7f896c06b0f0 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f8972b20700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f896c112f10 con 0x7f896c06a750 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f89710bd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c108a30 0x7f896c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f89710bd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c108a30 0x7f896c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38536/0 (socket says 172.21.15.113:38536) 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f89710bd700 1 -- 172.21.15.113:0/2732463182 learned_addr learned my addr 172.21.15.113:0/2732463182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f89708bc700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f896c06a750 0x7f896c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.927+0000 7f896bfff700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c06b0f0 0x7f896c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f89710bd700 1 -- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f896c06a750 msgr2=0x7f896c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:34.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f89710bd700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f896c06a750 0x7f896c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f89710bd700 1 -- 172.21.15.113:0/2732463182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c06b0f0 msgr2=0x7f896c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f89710bd700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f896c06b0f0 0x7f896c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f89710bd700 1 -- 172.21.15.113:0/2732463182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c19e1f0 con 0x7f896c108a30 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f89710bd700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c108a30 0x7f896c19dab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f896400b970 tx=0x7f896400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896400d860 con 0x7f896c108a30 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f896400e430 con 0x7f896c108a30 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c19e4e0 con 0x7f896c108a30 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964017660 con 0x7f896c108a30 2024-04-22T07:13:34.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.928+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1a5d60 con 0x7f896c108a30 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.929+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f896c063190 con 0x7f896c108a30 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.930+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f896400e5e0 con 0x7f896c108a30 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f8969ffb700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f895c0a3cd0 0x7f895c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f89708bc700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f895c0a3cd0 0x7f895c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f89640d05e0 con 0x7f896c108a30 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f8969ffb700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] conn(0x7f895c0ad4b0 0x7f895c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 --> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f896401de70 con 0x7f895c0ad4b0 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f89640d0ac0 con 0x7f896c108a30 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f89708bc700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f895c0a3cd0 0x7f895c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f896000ac50 tx=0x7f896000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.931+0000 7f896bfff700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] conn(0x7f895c0ad4b0 0x7f895c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.932+0000 7f896bfff700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] conn(0x7f895c0ad4b0 0x7f895c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.933+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== osd.2 v2:172.21.15.154:6800/343492984 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f896401de70 con 0x7f895c0ad4b0 2024-04-22T07:13:34.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.949+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 --> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f896c063610 con 0x7f895c0ad4b0 2024-04-22T07:13:34.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.949+0000 7f8969ffb700 1 -- 172.21.15.113:0/2732463182 <== osd.2 v2:172.21.15.154:6800/343492984 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f896c063610 con 0x7f895c0ad4b0 2024-04-22T07:13:34.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.949+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] conn(0x7f895c0ad4b0 msgr2=0x7f895c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] conn(0x7f895c0ad4b0 0x7f895c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f895c0a3cd0 msgr2=0x7f895c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f895c0a3cd0 0x7f895c0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f896000ac50 tx=0x7f896000a380 comp rx=0 tx=0).stop 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c108a30 msgr2=0x7f896c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 --2- 172.21.15.113:0/2732463182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f896c108a30 0x7f896c19dab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f896400b970 tx=0x7f896400bc80 comp rx=0 tx=0).stop 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f89710bd700 1 -- 172.21.15.113:0/2732463182 reap_dead start 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 shutdown_connections 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 >> 172.21.15.113:0/2732463182 conn(0x7f896c0788f0 msgr2=0x7f896c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 shutdown_connections 2024-04-22T07:13:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:34.950+0000 7f8972b20700 1 -- 172.21.15.113:0/2732463182 wait complete. 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 -- 172.21.15.113:0/4053818032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40106420 msgr2=0x7f4f4010dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 --2- 172.21.15.113:0/4053818032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40106420 0x7f4f4010dff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b0d0 tx=0x7f4f3800b3e0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 -- 172.21.15.113:0/4053818032 shutdown_connections 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 --2- 172.21.15.113:0/4053818032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40106420 0x7f4f4010dff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 --2- 172.21.15.113:0/4053818032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f40101340 0x7f4f40101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 --2- 172.21.15.113:0/4053818032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40101cf0 0x7f4f40105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.537+0000 7f4f46c32700 1 -- 172.21.15.113:0/4053818032 >> 172.21.15.113:0/4053818032 conn(0x7f4f40077f40 msgr2=0x7f4f400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 -- 172.21.15.113:0/4053818032 shutdown_connections 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 -- 172.21.15.113:0/4053818032 wait complete. 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 Processor -- start 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 -- start start 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f40101340 0x7f4f40199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40101cf0 0x7f4f401997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40106420 0x7f4f4019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40111d70 con 0x7f4f40101cf0 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40111bf0 con 0x7f4f40101340 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.538+0000 7f4f46c32700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40111ef0 con 0x7f4f40106420 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40101cf0 0x7f4f401997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40101cf0 0x7f4f401997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38548/0 (socket says 172.21.15.113:38548) 2024-04-22T07:13:35.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 -- 172.21.15.113:0/3984260103 learned_addr learned my addr 172.21.15.113:0/3984260103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f451cf700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40106420 0x7f4f4019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f449ce700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f40101340 0x7f4f40199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 -- 172.21.15.113:0/3984260103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40106420 msgr2=0x7f4f4019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40106420 0x7f4f4019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 -- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f40101340 msgr2=0x7f4f40199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f40101340 0x7f4f40199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 -- 172.21.15.113:0/3984260103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4019e2a0 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f451cf700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f40106420 0x7f4f4019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3ffff700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40101cf0 0x7f4f401997d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00b500 tx=0x7f4f2c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c015020 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4019e590 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4f2c011d00 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c010e40 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.539+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f400fd2b0 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.540+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f400631d0 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.541+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4f2c018020 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f3dffb700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4f300a3cd0 0x7f4f300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f449ce700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4f300a3cd0 0x7f4f300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4f2c0d0180 con 0x7f4f40101cf0 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f3dffb700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] conn(0x7f4f300ad4b0 0x7f4f300af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:35.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 --> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f2c01ce50 con 0x7f4f300ad4b0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4f2c0d0660 con 0x7f4f40101cf0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f449ce700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4f300a3cd0 0x7f4f300a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f34006fd0 tx=0x7f4f34008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.542+0000 7f4f451cf700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] conn(0x7f4f300ad4b0 0x7f4f300af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.543+0000 7f4f451cf700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] conn(0x7f4f300ad4b0 0x7f4f300af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.544+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== osd.1 v2:172.21.15.133:6800/283944628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f2c01ce50 con 0x7f4f300ad4b0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.561+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 --> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f40063650 con 0x7f4f300ad4b0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f3dffb700 1 -- 172.21.15.113:0/3984260103 <== osd.1 v2:172.21.15.133:6800/283944628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f40063650 con 0x7f4f300ad4b0 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] conn(0x7f4f300ad4b0 msgr2=0x7f4f300af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] conn(0x7f4f300ad4b0 0x7f4f300af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4f300a3cd0 msgr2=0x7f4f300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4f300a3cd0 0x7f4f300a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f34006fd0 tx=0x7f4f34008040 comp rx=0 tx=0).stop 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40101cf0 msgr2=0x7f4f401997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:35.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 --2- 172.21.15.113:0/3984260103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f40101cf0 0x7f4f401997d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00b500 tx=0x7f4f2c00b8c0 comp rx=0 tx=0).stop 2024-04-22T07:13:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f451cf700 1 -- 172.21.15.113:0/3984260103 reap_dead start 2024-04-22T07:13:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 shutdown_connections 2024-04-22T07:13:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 >> 172.21.15.113:0/3984260103 conn(0x7f4f40077f40 msgr2=0x7f4f400ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 shutdown_connections 2024-04-22T07:13:35.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:35.562+0000 7f4f46c32700 1 -- 172.21.15.113:0/3984260103 wait complete. 2024-04-22T07:13:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:77309411395 2024-04-22T07:13:36.342 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-22T07:13:36.450 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:36 smithi113 ceph-mon[27933]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-22T07:13:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:36 smithi133 ceph-mon[31198]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-22T07:13:36.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:36 smithi154 ceph-mon[31862]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-22T07:13:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:55834574920 2024-04-22T07:13:36.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-22T07:13:38.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 -- 172.21.15.113:0/678957653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 msgr2=0x7f4794102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 --2- 172.21.15.113:0/678957653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f4794102f10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4784009990 tx=0x7f4784009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 -- 172.21.15.113:0/678957653 shutdown_connections 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 --2- 172.21.15.113:0/678957653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794103450 0x7f479410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 --2- 172.21.15.113:0/678957653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4794108a50 0x7f4794108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 --2- 172.21.15.113:0/678957653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f4794102f10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 -- 172.21.15.113:0/678957653 >> 172.21.15.113:0/678957653 conn(0x7f4794079ba0 msgr2=0x7f4794079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 -- 172.21.15.113:0/678957653 shutdown_connections 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.149+0000 7f479c54b700 1 -- 172.21.15.113:0/678957653 wait complete. 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 Processor -- start 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 -- start start 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f47941991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794103450 0x7f47941996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4794108a50 0x7f479419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4794112da0 con 0x7f4794102a90 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4794112c20 con 0x7f4794108a50 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479c54b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4794112f20 con 0x7f4794103450 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479a2e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f47941991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479a2e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f47941991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41350/0 (socket says 172.21.15.113:41350) 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.150+0000 7f479a2e7700 1 -- 172.21.15.113:0/533049680 learned_addr learned my addr 172.21.15.113:0/533049680 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479aae8700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4794108a50 0x7f479419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f4799ae6700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794103450 0x7f47941996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479a2e7700 1 -- 172.21.15.113:0/533049680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794103450 msgr2=0x7f47941996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479a2e7700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794103450 0x7f47941996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479a2e7700 1 -- 172.21.15.113:0/533049680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4794108a50 msgr2=0x7f479419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479a2e7700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4794108a50 0x7f479419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479a2e7700 1 -- 172.21.15.113:0/533049680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479419e1c0 con 0x7f4794102a90 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479aae8700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4794108a50 0x7f479419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479a2e7700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f47941991b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f479000d950 tx=0x7f479000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790016070 con 0x7f4794102a90 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479419e4b0 con 0x7f4794102a90 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4790009b40 con 0x7f4794102a90 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479000f620 con 0x7f4794102a90 2024-04-22T07:13:38.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.151+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a5d30 con 0x7f4794102a90 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.152+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47940631d0 con 0x7f4794102a90 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.153+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4790019070 con 0x7f4794102a90 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f478b7fe700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f47800a3cd0 0x7f47800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f47900d0520 con 0x7f4794102a90 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f4799ae6700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f47800a3cd0 0x7f47800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f478b7fe700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] conn(0x7f47800ad4b0 0x7f47800af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 --> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4790014e80 con 0x7f47800ad4b0 2024-04-22T07:13:38.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f47900d0a00 con 0x7f4794102a90 2024-04-22T07:13:38.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.154+0000 7f479aae8700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] conn(0x7f47800ad4b0 0x7f47800af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.155+0000 7f4799ae6700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f47800a3cd0 0x7f47800a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4784006010 tx=0x7f478400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.155+0000 7f479aae8700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] conn(0x7f47800ad4b0 0x7f47800af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.155+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== osd.0 v2:172.21.15.113:6802/1926824887 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4790014e80 con 0x7f47800ad4b0 2024-04-22T07:13:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 --> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47940635d0 con 0x7f47800ad4b0 2024-04-22T07:13:38.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f478b7fe700 1 -- 172.21.15.113:0/533049680 <== osd.0 v2:172.21.15.113:6802/1926824887 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47940635d0 con 0x7f47800ad4b0 2024-04-22T07:13:38.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] conn(0x7f47800ad4b0 msgr2=0x7f47800af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] conn(0x7f47800ad4b0 0x7f47800af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f47800a3cd0 msgr2=0x7f47800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f47800a3cd0 0x7f47800a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4784006010 tx=0x7f478400b540 comp rx=0 tx=0).stop 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 msgr2=0x7f47941991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 --2- 172.21.15.113:0/533049680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4794102a90 0x7f47941991b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f479000d950 tx=0x7f479000dd10 comp rx=0 tx=0).stop 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479aae8700 1 -- 172.21.15.113:0/533049680 reap_dead start 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 shutdown_connections 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.172+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 >> 172.21.15.113:0/533049680 conn(0x7f4794079ba0 msgr2=0x7f4794101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:38.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.173+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 shutdown_connections 2024-04-22T07:13:38.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.173+0000 7f479c54b700 1 -- 172.21.15.113:0/533049680 wait complete. 2024-04-22T07:13:38.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:38 smithi113 ceph-mon[27933]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 -- 172.21.15.113:0/439835455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc1022f0 msgr2=0x7f59cc10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 --2- 172.21.15.113:0/439835455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc1022f0 0x7f59cc10e7e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009990 tx=0x7f59c4009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 -- 172.21.15.113:0/439835455 shutdown_connections 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 --2- 172.21.15.113:0/439835455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc1022f0 0x7f59cc10e7e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 --2- 172.21.15.113:0/439835455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59cc107980 0x7f59cc107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 --2- 172.21.15.113:0/439835455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59cc101930 0x7f59cc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 -- 172.21.15.113:0/439835455 >> 172.21.15.113:0/439835455 conn(0x7f59cc0788b0 msgr2=0x7f59cc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 -- 172.21.15.113:0/439835455 shutdown_connections 2024-04-22T07:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.369+0000 7f59d12c1700 1 -- 172.21.15.113:0/439835455 wait complete. 2024-04-22T07:13:38.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 Processor -- start 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 -- start start 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc101930 0x7f59cc199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59cc1022f0 0x7f59cc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59cc107980 0x7f59cc1a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc111c10 con 0x7f59cc101930 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc111a90 con 0x7f59cc1022f0 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59d12c1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc111d90 con 0x7f59cc107980 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59caffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc101930 0x7f59cc199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59caffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc101930 0x7f59cc199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41376/0 (socket says 172.21.15.113:41376) 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59caffd700 1 -- 172.21.15.113:0/4149944970 learned_addr learned my addr 172.21.15.113:0/4149944970 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.370+0000 7f59ca7fc700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59cc1022f0 0x7f59cc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59cb7fe700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59cc107980 0x7f59cc1a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59caffd700 1 -- 172.21.15.113:0/4149944970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59cc107980 msgr2=0x7f59cc1a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59caffd700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59cc107980 0x7f59cc1a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59caffd700 1 -- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59cc1022f0 msgr2=0x7f59cc19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59caffd700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59cc1022f0 0x7f59cc19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59caffd700 1 -- 172.21.15.113:0/4149944970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc1a54b0 con 0x7f59cc101930 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59cb7fe700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59cc107980 0x7f59cc1a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59caffd700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc101930 0x7f59cc199ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f59bc007e20 tx=0x7f59bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc016020 con 0x7f59cc101930 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc1a56e0 con 0x7f59cc101930 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1a5c40 con 0x7f59cc101930 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59bc00ad40 con 0x7f59cc101930 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.371+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc011700 con 0x7f59cc101930 2024-04-22T07:13:38.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.372+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59cc100280 con 0x7f59cc101930 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.373+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f59bc019020 con 0x7f59cc101930 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59c3fff700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f59b80a3d50 0x7f59b80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59ca7fc700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f59b80a3d50 0x7f59b80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f59bc0d07c0 con 0x7f59cc101930 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59c3fff700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] conn(0x7f59b80ad530 0x7f59b80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 --> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59bc018e50 con 0x7f59b80ad530 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f59bc0d0b70 con 0x7f59cc101930 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59ca7fc700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f59b80a3d50 0x7f59b80a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f59cc078d60 tx=0x7f59b40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.374+0000 7f59cb7fe700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] conn(0x7f59b80ad530 0x7f59b80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.375+0000 7f59cb7fe700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] conn(0x7f59b80ad530 0x7f59b80af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.376+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== osd.7 v2:172.21.15.133:6816/3422116555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59bc018e50 con 0x7f59b80ad530 2024-04-22T07:13:38.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.392+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 --> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59cc103050 con 0x7f59b80ad530 2024-04-22T07:13:38.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.392+0000 7f59c3fff700 1 -- 172.21.15.113:0/4149944970 <== osd.7 v2:172.21.15.133:6816/3422116555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59cc103050 con 0x7f59b80ad530 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] conn(0x7f59b80ad530 msgr2=0x7f59b80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] conn(0x7f59b80ad530 0x7f59b80af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f59b80a3d50 msgr2=0x7f59b80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f59b80a3d50 0x7f59b80a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f59cc078d60 tx=0x7f59b40073d0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc101930 msgr2=0x7f59cc199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 --2- 172.21.15.113:0/4149944970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59cc101930 0x7f59cc199ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f59bc007e20 tx=0x7f59bc00a340 comp rx=0 tx=0).stop 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59cb7fe700 1 -- 172.21.15.113:0/4149944970 reap_dead start 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 shutdown_connections 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 >> 172.21.15.113:0/4149944970 conn(0x7f59cc0788b0 msgr2=0x7f59cc0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 shutdown_connections 2024-04-22T07:13:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.393+0000 7f59d12c1700 1 -- 172.21.15.113:0/4149944970 wait complete. 2024-04-22T07:13:38.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:38 smithi133 ceph-mon[31198]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:13:38.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:38 smithi154 ceph-mon[31862]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:13:38.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1983139677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 msgr2=0x7fa1c410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1983139677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c410f900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00b0a0 tx=0x7fa1bc00b3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1983139677 shutdown_connections 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1983139677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c410f900 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1983139677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4108a30 0x7fa1c4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1983139677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1c406a750 0x7fa1c406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1983139677 >> 172.21.15.113:0/1983139677 conn(0x7fa1c40788f0 msgr2=0x7fa1c4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1983139677 shutdown_connections 2024-04-22T07:13:38.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1983139677 wait complete. 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.586+0000 7fa1cb6dc700 1 Processor -- start 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 -- start start 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1c406a750 0x7fa1c4101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c4101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4108a30 0x7fa1c4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4112d90 con 0x7fa1c406b0f0 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4112c10 con 0x7fa1c4108a30 2024-04-22T07:13:38.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1cb6dc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4112f10 con 0x7fa1c406a750 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c4101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c4101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41398/0 (socket says 172.21.15.113:41398) 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 -- 172.21.15.113:0/1978150612 learned_addr learned my addr 172.21.15.113:0/1978150612 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c9478700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1c406a750 0x7fa1c4101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c9c79700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4108a30 0x7fa1c4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 -- 172.21.15.113:0/1978150612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1c406a750 msgr2=0x7fa1c4101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1c406a750 0x7fa1c4101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 -- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4108a30 msgr2=0x7fa1c4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4108a30 0x7fa1c4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 -- 172.21.15.113:0/1978150612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c41a7690 con 0x7fa1c406b0f0 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c9c79700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4108a30 0x7fa1c4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1c8c77700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c4101f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d950 tx=0x7fa1b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.587+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4016070 con 0x7fa1c406b0f0 2024-04-22T07:13:38.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.588+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c41a7980 con 0x7fa1c406b0f0 2024-04-22T07:13:38.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.588+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1b4009e10 con 0x7fa1c406b0f0 2024-04-22T07:13:38.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.588+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400f620 con 0x7fa1c406b0f0 2024-04-22T07:13:38.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.588+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41a7ec0 con 0x7fa1c406b0f0 2024-04-22T07:13:38.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.588+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1c4063110 con 0x7fa1c406b0f0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa1b4019070 con 0x7fa1c406b0f0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1ba7fc700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1b00a3cd0 0x7fa1b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1c9478700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1b00a3cd0 0x7fa1b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa1b40d0490 con 0x7fa1c406b0f0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1ba7fc700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] conn(0x7fa1b00ad4b0 0x7fa1b00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 --> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1b00aff80 con 0x7fa1b00ad4b0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.590+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa1b40d0970 con 0x7fa1c406b0f0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.591+0000 7fa1c9478700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1b00a3cd0 0x7fa1b00a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005950 tx=0x7fa1c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.591+0000 7fa1c9c79700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] conn(0x7fa1b00ad4b0 0x7fa1b00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:38.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.591+0000 7fa1c9c79700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] conn(0x7fa1b00ad4b0 0x7fa1b00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:38.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.592+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== osd.10 v2:172.21.15.133:6824/1753665247 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1b00aff80 con 0x7fa1b00ad4b0 2024-04-22T07:13:38.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.608+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 --> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1c4063320 con 0x7fa1b00ad4b0 2024-04-22T07:13:38.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1ba7fc700 1 -- 172.21.15.113:0/1978150612 <== osd.10 v2:172.21.15.133:6824/1753665247 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa1c4063320 con 0x7fa1b00ad4b0 2024-04-22T07:13:38.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] conn(0x7fa1b00ad4b0 msgr2=0x7fa1b00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] conn(0x7fa1b00ad4b0 0x7fa1b00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:38.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1b00a3cd0 msgr2=0x7fa1b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fa1b00a3cd0 0x7fa1b00a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005950 tx=0x7fa1c000b410 comp rx=0 tx=0).stop 2024-04-22T07:13:38.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 msgr2=0x7fa1c4101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:38.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 --2- 172.21.15.113:0/1978150612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1c406b0f0 0x7fa1c4101f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d950 tx=0x7fa1b400dc60 comp rx=0 tx=0).stop 2024-04-22T07:13:38.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1c9c79700 1 -- 172.21.15.113:0/1978150612 reap_dead start 2024-04-22T07:13:38.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 shutdown_connections 2024-04-22T07:13:38.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 >> 172.21.15.113:0/1978150612 conn(0x7fa1c40788f0 msgr2=0x7fa1c40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:38.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 shutdown_connections 2024-04-22T07:13:38.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:38.609+0000 7fa1cb6dc700 1 -- 172.21.15.113:0/1978150612 wait complete. 2024-04-22T07:13:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:223338299435 2024-04-22T07:13:39.118 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-22T07:13:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:34359738445 2024-04-22T07:13:40.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-22T07:13:40.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:40 smithi113 ceph-mon[27933]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-22T07:13:40.449 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.449 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.449 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.453 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.456 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.463 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.463 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:313532612635 2024-04-22T07:13:40.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-22T07:13:40.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:40 smithi133 ceph-mon[31198]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-22T07:13:40.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:40 smithi154 ceph-mon[31862]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-22T07:13:42.241 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:42 smithi113 ceph-mon[27933]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:13:42.511 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:42.512 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:42 smithi133 ceph-mon[31198]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:13:42.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:42 smithi154 ceph-mon[31862]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:13:44.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:44 smithi133 ceph-mon[31198]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T07:13:44.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:44 smithi154 ceph-mon[31862]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T07:13:44.572 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:44 smithi113 ceph-mon[27933]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T07:13:46.422 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:46 smithi113 ceph-mon[27933]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-22T07:13:46.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:46 smithi133 ceph-mon[31198]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-22T07:13:46.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:46 smithi154 ceph-mon[31862]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-22T07:13:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 -- 172.21.15.113:0/103759023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14103450 msgr2=0x7f9e1410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/103759023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14103450 0x7f9e1410f8b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009990 tx=0x7f9e04009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 -- 172.21.15.113:0/103759023 shutdown_connections 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/103759023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14103450 0x7f9e1410f8b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/103759023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14108a50 0x7f9e14108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/103759023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e14102a90 0x7f9e14102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.169+0000 7f9e1b55f700 1 -- 172.21.15.113:0/103759023 >> 172.21.15.113:0/103759023 conn(0x7f9e14079ba0 msgr2=0x7f9e14079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.170+0000 7f9e1b55f700 1 -- 172.21.15.113:0/103759023 shutdown_connections 2024-04-22T07:13:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.170+0000 7f9e1b55f700 1 -- 172.21.15.113:0/103759023 wait complete. 2024-04-22T07:13:48.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.170+0000 7f9e1b55f700 1 Processor -- start 2024-04-22T07:13:48.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.170+0000 7f9e1b55f700 1 -- start start 2024-04-22T07:13:48.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e1b55f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 0x7f9e14199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e1b55f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14103450 0x7f9e14199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e1b55f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e14108a50 0x7f9e1419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e1b55f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14112e20 con 0x7f9e14102a90 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e1b55f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14112ca0 con 0x7f9e14108a50 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e1b55f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9e14112fa0 con 0x7f9e14103450 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 0x7f9e14199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 0x7f9e14199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35716/0 (socket says 172.21.15.113:35716) 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 -- 172.21.15.113:0/915004894 learned_addr learned my addr 172.21.15.113:0/915004894 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e18afa700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14103450 0x7f9e14199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e19afc700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e14108a50 0x7f9e1419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 -- 172.21.15.113:0/915004894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14103450 msgr2=0x7f9e14199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14103450 0x7f9e14199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 -- 172.21.15.113:0/915004894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e14108a50 msgr2=0x7f9e1419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e14108a50 0x7f9e1419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 -- 172.21.15.113:0/915004894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1419e360 con 0x7f9e14102a90 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e18afa700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14103450 0x7f9e14199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.171+0000 7f9e192fb700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 0x7f9e14199350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00ba50 tx=0x7f9e0c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.172+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c019070 con 0x7f9e14102a90 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.172+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1419e650 con 0x7f9e14102a90 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.172+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9e0c014440 con 0x7f9e14102a90 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.172+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0075d0 con 0x7f9e14102a90 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.172+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e141a5ed0 con 0x7f9e14102a90 2024-04-22T07:13:48.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.173+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e14069730 con 0x7f9e14102a90 2024-04-22T07:13:48.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.174+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9e0c01c070 con 0x7f9e14102a90 2024-04-22T07:13:48.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.174+0000 7f9e127fc700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9dfc0a3cd0 0x7f9dfc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:48.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.174+0000 7f9e18afa700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9dfc0a3cd0 0x7f9dfc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:48.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.174+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9e0c0cf790 con 0x7f9e14102a90 2024-04-22T07:13:48.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.175+0000 7f9e18afa700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9dfc0a3cd0 0x7f9dfc0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e1407a2c0 tx=0x7f9e08008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.179+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e0c09c2d0 con 0x7f9e14102a90 2024-04-22T07:13:48.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:48 smithi113 ceph-mon[27933]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:48.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.386+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f9e1419a6e0 con 0x7f9e14102a90 2024-04-22T07:13:48.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.386+0000 7f9e127fc700 1 -- 172.21.15.113:0/915004894 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e1419a6e0 con 0x7f9e14102a90 2024-04-22T07:13:48.384 INFO:teuthology.orchestra.run.smithi113.stdout:292057776162 2024-04-22T07:13:48.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.389+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9dfc0a3cd0 msgr2=0x7f9dfc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:48.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.389+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9dfc0a3cd0 0x7f9dfc0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e1407a2c0 tx=0x7f9e08008040 comp rx=0 tx=0).stop 2024-04-22T07:13:48.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 msgr2=0x7f9e14199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 0x7f9e14199350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00ba50 tx=0x7f9e0c00be10 comp rx=0 tx=0).stop 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 shutdown_connections 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9dfc0a3cd0 0x7f9dfc0a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e14108a50 0x7f9e1419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e14102a90 0x7f9e14199350 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 --2- 172.21.15.113:0/915004894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e14103450 0x7f9e14199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 >> 172.21.15.113:0/915004894 conn(0x7f9e14079ba0 msgr2=0x7f9e141018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.390+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 shutdown_connections 2024-04-22T07:13:48.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:48.391+0000 7f9e1b55f700 1 -- 172.21.15.113:0/915004894 wait complete. 2024-04-22T07:13:48.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:48 smithi133 ceph-mon[31198]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:48.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:48 smithi154 ceph-mon[31862]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:48.827 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:49 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/915004894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T07:13:49.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:49 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/915004894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T07:13:49.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:49 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/915004894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T07:13:50.297 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:50 smithi113 ceph-mon[27933]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:13:50.303 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:13:50.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:50 smithi133 ceph-mon[31198]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:13:50.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:50 smithi154 ceph-mon[31862]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:13:50.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 -- 172.21.15.113:0/2938779953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40102a90 msgr2=0x7f2c40102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:50.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 --2- 172.21.15.113:0/2938779953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40102a90 0x7f2c40102f10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009990 tx=0x7f2c30009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 -- 172.21.15.113:0/2938779953 shutdown_connections 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 --2- 172.21.15.113:0/2938779953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40103450 0x7f2c4010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 --2- 172.21.15.113:0/2938779953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 0x7f2c40108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 --2- 172.21.15.113:0/2938779953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40102a90 0x7f2c40102f10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 -- 172.21.15.113:0/2938779953 >> 172.21.15.113:0/2938779953 conn(0x7f2c40079bf0 msgr2=0x7f2c4007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 -- 172.21.15.113:0/2938779953 shutdown_connections 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.798+0000 7f2c475a8700 1 -- 172.21.15.113:0/2938779953 wait complete. 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 Processor -- start 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 -- start start 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40102a90 0x7f2c40199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 0x7f2c401998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 0x7f2c4019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:50.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40112e50 con 0x7f2c40103450 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40112cd0 con 0x7f2c40108a80 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c475a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40112fd0 con 0x7f2c40102a90 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c44b43700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 0x7f2c401998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c44b43700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 0x7f2c401998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35732/0 (socket says 172.21.15.113:35732) 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c44b43700 1 -- 172.21.15.113:0/1435367542 learned_addr learned my addr 172.21.15.113:0/1435367542 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.799+0000 7f2c45b45700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 0x7f2c4019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c45344700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40102a90 0x7f2c40199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c44b43700 1 -- 172.21.15.113:0/1435367542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40102a90 msgr2=0x7f2c40199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c44b43700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40102a90 0x7f2c40199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c44b43700 1 -- 172.21.15.113:0/1435367542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 msgr2=0x7f2c4019dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c44b43700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 0x7f2c4019dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c44b43700 1 -- 172.21.15.113:0/1435367542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4019e370 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c45b45700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 0x7f2c4019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c45344700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40102a90 0x7f2c40199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c44b43700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 0x7f2c401998a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2c30000c00 tx=0x7f2c300037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.800+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c30027070 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.801+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2c300041e0 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.801+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000f460 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.801+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4019e600 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.801+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401a5ee0 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.802+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c40069730 con 0x7f2c40103450 2024-04-22T07:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.803+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2c3002a070 con 0x7f2c40103450 2024-04-22T07:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.803+0000 7f2c367fc700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2c2c0a3d50 0x7f2c2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.804+0000 7f2c45344700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2c2c0a3d50 0x7f2c2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.804+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2c300d12b0 con 0x7f2c40103450 2024-04-22T07:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.804+0000 7f2c45344700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2c2c0a3d50 0x7f2c2c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c007900 tx=0x7f2c3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:50.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:50.808+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c30013070 con 0x7f2c40103450 2024-04-22T07:13:51.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.010+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2c4002df80 con 0x7f2c40103450 2024-04-22T07:13:51.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.010+0000 7f2c367fc700 1 -- 172.21.15.113:0/1435367542 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2c3009ddd0 con 0x7f2c40103450 2024-04-22T07:13:51.008 INFO:teuthology.orchestra.run.smithi113.stdout:223338299437 2024-04-22T07:13:51.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2c2c0a3d50 msgr2=0x7f2c2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2c2c0a3d50 0x7f2c2c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c007900 tx=0x7f2c3c008040 comp rx=0 tx=0).stop 2024-04-22T07:13:51.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 msgr2=0x7f2c401998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 0x7f2c401998a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2c30000c00 tx=0x7f2c300037a0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 shutdown_connections 2024-04-22T07:13:51.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2c2c0a3d50 0x7f2c2c0a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c40108a80 0x7f2c4019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c40102a90 0x7f2c40199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 --2- 172.21.15.113:0/1435367542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c40103450 0x7f2c401998a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 >> 172.21.15.113:0/1435367542 conn(0x7f2c40079bf0 msgr2=0x7f2c401017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:51.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 shutdown_connections 2024-04-22T07:13:51.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.012+0000 7f2c475a8700 1 -- 172.21.15.113:0/1435367542 wait complete. 2024-04-22T07:13:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:51 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1435367542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T07:13:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 -- 172.21.15.113:0/3391798775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698101940 msgr2=0x7f1698101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3391798775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698101940 0x7f1698101dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1694009990 tx=0x7f1694009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 -- 172.21.15.113:0/3391798775 shutdown_connections 2024-04-22T07:13:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3391798775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1698102300 0x7f169810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3391798775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1698107990 0x7f1698107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3391798775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698101940 0x7f1698101dc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 -- 172.21.15.113:0/3391798775 >> 172.21.15.113:0/3391798775 conn(0x7f16980788f0 msgr2=0x7f1698078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 -- 172.21.15.113:0/3391798775 shutdown_connections 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.390+0000 7f169f3d9700 1 -- 172.21.15.113:0/3391798775 wait complete. 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 Processor -- start 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 -- start start 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1698101940 0x7f1698198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1698102300 0x7f1698198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 0x7f169819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111c70 con 0x7f1698107990 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111af0 con 0x7f1698101940 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169f3d9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111df0 con 0x7f1698102300 2024-04-22T07:13:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169d976700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 0x7f169819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169d976700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 0x7f169819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35756/0 (socket says 172.21.15.113:35756) 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169d976700 1 -- 172.21.15.113:0/3492065037 learned_addr learned my addr 172.21.15.113:0/3492065037 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169d175700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1698101940 0x7f1698198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169c974700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1698102300 0x7f1698198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169d976700 1 -- 172.21.15.113:0/3492065037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1698102300 msgr2=0x7f1698198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.391+0000 7f169d976700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1698102300 0x7f1698198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f169d976700 1 -- 172.21.15.113:0/3492065037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1698101940 msgr2=0x7f1698198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f169d976700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1698101940 0x7f1698198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f169d976700 1 -- 172.21.15.113:0/3492065037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169819d0e0 con 0x7f1698107990 2024-04-22T07:13:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f169d976700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 0x7f169819c9a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f169000ef30 tx=0x7f169000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:51.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16900092e0 con 0x7f1698107990 2024-04-22T07:13:51.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169819d370 con 0x7f1698107990 2024-04-22T07:13:51.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1690004bb0 con 0x7f1698107990 2024-04-22T07:13:51.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004280 con 0x7f1698107990 2024-04-22T07:13:51.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.392+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169806a720 con 0x7f1698107990 2024-04-22T07:13:51.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.393+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1698069730 con 0x7f1698107990 2024-04-22T07:13:51.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.395+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f169001b030 con 0x7f1698107990 2024-04-22T07:13:51.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.395+0000 7f168e7fc700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f16840a3d50 0x7f16840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:51.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.395+0000 7f169d175700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f16840a3d50 0x7f16840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:51.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.395+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f16900d07e0 con 0x7f1698107990 2024-04-22T07:13:51.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.396+0000 7f169d175700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f16840a3d50 0x7f16840a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1688009be0 tx=0x7f1688009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:51.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.399+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1690022020 con 0x7f1698107990 2024-04-22T07:13:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:51 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1435367542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T07:13:51.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:51 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1435367542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T07:13:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.607+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f169806b210 con 0x7f1698107990 2024-04-22T07:13:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.608+0000 7f168e7fc700 1 -- 172.21.15.113:0/3492065037 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1690010040 con 0x7f1698107990 2024-04-22T07:13:51.606 INFO:teuthology.orchestra.run.smithi113.stdout:257698037800 2024-04-22T07:13:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f16840a3d50 msgr2=0x7f16840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f16840a3d50 0x7f16840a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1688009be0 tx=0x7f1688009260 comp rx=0 tx=0).stop 2024-04-22T07:13:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 msgr2=0x7f169819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 0x7f169819c9a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f169000ef30 tx=0x7f169000c3a0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 shutdown_connections 2024-04-22T07:13:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f16840a3d50 0x7f16840a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1698107990 0x7f169819c9a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1698101940 0x7f1698198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 --2- 172.21.15.113:0/3492065037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1698102300 0x7f1698198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 >> 172.21.15.113:0/3492065037 conn(0x7f16980788f0 msgr2=0x7f16980ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 shutdown_connections 2024-04-22T07:13:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:51.610+0000 7f169f3d9700 1 -- 172.21.15.113:0/3492065037 wait complete. 2024-04-22T07:13:52.429 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:52 smithi113 ceph-mon[27933]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:52.429 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3492065037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T07:13:52.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:52 smithi133 ceph-mon[31198]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:52.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3492065037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T07:13:52.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:52 smithi154 ceph-mon[31862]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:52.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3492065037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T07:13:52.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 -- 172.21.15.113:0/32404394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c107980 msgr2=0x7f414c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:52.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 --2- 172.21.15.113:0/32404394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c107980 0x7f414c107d60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4148009990 tx=0x7f4148009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 -- 172.21.15.113:0/32404394 shutdown_connections 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 --2- 172.21.15.113:0/32404394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f414c1022f0 0x7f414c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 --2- 172.21.15.113:0/32404394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c107980 0x7f414c107d60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 --2- 172.21.15.113:0/32404394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c101930 0x7f414c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 -- 172.21.15.113:0/32404394 >> 172.21.15.113:0/32404394 conn(0x7f414c0788b0 msgr2=0x7f414c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 -- 172.21.15.113:0/32404394 shutdown_connections 2024-04-22T07:13:52.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.620+0000 7f4153e50700 1 -- 172.21.15.113:0/32404394 wait complete. 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.621+0000 7f4153e50700 1 Processor -- start 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.621+0000 7f4153e50700 1 -- start start 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.621+0000 7f4153e50700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f414c101930 0x7f414c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.621+0000 7f4153e50700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 0x7f414c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4153e50700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c107980 0x7f414c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4153e50700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f414c111c80 con 0x7f414c1022f0 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4153e50700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f414c111b00 con 0x7f414c107980 2024-04-22T07:13:52.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4153e50700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f414c111e00 con 0x7f414c101930 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 0x7f414c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 0x7f414c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35766/0 (socket says 172.21.15.113:35766) 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 -- 172.21.15.113:0/1333089938 learned_addr learned my addr 172.21.15.113:0/1333089938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4151bec700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f414c101930 0x7f414c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41523ed700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c107980 0x7f414c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 -- 172.21.15.113:0/1333089938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f414c101930 msgr2=0x7f414c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f414c101930 0x7f414c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 -- 172.21.15.113:0/1333089938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c107980 msgr2=0x7f414c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c107980 0x7f414c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 -- 172.21.15.113:0/1333089938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c19d120 con 0x7f414c1022f0 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41523ed700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c107980 0x7f414c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f41513eb700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 0x7f414c198650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c920 tx=0x7f413c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c007b00 con 0x7f414c1022f0 2024-04-22T07:13:52.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c19d410 con 0x7f414c1022f0 2024-04-22T07:13:52.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.622+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f413c004d00 con 0x7f414c1022f0 2024-04-22T07:13:52.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.623+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c005560 con 0x7f414c1022f0 2024-04-22T07:13:52.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.623+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c06afb0 con 0x7f414c1022f0 2024-04-22T07:13:52.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.623+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414c069730 con 0x7f414c1022f0 2024-04-22T07:13:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.624+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f413c004750 con 0x7f414c1022f0 2024-04-22T07:13:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.625+0000 7f4142ffd700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f41380a3cd0 0x7f41380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.625+0000 7f4151bec700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f41380a3cd0 0x7f41380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.625+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f413c0cff90 con 0x7f414c1022f0 2024-04-22T07:13:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.625+0000 7f4151bec700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f41380a3cd0 0x7f41380a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41480098e0 tx=0x7f4148005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:52.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.629+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f413c014030 con 0x7f414c1022f0 2024-04-22T07:13:52.834 INFO:teuthology.orchestra.run.smithi113.stdout:163208757302 2024-04-22T07:13:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.835+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f414c06ba40 con 0x7f414c1022f0 2024-04-22T07:13:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.836+0000 7f4142ffd700 1 -- 172.21.15.113:0/1333089938 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f413c09c9c0 con 0x7f414c1022f0 2024-04-22T07:13:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f41380a3cd0 msgr2=0x7f41380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f41380a3cd0 0x7f41380a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f41480098e0 tx=0x7f4148005960 comp rx=0 tx=0).stop 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 msgr2=0x7f414c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 0x7f414c198650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c920 tx=0x7f413c00cc30 comp rx=0 tx=0).stop 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 shutdown_connections 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f41380a3cd0 0x7f41380a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f414c107980 0x7f414c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f414c101930 0x7f414c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 --2- 172.21.15.113:0/1333089938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f414c1022f0 0x7f414c198650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 >> 172.21.15.113:0/1333089938 conn(0x7f414c0788b0 msgr2=0x7f414c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 shutdown_connections 2024-04-22T07:13:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:52.838+0000 7f4153e50700 1 -- 172.21.15.113:0/1333089938 wait complete. 2024-04-22T07:13:53.423 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1333089938' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T07:13:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:53 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1333089938' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T07:13:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1333089938' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T07:13:53.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 -- 172.21.15.113:0/1486375995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8103480 msgr2=0x7fc4b810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:53.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/1486375995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8103480 0x7fc4b810f8d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b0a0 tx=0x7fc4b000b3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 -- 172.21.15.113:0/1486375995 shutdown_connections 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/1486375995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8103480 0x7fc4b810f8d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/1486375995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8108a70 0x7fc4b8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/1486375995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 0x7fc4b8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 -- 172.21.15.113:0/1486375995 >> 172.21.15.113:0/1486375995 conn(0x7fc4b8079bb0 msgr2=0x7fc4b8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 -- 172.21.15.113:0/1486375995 shutdown_connections 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.955+0000 7fc4be3c9700 1 -- 172.21.15.113:0/1486375995 wait complete. 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 Processor -- start 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 -- start start 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 0x7fc4b8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8103480 0x7fc4b8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 0x7fc4b80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8112de0 con 0x7fc4b8108a70 2024-04-22T07:13:53.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8112c60 con 0x7fc4b8102ac0 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4be3c9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b8112f60 con 0x7fc4b8103480 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4bc966700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 0x7fc4b80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4bc966700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 0x7fc4b80757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35782/0 (socket says 172.21.15.113:35782) 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4bc966700 1 -- 172.21.15.113:0/3923262556 learned_addr learned my addr 172.21.15.113:0/3923262556 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4b7fff700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 0x7fc4b8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.956+0000 7fc4b77fe700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8103480 0x7fc4b8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4bc966700 1 -- 172.21.15.113:0/3923262556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8103480 msgr2=0x7fc4b8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4bc966700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8103480 0x7fc4b8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4bc966700 1 -- 172.21.15.113:0/3923262556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 msgr2=0x7fc4b8074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4bc966700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 0x7fc4b8074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4bc966700 1 -- 172.21.15.113:0/3923262556 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b81a7720 con 0x7fc4b8108a70 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4b7fff700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 0x7fc4b8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4b77fe700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8103480 0x7fc4b8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4bc966700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 0x7fc4b80757c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b070 tx=0x7fc4b0007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:53.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0021070 con 0x7fc4b8108a70 2024-04-22T07:13:53.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc4b001db20 con 0x7fc4b8108a70 2024-04-22T07:13:53.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b81a79b0 con 0x7fc4b8108a70 2024-04-22T07:13:53.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001bc50 con 0x7fc4b8108a70 2024-04-22T07:13:53.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.957+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b81a7ef0 con 0x7fc4b8108a70 2024-04-22T07:13:53.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.958+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b8069730 con 0x7fc4b8108a70 2024-04-22T07:13:53.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.959+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc4b0024070 con 0x7fc4b8108a70 2024-04-22T07:13:53.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.959+0000 7fc4b57fa700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc4a00a3cd0 0x7fc4a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:53.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.960+0000 7fc4b7fff700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc4a00a3cd0 0x7fc4a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:53.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.960+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc4b000f020 con 0x7fc4b8108a70 2024-04-22T07:13:53.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.960+0000 7fc4b7fff700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc4a00a3cd0 0x7fc4a00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800bcf0 tx=0x7fc4a800b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:53.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:53.964+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4b002b020 con 0x7fc4b8108a70 2024-04-22T07:13:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.180+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc4b802e140 con 0x7fc4b8108a70 2024-04-22T07:13:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.180+0000 7fc4b57fa700 1 -- 172.21.15.113:0/3923262556 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc4b009e520 con 0x7fc4b8108a70 2024-04-22T07:13:54.178 INFO:teuthology.orchestra.run.smithi113.stdout:197568495665 2024-04-22T07:13:54.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc4a00a3cd0 msgr2=0x7fc4a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:54.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc4a00a3cd0 0x7fc4a00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800bcf0 tx=0x7fc4a800b2c0 comp rx=0 tx=0).stop 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 msgr2=0x7fc4b80757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 0x7fc4b80757c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b070 tx=0x7fc4b0007ad0 comp rx=0 tx=0).stop 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 shutdown_connections 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc4a00a3cd0 0x7fc4a00a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4b8108a70 0x7fc4b80757c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b8102ac0 0x7fc4b8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 --2- 172.21.15.113:0/3923262556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b8103480 0x7fc4b8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 >> 172.21.15.113:0/3923262556 conn(0x7fc4b8079bb0 msgr2=0x7fc4b81016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 shutdown_connections 2024-04-22T07:13:54.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:54.183+0000 7fc4be3c9700 1 -- 172.21.15.113:0/3923262556 wait complete. 2024-04-22T07:13:54.407 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:54 smithi113 ceph-mon[27933]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:54.407 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:54 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:13:54.435 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776158 got 292057776162 for osd.9 2024-04-22T07:13:54.435 DEBUG:teuthology.parallel:result is None 2024-04-22T07:13:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:54 smithi133 ceph-mon[31198]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:54 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:13:54.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:54 smithi154 ceph-mon[31862]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:13:54.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:54 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:13:55.494 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3923262556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-22T07:13:55.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:55 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 -- 172.21.15.113:0/2611956184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f406b710 msgr2=0x7fd8f406bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/2611956184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f406b710 0x7fd8f406bb70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009990 tx=0x7fd8f0009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 -- 172.21.15.113:0/2611956184 shutdown_connections 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/2611956184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f4105480 0x7fd8f4105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/2611956184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f4107970 0x7fd8f406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/2611956184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f406b710 0x7fd8f406bb70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 -- 172.21.15.113:0/2611956184 >> 172.21.15.113:0/2611956184 conn(0x7fd8f4078870 msgr2=0x7fd8f4078c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 -- 172.21.15.113:0/2611956184 shutdown_connections 2024-04-22T07:13:55.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 -- 172.21.15.113:0/2611956184 wait complete. 2024-04-22T07:13:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 Processor -- start 2024-04-22T07:13:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.532+0000 7fd8fb91a700 1 -- start start 2024-04-22T07:13:55.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8fb91a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 0x7fd8f41a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8fb91a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f4105480 0x7fd8f41a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8fb91a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 0x7fd8f419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8fb91a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4119300 con 0x7fd8f4107970 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8fb91a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4119180 con 0x7fd8f4105480 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8fb91a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4119480 con 0x7fd8f406b710 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 0x7fd8f419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f96b6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 0x7fd8f41a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f96b6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 0x7fd8f41a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:56598/0 (socket says 172.21.15.113:56598) 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f96b6700 1 -- 172.21.15.113:0/3562612995 learned_addr learned my addr 172.21.15.113:0/3562612995 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 0x7fd8f419f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35806/0 (socket says 172.21.15.113:35806) 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f8eb5700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f4105480 0x7fd8f41a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 -- 172.21.15.113:0/3562612995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 msgr2=0x7fd8f41a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 0x7fd8f41a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 -- 172.21.15.113:0/3562612995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f4105480 msgr2=0x7fd8f41a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f4105480 0x7fd8f41a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 -- 172.21.15.113:0/3562612995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f419fd40 con 0x7fd8f4107970 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f96b6700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 0x7fd8f41a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8f9eb7700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 0x7fd8f419f5e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec007f10 tx=0x7fd8ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec004db0 con 0x7fd8f4107970 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.533+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd8ec003a00 con 0x7fd8f4107970 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.534+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f41a0030 con 0x7fd8f4107970 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.534+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec004550 con 0x7fd8f4107970 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.534+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41ac2f0 con 0x7fd8f4107970 2024-04-22T07:13:55.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.534+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f4069730 con 0x7fd8f4107970 2024-04-22T07:13:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.536+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd8ec01c020 con 0x7fd8f4107970 2024-04-22T07:13:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.536+0000 7fd8e67fc700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd8e00a3d50 0x7fd8e00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.536+0000 7fd8f96b6700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd8e00a3d50 0x7fd8e00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.536+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd8ec0d06b0 con 0x7fd8f4107970 2024-04-22T07:13:55.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.537+0000 7fd8f96b6700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd8e00a3d50 0x7fd8e00a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8011550 tx=0x7fd8e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:55.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.540+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8ec023030 con 0x7fd8f4107970 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3923262556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-22T07:13:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-22T07:13:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-22T07:13:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:55 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-04-22T07:13:55.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3923262556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-22T07:13:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:55 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-04-22T07:13:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.746+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd8f41ac520 con 0x7fd8f4107970 2024-04-22T07:13:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.746+0000 7fd8e67fc700 1 -- 172.21.15.113:0/3562612995 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd8ec010070 con 0x7fd8f4107970 2024-04-22T07:13:55.745 INFO:teuthology.orchestra.run.smithi113.stdout:77309411399 2024-04-22T07:13:55.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd8e00a3d50 msgr2=0x7fd8e00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:55.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd8e00a3d50 0x7fd8e00a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8011550 tx=0x7fd8e8009260 comp rx=0 tx=0).stop 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 msgr2=0x7fd8f419f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 0x7fd8f419f5e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec007f10 tx=0x7fd8ec00d3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 shutdown_connections 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fd8e00a3d50 0x7fd8e00a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8f4107970 0x7fd8f419f5e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8f406b710 0x7fd8f41a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 --2- 172.21.15.113:0/3562612995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f4105480 0x7fd8f41a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:55.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 >> 172.21.15.113:0/3562612995 conn(0x7fd8f4078870 msgr2=0x7fd8f4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:55.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 shutdown_connections 2024-04-22T07:13:55.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:55.748+0000 7fd8fb91a700 1 -- 172.21.15.113:0/3562612995 wait complete. 2024-04-22T07:13:55.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037800 for osd.8 2024-04-22T07:13:55.915 DEBUG:teuthology.parallel:result is None 2024-04-22T07:13:56.508 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.509 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.510 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.510 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3562612995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-22T07:13:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3562612995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T07:13:56.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:13:56.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3562612995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T07:13:56.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 -- 172.21.15.113:0/1965289171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 msgr2=0x7f99fc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/1965289171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc101df0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009990 tx=0x7f99ec009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 -- 172.21.15.113:0/1965289171 shutdown_connections 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/1965289171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 0x7f99fc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/1965289171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 0x7f99fc107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/1965289171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc101df0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 -- 172.21.15.113:0/1965289171 >> 172.21.15.113:0/1965289171 conn(0x7f99fc078910 msgr2=0x7f99fc078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 -- 172.21.15.113:0/1965289171 shutdown_connections 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.971+0000 7f9a02b1d700 1 -- 172.21.15.113:0/1965289171 wait complete. 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 Processor -- start 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 -- start start 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:56.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 0x7f99fc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 0x7f99fc1a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc111ca0 con 0x7f99fc101970 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc111b20 con 0x7f99fc102330 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a02b1d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc111e20 con 0x7f99fc107920 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a008b9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a008b9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc19a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48458/0 (socket says 172.21.15.113:48458) 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f9a008b9700 1 -- 172.21.15.113:0/2916741653 learned_addr learned my addr 172.21.15.113:0/2916741653 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.972+0000 7f99fbfff700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 0x7f99fc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a010ba700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 0x7f99fc1a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a008b9700 1 -- 172.21.15.113:0/2916741653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 msgr2=0x7f99fc1a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a008b9700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 0x7f99fc1a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a008b9700 1 -- 172.21.15.113:0/2916741653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 msgr2=0x7f99fc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:56.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a008b9700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 0x7f99fc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a008b9700 1 -- 172.21.15.113:0/2916741653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc197fb0 con 0x7f99fc101970 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a010ba700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 0x7f99fc1a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f99fbfff700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 0x7f99fc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a008b9700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc19a1c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f99f4007f10 tx=0x7f99f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400dc90 con 0x7f99fc101970 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc1982a0 con 0x7f99fc101970 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f99f400de30 con 0x7f99fc101970 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40133f0 con 0x7f99fc101970 2024-04-22T07:13:56.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.973+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc198790 con 0x7f99fc101970 2024-04-22T07:13:56.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.974+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99fc069730 con 0x7f99fc101970 2024-04-22T07:13:56.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.975+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f99f401a020 con 0x7f99fc101970 2024-04-22T07:13:56.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.976+0000 7f99f9ffb700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f99e80a3cd0 0x7f99e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:56.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.976+0000 7f99fbfff700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f99e80a3cd0 0x7f99e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:56.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.976+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f99f40d04c0 con 0x7f99fc101970 2024-04-22T07:13:56.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.976+0000 7f99fbfff700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f99e80a3cd0 0x7f99e80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99ec006010 tx=0x7f99ec00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:56.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:56.980+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99f4010070 con 0x7f99fc101970 2024-04-22T07:13:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 -- 172.21.15.113:0/662829966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406a750 msgr2=0x7f976406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 --2- 172.21.15.113:0/662829966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406a750 0x7f976406abb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9754009990 tx=0x7f9754009ca0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 -- 172.21.15.113:0/662829966 shutdown_connections 2024-04-22T07:13:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 --2- 172.21.15.113:0/662829966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f976406b0f0 0x7f976410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 --2- 172.21.15.113:0/662829966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9764108a30 0x7f9764108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 --2- 172.21.15.113:0/662829966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406a750 0x7f976406abb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 -- 172.21.15.113:0/662829966 >> 172.21.15.113:0/662829966 conn(0x7f97640788f0 msgr2=0x7f9764078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 -- 172.21.15.113:0/662829966 shutdown_connections 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 -- 172.21.15.113:0/662829966 wait complete. 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 Processor -- start 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.166+0000 7f976a2bb700 1 -- start start 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f976a2bb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f976406a750 0x7f9764101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f976a2bb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 0x7f9764101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f976a2bb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9764108a30 0x7f9764102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f976a2bb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9764112d90 con 0x7f976406b0f0 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f976a2bb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9764112c10 con 0x7f976406a750 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f976a2bb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9764112f10 con 0x7f9764108a30 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 0x7f9764101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 0x7f9764101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48470/0 (socket says 172.21.15.113:48470) 2024-04-22T07:13:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 -- 172.21.15.113:0/3095551325 learned_addr learned my addr 172.21.15.113:0/3095551325 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f9763fff700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f976406a750 0x7f9764101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f9768858700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9764108a30 0x7f9764102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 -- 172.21.15.113:0/3095551325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9764108a30 msgr2=0x7f9764102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9764108a30 0x7f9764102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 -- 172.21.15.113:0/3095551325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f976406a750 msgr2=0x7f9764101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f976406a750 0x7f9764101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 -- 172.21.15.113:0/3095551325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97641a77d0 con 0x7f976406b0f0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.167+0000 7f97637fe700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 0x7f9764101f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9754000c00 tx=0x7f9754009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.168+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754027070 con 0x7f976406b0f0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.168+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97641a7a60 con 0x7f976406b0f0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.168+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9754003fb0 con 0x7f976406b0f0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.168+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754022e10 con 0x7f976406b0f0 2024-04-22T07:13:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.168+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a7f10 con 0x7f976406b0f0 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.169+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764069730 con 0x7f976406b0f0 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.170+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f975402a070 con 0x7f976406b0f0 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.170+0000 7f97617fa700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f97500a3cd0 0x7f97500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.170+0000 7f9763fff700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f97500a3cd0 0x7f97500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.170+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f97540d10f0 con 0x7f976406b0f0 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.171+0000 7f9763fff700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f97500a3cd0 0x7f97500a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f974c005950 tx=0x7f974c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.175+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9754013070 con 0x7f976406b0f0 2024-04-22T07:13:57.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.188+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f99fc02df80 con 0x7f99fc101970 2024-04-22T07:13:57.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.188+0000 7f99f9ffb700 1 -- 172.21.15.113:0/2916741653 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99f409cd50 con 0x7f99fc101970 2024-04-22T07:13:57.186 INFO:teuthology.orchestra.run.smithi113.stdout:107374182466 2024-04-22T07:13:57.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f99e80a3cd0 msgr2=0x7f99e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f99e80a3cd0 0x7f99e80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99ec006010 tx=0x7f99ec00b540 comp rx=0 tx=0).stop 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 msgr2=0x7f99fc19a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc19a1c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f99f4007f10 tx=0x7f99f400d3b0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 shutdown_connections 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f99e80a3cd0 0x7f99e80a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99fc107920 0x7f99fc1a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc101970 0x7f99fc19a1c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 --2- 172.21.15.113:0/2916741653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99fc102330 0x7f99fc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 >> 172.21.15.113:0/2916741653 conn(0x7f99fc078910 msgr2=0x7f99fc0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:57.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 shutdown_connections 2024-04-22T07:13:57.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.191+0000 7f9a02b1d700 1 -- 172.21.15.113:0/2916741653 wait complete. 2024-04-22T07:13:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:57 smithi113 ceph-mon[27933]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T07:13:57.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.380+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f976402df80 con 0x7f976406b0f0 2024-04-22T07:13:57.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.381+0000 7f97617fa700 1 -- 172.21.15.113:0/3095551325 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f975409db30 con 0x7f976406b0f0 2024-04-22T07:13:57.379 INFO:teuthology.orchestra.run.smithi113.stdout:55834574925 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f97500a3cd0 msgr2=0x7f97500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f97500a3cd0 0x7f97500a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f974c005950 tx=0x7f974c00b410 comp rx=0 tx=0).stop 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 msgr2=0x7f9764101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 0x7f9764101f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9754000c00 tx=0x7f9754009f50 comp rx=0 tx=0).stop 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 shutdown_connections 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f97500a3cd0 0x7f97500a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9764108a30 0x7f9764102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f976406a750 0x7f9764101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 --2- 172.21.15.113:0/3095551325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f976406b0f0 0x7f9764101f50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 >> 172.21.15.113:0/3095551325 conn(0x7f97640788f0 msgr2=0x7f97640ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:57.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 shutdown_connections 2024-04-22T07:13:57.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:57.383+0000 7f976a2bb700 1 -- 172.21.15.113:0/3095551325 wait complete. 2024-04-22T07:13:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:57 smithi133 ceph-mon[31198]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T07:13:57.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:57 smithi154 ceph-mon[31862]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 -- 172.21.15.113:0/139118647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102ed0 msgr2=0x7fe83410e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 --2- 172.21.15.113:0/139118647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102ed0 0x7fe83410e8e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe8240099c0 tx=0x7fe824009cd0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 -- 172.21.15.113:0/139118647 shutdown_connections 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 --2- 172.21.15.113:0/139118647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102ed0 0x7fe83410e8e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 --2- 172.21.15.113:0/139118647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe834107c30 0x7fe834108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 --2- 172.21.15.113:0/139118647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834102510 0x7fe834102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 -- 172.21.15.113:0/139118647 >> 172.21.15.113:0/139118647 conn(0x7fe834076c60 msgr2=0x7fe834077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 -- 172.21.15.113:0/139118647 shutdown_connections 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 -- 172.21.15.113:0/139118647 wait complete. 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 Processor -- start 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.407+0000 7fe8397ee700 1 -- start start 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 0x7fe834198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe834102ed0 0x7fe834198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834107c30 0x7fe83419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe834111e70 con 0x7fe834102510 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe834111cf0 con 0x7fe834107c30 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe834111ff0 con 0x7fe834102ed0 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 0x7fe834198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 0x7fe834198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48486/0 (socket says 172.21.15.113:48486) 2024-04-22T07:13:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 -- 172.21.15.113:0/3245486858 learned_addr learned my addr 172.21.15.113:0/3245486858 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8327fc700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe834102ed0 0x7fe834198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8337fe700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834107c30 0x7fe83419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 -- 172.21.15.113:0/3245486858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe834102ed0 msgr2=0x7fe834198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe834102ed0 0x7fe834198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 -- 172.21.15.113:0/3245486858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834107c30 msgr2=0x7fe83419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834107c30 0x7fe83419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 -- 172.21.15.113:0/3245486858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83419d230 con 0x7fe834102510 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8337fe700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834107c30 0x7fe83419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe832ffd700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 0x7fe834198220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ba50 tx=0x7fe82c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c019070 con 0x7fe834102510 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83419d520 con 0x7fe834102510 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe82c014440 con 0x7fe834102510 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0075d0 con 0x7fe834102510 2024-04-22T07:13:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.408+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8341a4da0 con 0x7fe834102510 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.409+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe834069730 con 0x7fe834102510 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.411+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe82c01c070 con 0x7fe834102510 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.411+0000 7fe81bfff700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe81c0a3cd0 0x7fe81c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.411+0000 7fe8327fc700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe81c0a3cd0 0x7fe81c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.411+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe82c0cf9a0 con 0x7fe834102510 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.411+0000 7fe8327fc700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe81c0a3cd0 0x7fe81c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe828006fd0 tx=0x7fe828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:13:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.416+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe82c09c340 con 0x7fe834102510 2024-04-22T07:13:58.563 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:58 smithi113 ceph-mon[27933]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-22T07:13:58.563 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:58 smithi113 ceph-mon[27933]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T07:13:58.563 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:58 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2916741653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T07:13:58.563 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:58 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3095551325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T07:13:58.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:58 smithi133 ceph-mon[31198]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-22T07:13:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:58 smithi133 ceph-mon[31198]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T07:13:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:58 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2916741653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T07:13:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:58 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3095551325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T07:13:58.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:58 smithi154 ceph-mon[31862]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-22T07:13:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:58 smithi154 ceph-mon[31862]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T07:13:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:58 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2916741653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T07:13:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:58 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3095551325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T07:13:58.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.621+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe834199580 con 0x7fe834102510 2024-04-22T07:13:58.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.622+0000 7fe81bfff700 1 -- 172.21.15.113:0/3245486858 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe82c09bc40 con 0x7fe834102510 2024-04-22T07:13:58.620 INFO:teuthology.orchestra.run.smithi113.stdout:141733920829 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe81c0a3cd0 msgr2=0x7fe81c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe81c0a3cd0 0x7fe81c0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe828006fd0 tx=0x7fe828008040 comp rx=0 tx=0).stop 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 msgr2=0x7fe834198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 0x7fe834198220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ba50 tx=0x7fe82c00be10 comp rx=0 tx=0).stop 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 shutdown_connections 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe81c0a3cd0 0x7fe81c0a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe834107c30 0x7fe83419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe834102510 0x7fe834198220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 --2- 172.21.15.113:0/3245486858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe834102ed0 0x7fe834198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:13:58.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 >> 172.21.15.113:0/3245486858 conn(0x7fe834076c60 msgr2=0x7fe8340ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:13:58.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 shutdown_connections 2024-04-22T07:13:58.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:13:58.624+0000 7fe8397ee700 1 -- 172.21.15.113:0/3245486858 wait complete. 2024-04-22T07:13:58.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757296 got 163208757302 for osd.5 2024-04-22T07:13:58.895 DEBUG:teuthology.parallel:result is None 2024-04-22T07:13:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:13:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3245486858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T07:13:59.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:13:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3245486858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T07:13:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:13:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3245486858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T07:13:59.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495661 got 197568495665 for osd.6 2024-04-22T07:13:59.734 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:00.094 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182466 for osd.3 2024-04-22T07:14:00.094 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:00 smithi113 ceph-mon[27933]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T07:14:00.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411399 for osd.2 2024-04-22T07:14:00.439 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:00 smithi133 ceph-mon[31198]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T07:14:00.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:00 smithi154 ceph-mon[31862]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T07:14:00.865 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299437 for osd.7 2024-04-22T07:14:00.866 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:01.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 -- 172.21.15.113:0/3916099981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c103450 msgr2=0x7f5b6c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3916099981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c103450 0x7f5b6c10f8e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400b0a0 tx=0x7f5b6400b3b0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 -- 172.21.15.113:0/3916099981 shutdown_connections 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3916099981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c103450 0x7f5b6c10f8e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3916099981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c108a80 0x7f5b6c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3916099981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c102a90 0x7f5b6c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 -- 172.21.15.113:0/3916099981 >> 172.21.15.113:0/3916099981 conn(0x7f5b6c079bf0 msgr2=0x7f5b6c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 -- 172.21.15.113:0/3916099981 shutdown_connections 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.053+0000 7f5b746d0700 1 -- 172.21.15.113:0/3916099981 wait complete. 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 Processor -- start 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 -- start start 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 0x7f5b6c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c103450 0x7f5b6c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c108a80 0x7f5b6c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c112e50 con 0x7f5b6c102a90 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c112cd0 con 0x7f5b6c108a80 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b746d0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c112fd0 con 0x7f5b6c103450 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 0x7f5b6c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 0x7f5b6c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48510/0 (socket says 172.21.15.113:48510) 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 -- 172.21.15.113:0/3440765399 learned_addr learned my addr 172.21.15.113:0/3440765399 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b72c6d700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c108a80 0x7f5b6c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b71c6b700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c103450 0x7f5b6c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 -- 172.21.15.113:0/3440765399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c103450 msgr2=0x7f5b6c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c103450 0x7f5b6c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 -- 172.21.15.113:0/3440765399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c108a80 msgr2=0x7f5b6c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:01.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c108a80 0x7f5b6c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 -- 172.21.15.113:0/3440765399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1a7840 con 0x7f5b6c102a90 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b72c6d700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c108a80 0x7f5b6c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.054+0000 7f5b7246c700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 0x7f5b6c074d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800b530 tx=0x7f5b6800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.055+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68015020 con 0x7f5b6c102a90 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.055+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5b68011d00 con 0x7f5b6c102a90 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.055+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1a7b30 con 0x7f5b6c102a90 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.055+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68010e60 con 0x7f5b6c102a90 2024-04-22T07:14:01.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.055+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a8020 con 0x7f5b6c102a90 2024-04-22T07:14:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.056+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c069730 con 0x7f5b6c102a90 2024-04-22T07:14:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.058+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b68018020 con 0x7f5b6c102a90 2024-04-22T07:14:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.058+0000 7f5b637fe700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5b580a3cd0 0x7f5b580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.058+0000 7f5b71c6b700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5b580a3cd0 0x7f5b580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.058+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b680d0020 con 0x7f5b6c102a90 2024-04-22T07:14:01.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.059+0000 7f5b71c6b700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5b580a3cd0 0x7f5b580a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006fd0 tx=0x7f5b5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:01.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.062+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b6801f030 con 0x7f5b6c102a90 2024-04-22T07:14:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.268+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5b6c02df80 con 0x7f5b6c102a90 2024-04-22T07:14:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.268+0000 7f5b637fe700 1 -- 172.21.15.113:0/3440765399 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5b6809c9c0 con 0x7f5b6c102a90 2024-04-22T07:14:01.266 INFO:teuthology.orchestra.run.smithi113.stdout:347892351004 2024-04-22T07:14:01.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.270+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5b580a3cd0 msgr2=0x7f5b580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:01.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.270+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5b580a3cd0 0x7f5b580a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c006fd0 tx=0x7f5b5c008040 comp rx=0 tx=0).stop 2024-04-22T07:14:01.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.270+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 msgr2=0x7f5b6c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.270+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 0x7f5b6c074d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800b530 tx=0x7f5b6800b8f0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 shutdown_connections 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5b580a3cd0 0x7f5b580a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6c108a80 0x7f5b6c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b6c102a90 0x7f5b6c074d40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 --2- 172.21.15.113:0/3440765399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b6c103450 0x7f5b6c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 >> 172.21.15.113:0/3440765399 conn(0x7f5b6c079bf0 msgr2=0x7f5b6c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 shutdown_connections 2024-04-22T07:14:01.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:01.271+0000 7f5b746d0700 1 -- 172.21.15.113:0/3440765399 wait complete. 2024-04-22T07:14:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:01 smithi133 ceph-mon[31198]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:01 smithi133 ceph-mon[31198]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-22T07:14:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:01 smithi133 ceph-mon[31198]: Cluster is now healthy 2024-04-22T07:14:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:01 smithi154 ceph-mon[31862]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:01 smithi154 ceph-mon[31862]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-22T07:14:01.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:01 smithi154 ceph-mon[31862]: Cluster is now healthy 2024-04-22T07:14:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:01 smithi113 ceph-mon[27933]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:01 smithi113 ceph-mon[27933]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-22T07:14:01.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:01 smithi113 ceph-mon[27933]: Cluster is now healthy 2024-04-22T07:14:01.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350997 got 347892351004 for osd.11 2024-04-22T07:14:01.893 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3440765399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T07:14:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3440765399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T07:14:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3440765399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T07:14:02.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920829 for osd.4 2024-04-22T07:14:02.640 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:02.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574925 for osd.1 2024-04-22T07:14:02.974 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:03 smithi133 ceph-mon[31198]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:03 smithi154 ceph-mon[31862]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:03 smithi113 ceph-mon[27933]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 76 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 -- 172.21.15.113:0/3609761738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9920108a20 msgr2=0x7f9920108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 --2- 172.21.15.113:0/3609761738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9920108a20 0x7f9920108e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f991c009990 tx=0x7f991c009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 -- 172.21.15.113:0/3609761738 shutdown_connections 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 --2- 172.21.15.113:0/3609761738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 0x7f992010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 --2- 172.21.15.113:0/3609761738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9920108a20 0x7f9920108e00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 --2- 172.21.15.113:0/3609761738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f992006afe0 0x7f992006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 -- 172.21.15.113:0/3609761738 >> 172.21.15.113:0/3609761738 conn(0x7f99200788b0 msgr2=0x7f9920078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:03.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 -- 172.21.15.113:0/3609761738 shutdown_connections 2024-04-22T07:14:03.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.700+0000 7f992846a700 1 -- 172.21.15.113:0/3609761738 wait complete. 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 Processor -- start 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 -- start start 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 0x7f9920199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 0x7f9920199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9920108a20 0x7f992019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112de0 con 0x7f992006afe0 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112c60 con 0x7f9920108a20 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f992846a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112f60 con 0x7f992006b980 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f9926206700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 0x7f9920199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f9926206700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 0x7f9920199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48536/0 (socket says 172.21.15.113:48536) 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f9926206700 1 -- 172.21.15.113:0/1765182492 learned_addr learned my addr 172.21.15.113:0/1765182492 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.701+0000 7f9926a07700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9920108a20 0x7f992019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:03.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9925a05700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 0x7f9920199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9926206700 1 -- 172.21.15.113:0/1765182492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 msgr2=0x7f9920199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9926206700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 0x7f9920199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9926206700 1 -- 172.21.15.113:0/1765182492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9920108a20 msgr2=0x7f992019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9926206700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9920108a20 0x7f992019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9926206700 1 -- 172.21.15.113:0/1765182492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992019e3e0 con 0x7f992006afe0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9925a05700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 0x7f9920199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f9926206700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 0x7f9920199420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f991c000c00 tx=0x7f991c0037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c027070 con 0x7f992006afe0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992019e670 con 0x7f992006afe0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f991c0041e0 con 0x7f992006afe0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c00f460 con 0x7f992006afe0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.702+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a5f50 con 0x7f992006afe0 2024-04-22T07:14:03.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.703+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9920069730 con 0x7f992006afe0 2024-04-22T07:14:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.704+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f991c02a070 con 0x7f992006afe0 2024-04-22T07:14:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.705+0000 7f99177fe700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f990c0a3cd0 0x7f990c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:03.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.705+0000 7f9925a05700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f990c0a3cd0 0x7f990c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.705+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f991c0d1600 con 0x7f992006afe0 2024-04-22T07:14:03.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.705+0000 7f9925a05700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f990c0a3cd0 0x7f990c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9920079060 tx=0x7f9910008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:03.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.708+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f991c013070 con 0x7f992006afe0 2024-04-22T07:14:03.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.910+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f992002df80 con 0x7f992006afe0 2024-04-22T07:14:03.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.911+0000 7f99177fe700 1 -- 172.21.15.113:0/1765182492 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f991c09df20 con 0x7f992006afe0 2024-04-22T07:14:03.909 INFO:teuthology.orchestra.run.smithi113.stdout:313532612640 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f990c0a3cd0 msgr2=0x7f990c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f990c0a3cd0 0x7f990c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9920079060 tx=0x7f9910008040 comp rx=0 tx=0).stop 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 msgr2=0x7f9920199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 0x7f9920199420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f991c000c00 tx=0x7f991c0037a0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 shutdown_connections 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f990c0a3cd0 0x7f990c0a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9920108a20 0x7f992019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f992006afe0 0x7f9920199420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 --2- 172.21.15.113:0/1765182492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992006b980 0x7f9920199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 >> 172.21.15.113:0/1765182492 conn(0x7f99200788b0 msgr2=0x7f99200ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 shutdown_connections 2024-04-22T07:14:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:03.913+0000 7f992846a700 1 -- 172.21.15.113:0/1765182492 wait complete. 2024-04-22T07:14:04.435 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1765182492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T07:14:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1765182492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T07:14:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1765182492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 -- 172.21.15.113:0/443348145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e4108a80 msgr2=0x7f18e4108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 --2- 172.21.15.113:0/443348145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e4108a80 0x7f18e4108e60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009990 tx=0x7f18d4009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 -- 172.21.15.113:0/443348145 shutdown_connections 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 --2- 172.21.15.113:0/443348145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e41034b0 0x7f18e410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 --2- 172.21.15.113:0/443348145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e4108a80 0x7f18e4108e60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 --2- 172.21.15.113:0/443348145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4102af0 0x7f18e4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 -- 172.21.15.113:0/443348145 >> 172.21.15.113:0/443348145 conn(0x7f18e4079ad0 msgr2=0x7f18e4079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 -- 172.21.15.113:0/443348145 shutdown_connections 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.936+0000 7f18e9640700 1 -- 172.21.15.113:0/443348145 wait complete. 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 Processor -- start 2024-04-22T07:14:04.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 -- start start 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4102af0 0x7f18e4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 0x7f18e4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e4108a80 0x7f18e419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4112dd0 con 0x7f18e41034b0 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4112c50 con 0x7f18e4102af0 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e9640700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f18e4112f50 con 0x7f18e4108a80 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18da7fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 0x7f18e4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18da7fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 0x7f18e4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48554/0 (socket says 172.21.15.113:48554) 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18da7fc700 1 -- 172.21.15.113:0/4219200854 learned_addr learned my addr 172.21.15.113:0/4219200854 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e2ffd700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4102af0 0x7f18e4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.937+0000 7f18e37fe700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e4108a80 0x7f18e419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18da7fc700 1 -- 172.21.15.113:0/4219200854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e4108a80 msgr2=0x7f18e419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18da7fc700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e4108a80 0x7f18e419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18da7fc700 1 -- 172.21.15.113:0/4219200854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4102af0 msgr2=0x7f18e4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18da7fc700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4102af0 0x7f18e4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18da7fc700 1 -- 172.21.15.113:0/4219200854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e419e230 con 0x7f18e41034b0 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18e37fe700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e4108a80 0x7f18e419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18da7fc700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 0x7f18e4199760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d950 tx=0x7f18cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc016070 con 0x7f18e41034b0 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e419e520 con 0x7f18e41034b0 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f18cc009e10 con 0x7f18e41034b0 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00f620 con 0x7f18e41034b0 2024-04-22T07:14:04.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.938+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41a5da0 con 0x7f18e41034b0 2024-04-22T07:14:04.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.939+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e4069730 con 0x7f18e41034b0 2024-04-22T07:14:04.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.940+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f18cc019070 con 0x7f18e41034b0 2024-04-22T07:14:04.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.941+0000 7f18e0ff9700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f18d00a3cd0 0x7f18d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:04.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.941+0000 7f18e2ffd700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f18d00a3cd0 0x7f18d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:04.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.941+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f18cc0d0130 con 0x7f18e41034b0 2024-04-22T07:14:04.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.941+0000 7f18e2ffd700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f18d00a3cd0 0x7f18d00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098e0 tx=0x7f18d4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:04.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:04.945+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18cc09c9c0 con 0x7f18e41034b0 2024-04-22T07:14:05.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.149+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f18e402df80 con 0x7f18e41034b0 2024-04-22T07:14:05.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.150+0000 7f18e0ff9700 1 -- 172.21.15.113:0/4219200854 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18cc09c2c0 con 0x7f18e41034b0 2024-04-22T07:14:05.148 INFO:teuthology.orchestra.run.smithi113.stdout:34359738450 2024-04-22T07:14:05.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.151+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f18d00a3cd0 msgr2=0x7f18d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.151+0000 7f18e9640700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f18d00a3cd0 0x7f18d00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098e0 tx=0x7f18d4005960 comp rx=0 tx=0).stop 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 msgr2=0x7f18e4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 0x7f18e4199760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d950 tx=0x7f18cc00dc60 comp rx=0 tx=0).stop 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 shutdown_connections 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f18d00a3cd0 0x7f18d00a6190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e4108a80 0x7f18e419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4102af0 0x7f18e4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 --2- 172.21.15.113:0/4219200854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e41034b0 0x7f18e4199760 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 >> 172.21.15.113:0/4219200854 conn(0x7f18e4079ad0 msgr2=0x7f18e4101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 shutdown_connections 2024-04-22T07:14:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:05.152+0000 7f18e9640700 1 -- 172.21.15.113:0/4219200854 wait complete. 2024-04-22T07:14:05.380 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612640 for osd.10 2024-04-22T07:14:05.380 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:05.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:05 smithi133 ceph-mon[31198]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 62 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:05.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:05 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4219200854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T07:14:05.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:05 smithi154 ceph-mon[31862]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 62 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:05.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:05 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4219200854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T07:14:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:05 smithi113 ceph-mon[27933]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 62 B/s, 1 keys/s, 1 objects/s recovering 2024-04-22T07:14:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:05 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4219200854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T07:14:05.978 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738450 for osd.0 2024-04-22T07:14:05.978 DEBUG:teuthology.parallel:result is None 2024-04-22T07:14:05.978 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-22T07:14:05.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T07:14:06.733 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 -- 172.21.15.113:0/2327172301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 msgr2=0x7f78ec107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/2327172301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec107d20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009990 tx=0x7f78e8009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 -- 172.21.15.113:0/2327172301 shutdown_connections 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/2327172301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec102390 0x7f78ec10e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/2327172301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec107d20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/2327172301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78ec1019d0 0x7f78ec101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 -- 172.21.15.113:0/2327172301 >> 172.21.15.113:0/2327172301 conn(0x7f78ec077990 msgr2=0x7f78ec077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 -- 172.21.15.113:0/2327172301 shutdown_connections 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.288+0000 7f78f3dd0700 1 -- 172.21.15.113:0/2327172301 wait complete. 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 Processor -- start 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 -- start start 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78ec1019d0 0x7f78ec1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec102390 0x7f78ec199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec111c60 con 0x7f78ec107940 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec111ae0 con 0x7f78ec1019d0 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f3dd0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec111de0 con 0x7f78ec102390 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f236d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f236d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35750/0 (socket says 172.21.15.113:35750) 2024-04-22T07:14:08.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f236d700 1 -- 172.21.15.113:0/3697939658 learned_addr learned my addr 172.21.15.113:0/3697939658 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.289+0000 7f78f136b700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec102390 0x7f78ec199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f1b6c700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78ec1019d0 0x7f78ec1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f236d700 1 -- 172.21.15.113:0/3697939658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec102390 msgr2=0x7f78ec199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f236d700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec102390 0x7f78ec199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f236d700 1 -- 172.21.15.113:0/3697939658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78ec1019d0 msgr2=0x7f78ec1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f236d700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78ec1019d0 0x7f78ec1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f236d700 1 -- 172.21.15.113:0/3697939658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec19e200 con 0x7f78ec107940 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f236d700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec19dac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f78e4007ee0 tx=0x7f78e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400dbd0 con 0x7f78ec107940 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec19e4f0 con 0x7f78ec107940 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f78e400dd70 con 0x7f78ec107940 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4005a50 con 0x7f78ec107940 2024-04-22T07:14:08.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.290+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1a5d70 con 0x7f78ec107940 2024-04-22T07:14:08.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.291+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ec069730 con 0x7f78ec107940 2024-04-22T07:14:08.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.292+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f78e401a020 con 0x7f78ec107940 2024-04-22T07:14:08.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.293+0000 7f78e2ffd700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f78d80a3e70 0x7f78d80a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:08.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.293+0000 7f78f1b6c700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f78d80a3e70 0x7f78d80a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:08.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.293+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f78e40d0350 con 0x7f78ec107940 2024-04-22T07:14:08.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.293+0000 7f78f1b6c700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f78d80a3e70 0x7f78d80a6330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78e8006010 tx=0x7f78e800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.296+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78e4010070 con 0x7f78ec107940 2024-04-22T07:14:08.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:08 smithi113 ceph-mon[27933]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 56 B/s, 0 keys/s, 1 objects/s recovering 2024-04-22T07:14:08.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.539+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78ec19a550 con 0x7f78d80a3e70 2024-04-22T07:14:08.540 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:08 smithi154 ceph-mon[31862]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 56 B/s, 0 keys/s, 1 objects/s recovering 2024-04-22T07:14:08.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.552+0000 7f78e2ffd700 1 -- 172.21.15.113:0/3697939658 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258725 (secure 0 0 0) 0x7f78ec19a550 con 0x7f78d80a3e70 2024-04-22T07:14:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f78d80a3e70 msgr2=0x7f78d80a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f78d80a3e70 0x7f78d80a6330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78e8006010 tx=0x7f78e800b540 comp rx=0 tx=0).stop 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 msgr2=0x7f78ec19dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec19dac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f78e4007ee0 tx=0x7f78e400d3b0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 shutdown_connections 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f78d80a3e70 0x7f78d80a6330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78ec107940 0x7f78ec19dac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78ec1019d0 0x7f78ec1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 --2- 172.21.15.113:0/3697939658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78ec102390 0x7f78ec199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:08.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 >> 172.21.15.113:0/3697939658 conn(0x7f78ec077990 msgr2=0x7f78ec06b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 shutdown_connections 2024-04-22T07:14:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:08.555+0000 7f78f3dd0700 1 -- 172.21.15.113:0/3697939658 wait complete. 2024-04-22T07:14:08.558 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:14:08.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:08 smithi133 ceph-mon[31198]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 56 B/s, 0 keys/s, 1 objects/s recovering 2024-04-22T07:14:09.254 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-22T07:14:07.089755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655258,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":556,"num_read_kb":397,"num_write":338,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366696,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575128,"statfs":{"total":1151940427776,"available":1151564931072,"internally_reserved":0,"allocated":13627392,"data_stored":7497546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.905527"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186071+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:13:56.186071+0000","last_peered":"2024-04-22T07:13:56.186071+0000","last_clean":"2024-04-22T07:13:56.186071+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:13:56.186071+0000","last_undegraded":"2024-04-22T07:13:56.186071+0000","last_fullsized":"2024-04-22T07:13:56.186071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427115+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:13:56.427115+0000","last_peered":"2024-04-22T07:13:56.427115+0000","last_clean":"2024-04-22T07:13:56.427115+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:13:56.427115+0000","last_undegraded":"2024-04-22T07:13:56.427115+0000","last_fullsized":"2024-04-22T07:13:56.427115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426976+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:13:56.426976+0000","last_peered":"2024-04-22T07:13:56.426976+0000","last_clean":"2024-04-22T07:13:56.426976+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:13:56.426976+0000","last_undegraded":"2024-04-22T07:13:56.426976+0000","last_fullsized":"2024-04-22T07:13:56.426976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427176+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:13:56.427176+0000","last_peered":"2024-04-22T07:13:56.427176+0000","last_clean":"2024-04-22T07:13:56.427176+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:13:56.427176+0000","last_undegraded":"2024-04-22T07:13:56.427176+0000","last_fullsized":"2024-04-22T07:13:56.427176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187837+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:13:56.187837+0000","last_peered":"2024-04-22T07:13:56.187837+0000","last_clean":"2024-04-22T07:13:56.187837+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:13:56.187837+0000","last_undegraded":"2024-04-22T07:13:56.187837+0000","last_fullsized":"2024-04-22T07:13:56.187837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186077+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:13:56.186077+0000","last_peered":"2024-04-22T07:13:56.186077+0000","last_clean":"2024-04-22T07:13:56.186077+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:13:56.186077+0000","last_undegraded":"2024-04-22T07:13:56.186077+0000","last_fullsized":"2024-04-22T07:13:56.186077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187387+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:13:56.187387+0000","last_peered":"2024-04-22T07:13:56.187387+0000","last_clean":"2024-04-22T07:13:56.187387+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:13:56.187387+0000","last_undegraded":"2024-04-22T07:13:56.187387+0000","last_fullsized":"2024-04-22T07:13:56.187387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187759+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:13:56.187759+0000","last_peered":"2024-04-22T07:13:56.187759+0000","last_clean":"2024-04-22T07:13:56.187759+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:13:56.187759+0000","last_undegraded":"2024-04-22T07:13:56.187759+0000","last_fullsized":"2024-04-22T07:13:56.187759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189390+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:13:56.189390+0000","last_peered":"2024-04-22T07:13:56.189390+0000","last_clean":"2024-04-22T07:13:56.189390+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:13:56.189390+0000","last_undegraded":"2024-04-22T07:13:56.189390+0000","last_fullsized":"2024-04-22T07:13:56.189390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186260+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:13:56.186260+0000","last_peered":"2024-04-22T07:13:56.186260+0000","last_clean":"2024-04-22T07:13:56.186260+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:13:56.186260+0000","last_undegraded":"2024-04-22T07:13:56.186260+0000","last_fullsized":"2024-04-22T07:13:56.186260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188147+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:13:56.188147+0000","last_peered":"2024-04-22T07:13:56.188147+0000","last_clean":"2024-04-22T07:13:56.188147+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:13:56.188147+0000","last_undegraded":"2024-04-22T07:13:56.188147+0000","last_fullsized":"2024-04-22T07:13:56.188147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427139+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:13:56.427139+0000","last_peered":"2024-04-22T07:13:56.427139+0000","last_clean":"2024-04-22T07:13:56.427139+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:13:56.427139+0000","last_undegraded":"2024-04-22T07:13:56.427139+0000","last_fullsized":"2024-04-22T07:13:56.427139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426907+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:13:56.426907+0000","last_peered":"2024-04-22T07:13:56.426907+0000","last_clean":"2024-04-22T07:13:56.426907+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:13:56.426907+0000","last_undegraded":"2024-04-22T07:13:56.426907+0000","last_fullsized":"2024-04-22T07:13:56.426907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426641+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:13:56.426641+0000","last_peered":"2024-04-22T07:13:56.426641+0000","last_clean":"2024-04-22T07:13:56.426641+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:13:56.426641+0000","last_undegraded":"2024-04-22T07:13:56.426641+0000","last_fullsized":"2024-04-22T07:13:56.426641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426574+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:13:56.426574+0000","last_peered":"2024-04-22T07:13:56.426574+0000","last_clean":"2024-04-22T07:13:56.426574+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:13:56.426574+0000","last_undegraded":"2024-04-22T07:13:56.426574+0000","last_fullsized":"2024-04-22T07:13:56.426574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188769+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:13:56.188769+0000","last_peered":"2024-04-22T07:13:56.188769+0000","last_clean":"2024-04-22T07:13:56.188769+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:13:56.188769+0000","last_undegraded":"2024-04-22T07:13:56.188769+0000","last_fullsized":"2024-04-22T07:13:56.188769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186175+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:13:56.186175+0000","last_peered":"2024-04-22T07:13:56.186175+0000","last_clean":"2024-04-22T07:13:56.186175+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:13:56.186175+0000","last_undegraded":"2024-04-22T07:13:56.186175+0000","last_fullsized":"2024-04-22T07:13:56.186175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185353+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:13:56.185353+0000","last_peered":"2024-04-22T07:13:56.185353+0000","last_clean":"2024-04-22T07:13:56.185353+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:13:56.185353+0000","last_undegraded":"2024-04-22T07:13:56.185353+0000","last_fullsized":"2024-04-22T07:13:56.185353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189809+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:13:56.189809+0000","last_peered":"2024-04-22T07:13:56.189809+0000","last_clean":"2024-04-22T07:13:56.189809+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:13:56.189809+0000","last_undegraded":"2024-04-22T07:13:56.189809+0000","last_fullsized":"2024-04-22T07:13:56.189809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.196930+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:13:56.196930+0000","last_peered":"2024-04-22T07:13:56.196930+0000","last_clean":"2024-04-22T07:13:56.196930+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:13:56.196930+0000","last_undegraded":"2024-04-22T07:13:56.196930+0000","last_fullsized":"2024-04-22T07:13:56.196930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187929+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:13:56.187929+0000","last_peered":"2024-04-22T07:13:56.187929+0000","last_clean":"2024-04-22T07:13:56.187929+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:13:56.187929+0000","last_undegraded":"2024-04-22T07:13:56.187929+0000","last_fullsized":"2024-04-22T07:13:56.187929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188219+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:13:56.188219+0000","last_peered":"2024-04-22T07:13:56.188219+0000","last_clean":"2024-04-22T07:13:56.188219+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:13:56.188219+0000","last_undegraded":"2024-04-22T07:13:56.188219+0000","last_fullsized":"2024-04-22T07:13:56.188219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427193+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:13:56.427193+0000","last_peered":"2024-04-22T07:13:56.427193+0000","last_clean":"2024-04-22T07:13:56.427193+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:13:56.427193+0000","last_undegraded":"2024-04-22T07:13:56.427193+0000","last_fullsized":"2024-04-22T07:13:56.427193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188363+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:13:56.188363+0000","last_peered":"2024-04-22T07:13:56.188363+0000","last_clean":"2024-04-22T07:13:56.188363+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:13:56.188363+0000","last_undegraded":"2024-04-22T07:13:56.188363+0000","last_fullsized":"2024-04-22T07:13:56.188363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188250+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:13:56.188250+0000","last_peered":"2024-04-22T07:13:56.188250+0000","last_clean":"2024-04-22T07:13:56.188250+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:13:56.188250+0000","last_undegraded":"2024-04-22T07:13:56.188250+0000","last_fullsized":"2024-04-22T07:13:56.188250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.200838+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:13:56.200838+0000","last_peered":"2024-04-22T07:13:56.200838+0000","last_clean":"2024-04-22T07:13:56.200838+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:13:56.200838+0000","last_undegraded":"2024-04-22T07:13:56.200838+0000","last_fullsized":"2024-04-22T07:13:56.200838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190726+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:13:56.190726+0000","last_peered":"2024-04-22T07:13:56.190726+0000","last_clean":"2024-04-22T07:13:56.190726+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:13:56.190726+0000","last_undegraded":"2024-04-22T07:13:56.190726+0000","last_fullsized":"2024-04-22T07:13:56.190726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:01.434712+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:01.434712+0000","last_peered":"2024-04-22T07:14:01.434712+0000","last_clean":"2024-04-22T07:14:01.434712+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:01.434712+0000","last_undegraded":"2024-04-22T07:14:01.434712+0000","last_fullsized":"2024-04-22T07:14:01.434712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188083+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:13:56.188083+0000","last_peered":"2024-04-22T07:13:56.188083+0000","last_clean":"2024-04-22T07:13:56.188083+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:13:56.188083+0000","last_undegraded":"2024-04-22T07:13:56.188083+0000","last_fullsized":"2024-04-22T07:13:56.188083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186061+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:13:56.186061+0000","last_peered":"2024-04-22T07:13:56.186061+0000","last_clean":"2024-04-22T07:13:56.186061+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:13:56.186061+0000","last_undegraded":"2024-04-22T07:13:56.186061+0000","last_fullsized":"2024-04-22T07:13:56.186061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189974+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:13:56.189974+0000","last_peered":"2024-04-22T07:13:56.189974+0000","last_clean":"2024-04-22T07:13:56.189974+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:13:56.189974+0000","last_undegraded":"2024-04-22T07:13:56.189974+0000","last_fullsized":"2024-04-22T07:13:56.189974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426878+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:13:56.426878+0000","last_peered":"2024-04-22T07:13:56.426878+0000","last_clean":"2024-04-22T07:13:56.426878+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:13:56.426878+0000","last_undegraded":"2024-04-22T07:13:56.426878+0000","last_fullsized":"2024-04-22T07:13:56.426878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187699+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:13:56.187699+0000","last_peered":"2024-04-22T07:13:56.187699+0000","last_clean":"2024-04-22T07:13:56.187699+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:13:56.187699+0000","last_undegraded":"2024-04-22T07:13:56.187699+0000","last_fullsized":"2024-04-22T07:13:56.187699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.191059+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:13:56.191059+0000","last_peered":"2024-04-22T07:13:56.191059+0000","last_clean":"2024-04-22T07:13:56.191059+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:13:56.191059+0000","last_undegraded":"2024-04-22T07:13:56.191059+0000","last_fullsized":"2024-04-22T07:13:56.191059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185405+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:13:56.185405+0000","last_peered":"2024-04-22T07:13:56.185405+0000","last_clean":"2024-04-22T07:13:56.185405+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:13:56.185405+0000","last_undegraded":"2024-04-22T07:13:56.185405+0000","last_fullsized":"2024-04-22T07:13:56.185405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:02.435472+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:02.435472+0000","last_peered":"2024-04-22T07:14:02.435472+0000","last_clean":"2024-04-22T07:14:02.435472+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:02.435472+0000","last_undegraded":"2024-04-22T07:14:02.435472+0000","last_fullsized":"2024-04-22T07:14:02.435472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188637+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:13:56.188637+0000","last_peered":"2024-04-22T07:13:56.188637+0000","last_clean":"2024-04-22T07:13:56.188637+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:13:56.188637+0000","last_undegraded":"2024-04-22T07:13:56.188637+0000","last_fullsized":"2024-04-22T07:13:56.188637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188409+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:13:56.188409+0000","last_peered":"2024-04-22T07:13:56.188409+0000","last_clean":"2024-04-22T07:13:56.188409+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:13:56.188409+0000","last_undegraded":"2024-04-22T07:13:56.188409+0000","last_fullsized":"2024-04-22T07:13:56.188409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427191+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:13:56.427191+0000","last_peered":"2024-04-22T07:13:56.427191+0000","last_clean":"2024-04-22T07:13:56.427191+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:13:56.427191+0000","last_undegraded":"2024-04-22T07:13:56.427191+0000","last_fullsized":"2024-04-22T07:13:56.427191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190827+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:13:56.190827+0000","last_peered":"2024-04-22T07:13:56.190827+0000","last_clean":"2024-04-22T07:13:56.190827+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:13:56.190827+0000","last_undegraded":"2024-04-22T07:13:56.190827+0000","last_fullsized":"2024-04-22T07:13:56.190827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188362+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:13:56.188362+0000","last_peered":"2024-04-22T07:13:56.188362+0000","last_clean":"2024-04-22T07:13:56.188362+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:13:56.188362+0000","last_undegraded":"2024-04-22T07:13:56.188362+0000","last_fullsized":"2024-04-22T07:13:56.188362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":187,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:03.436824+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:03.436824+0000","last_peered":"2024-04-22T07:14:03.436824+0000","last_clean":"2024-04-22T07:14:03.436824+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:03.436824+0000","last_undegraded":"2024-04-22T07:14:03.436824+0000","last_fullsized":"2024-04-22T07:14:03.436824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":140,"num_read_kb":129,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187758+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:13:56.187758+0000","last_peered":"2024-04-22T07:13:56.187758+0000","last_clean":"2024-04-22T07:13:56.187758+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:13:56.187758+0000","last_undegraded":"2024-04-22T07:13:56.187758+0000","last_fullsized":"2024-04-22T07:13:56.187758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.198776+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:13:56.198776+0000","last_peered":"2024-04-22T07:13:56.198776+0000","last_clean":"2024-04-22T07:13:56.198776+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:13:56.198776+0000","last_undegraded":"2024-04-22T07:13:56.198776+0000","last_fullsized":"2024-04-22T07:13:56.198776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427295+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:13:56.427295+0000","last_peered":"2024-04-22T07:13:56.427295+0000","last_clean":"2024-04-22T07:13:56.427295+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:13:56.427295+0000","last_undegraded":"2024-04-22T07:13:56.427295+0000","last_fullsized":"2024-04-22T07:13:56.427295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189703+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:13:56.189703+0000","last_peered":"2024-04-22T07:13:56.189703+0000","last_clean":"2024-04-22T07:13:56.189703+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:13:56.189703+0000","last_undegraded":"2024-04-22T07:13:56.189703+0000","last_fullsized":"2024-04-22T07:13:56.189703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188014+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:13:56.188014+0000","last_peered":"2024-04-22T07:13:56.188014+0000","last_clean":"2024-04-22T07:13:56.188014+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:13:56.188014+0000","last_undegraded":"2024-04-22T07:13:56.188014+0000","last_fullsized":"2024-04-22T07:13:56.188014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:02.441161+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:02.441161+0000","last_peered":"2024-04-22T07:14:02.441161+0000","last_clean":"2024-04-22T07:14:02.441161+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:02.441161+0000","last_undegraded":"2024-04-22T07:14:02.441161+0000","last_fullsized":"2024-04-22T07:14:02.441161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187398+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:13:56.187398+0000","last_peered":"2024-04-22T07:13:56.187398+0000","last_clean":"2024-04-22T07:13:56.187398+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:13:56.187398+0000","last_undegraded":"2024-04-22T07:13:56.187398+0000","last_fullsized":"2024-04-22T07:13:56.187398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187869+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:13:56.187869+0000","last_peered":"2024-04-22T07:13:56.187869+0000","last_clean":"2024-04-22T07:13:56.187869+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:13:56.187869+0000","last_undegraded":"2024-04-22T07:13:56.187869+0000","last_fullsized":"2024-04-22T07:13:56.187869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190144+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:13:56.190144+0000","last_peered":"2024-04-22T07:13:56.190144+0000","last_clean":"2024-04-22T07:13:56.190144+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:13:56.190144+0000","last_undegraded":"2024-04-22T07:13:56.190144+0000","last_fullsized":"2024-04-22T07:13:56.190144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189378+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:13:56.189378+0000","last_peered":"2024-04-22T07:13:56.189378+0000","last_clean":"2024-04-22T07:13:56.189378+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:13:56.189378+0000","last_undegraded":"2024-04-22T07:13:56.189378+0000","last_fullsized":"2024-04-22T07:13:56.189378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185895+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:13:56.185895+0000","last_peered":"2024-04-22T07:13:56.185895+0000","last_clean":"2024-04-22T07:13:56.185895+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:13:56.185895+0000","last_undegraded":"2024-04-22T07:13:56.185895+0000","last_fullsized":"2024-04-22T07:13:56.185895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185400+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:13:56.185400+0000","last_peered":"2024-04-22T07:13:56.185400+0000","last_clean":"2024-04-22T07:13:56.185400+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:13:56.185400+0000","last_undegraded":"2024-04-22T07:13:56.185400+0000","last_fullsized":"2024-04-22T07:13:56.185400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186182+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:13:56.186182+0000","last_peered":"2024-04-22T07:13:56.186182+0000","last_clean":"2024-04-22T07:13:56.186182+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:13:56.186182+0000","last_undegraded":"2024-04-22T07:13:56.186182+0000","last_fullsized":"2024-04-22T07:13:56.186182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187814+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:13:56.187814+0000","last_peered":"2024-04-22T07:13:56.187814+0000","last_clean":"2024-04-22T07:13:56.187814+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:13:56.187814+0000","last_undegraded":"2024-04-22T07:13:56.187814+0000","last_fullsized":"2024-04-22T07:13:56.187814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.428324+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:13:56.428324+0000","last_peered":"2024-04-22T07:13:56.428324+0000","last_clean":"2024-04-22T07:13:56.428324+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:13:56.428324+0000","last_undegraded":"2024-04-22T07:13:56.428324+0000","last_fullsized":"2024-04-22T07:13:56.428324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.194115+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:13:56.194115+0000","last_peered":"2024-04-22T07:13:56.194115+0000","last_clean":"2024-04-22T07:13:56.194115+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:13:56.194115+0000","last_undegraded":"2024-04-22T07:13:56.194115+0000","last_fullsized":"2024-04-22T07:13:56.194115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:01.438088+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:01.438088+0000","last_peered":"2024-04-22T07:14:01.438088+0000","last_clean":"2024-04-22T07:14:01.438088+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:01.438088+0000","last_undegraded":"2024-04-22T07:14:01.438088+0000","last_fullsized":"2024-04-22T07:14:01.438088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188073+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:13:56.188073+0000","last_peered":"2024-04-22T07:13:56.188073+0000","last_clean":"2024-04-22T07:13:56.188073+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:13:56.188073+0000","last_undegraded":"2024-04-22T07:13:56.188073+0000","last_fullsized":"2024-04-22T07:13:56.188073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427498+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:13:56.427498+0000","last_peered":"2024-04-22T07:13:56.427498+0000","last_clean":"2024-04-22T07:13:56.427498+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:13:56.427498+0000","last_undegraded":"2024-04-22T07:13:56.427498+0000","last_fullsized":"2024-04-22T07:13:56.427498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187565+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:13:56.187565+0000","last_peered":"2024-04-22T07:13:56.187565+0000","last_clean":"2024-04-22T07:13:56.187565+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:13:56.187565+0000","last_undegraded":"2024-04-22T07:13:56.187565+0000","last_fullsized":"2024-04-22T07:13:56.187565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426618+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:13:56.426618+0000","last_peered":"2024-04-22T07:13:56.426618+0000","last_clean":"2024-04-22T07:13:56.426618+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:13:56.426618+0000","last_undegraded":"2024-04-22T07:13:56.426618+0000","last_fullsized":"2024-04-22T07:13:56.426618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187708+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:13:56.187708+0000","last_peered":"2024-04-22T07:13:56.187708+0000","last_clean":"2024-04-22T07:13:56.187708+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:13:56.187708+0000","last_undegraded":"2024-04-22T07:13:56.187708+0000","last_fullsized":"2024-04-22T07:13:56.187708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188179+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:13:56.188179+0000","last_peered":"2024-04-22T07:13:56.188179+0000","last_clean":"2024-04-22T07:13:56.188179+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:13:56.188179+0000","last_undegraded":"2024-04-22T07:13:56.188179+0000","last_fullsized":"2024-04-22T07:13:56.188179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426788+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:13:56.426788+0000","last_peered":"2024-04-22T07:13:56.426788+0000","last_clean":"2024-04-22T07:13:56.426788+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:13:56.426788+0000","last_undegraded":"2024-04-22T07:13:56.426788+0000","last_fullsized":"2024-04-22T07:13:56.426788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185360+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:13:56.185360+0000","last_peered":"2024-04-22T07:13:56.185360+0000","last_clean":"2024-04-22T07:13:56.185360+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:13:56.185360+0000","last_undegraded":"2024-04-22T07:13:56.185360+0000","last_fullsized":"2024-04-22T07:13:56.185360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187867+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:13:56.187867+0000","last_peered":"2024-04-22T07:13:56.187867+0000","last_clean":"2024-04-22T07:13:56.187867+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:13:56.187867+0000","last_undegraded":"2024-04-22T07:13:56.187867+0000","last_fullsized":"2024-04-22T07:13:56.187867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427108+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:13:56.427108+0000","last_peered":"2024-04-22T07:13:56.427108+0000","last_clean":"2024-04-22T07:13:56.427108+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:13:56.427108+0000","last_undegraded":"2024-04-22T07:13:56.427108+0000","last_fullsized":"2024-04-22T07:13:56.427108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427048+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:13:56.427048+0000","last_peered":"2024-04-22T07:13:56.427048+0000","last_clean":"2024-04-22T07:13:56.427048+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:13:56.427048+0000","last_undegraded":"2024-04-22T07:13:56.427048+0000","last_fullsized":"2024-04-22T07:13:56.427048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188070+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:13:56.188070+0000","last_peered":"2024-04-22T07:13:56.188070+0000","last_clean":"2024-04-22T07:13:56.188070+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:13:56.188070+0000","last_undegraded":"2024-04-22T07:13:56.188070+0000","last_fullsized":"2024-04-22T07:13:56.188070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187785+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:13:56.187785+0000","last_peered":"2024-04-22T07:13:56.187785+0000","last_clean":"2024-04-22T07:13:56.187785+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:13:56.187785+0000","last_undegraded":"2024-04-22T07:13:56.187785+0000","last_fullsized":"2024-04-22T07:13:56.187785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187805+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:13:56.187805+0000","last_peered":"2024-04-22T07:13:56.187805+0000","last_clean":"2024-04-22T07:13:56.187805+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:13:56.187805+0000","last_undegraded":"2024-04-22T07:13:56.187805+0000","last_fullsized":"2024-04-22T07:13:56.187805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185667+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:13:56.185667+0000","last_peered":"2024-04-22T07:13:56.185667+0000","last_clean":"2024-04-22T07:13:56.185667+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:13:56.185667+0000","last_undegraded":"2024-04-22T07:13:56.185667+0000","last_fullsized":"2024-04-22T07:13:56.185667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189265+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:13:56.189265+0000","last_peered":"2024-04-22T07:13:56.189265+0000","last_clean":"2024-04-22T07:13:56.189265+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:13:56.189265+0000","last_undegraded":"2024-04-22T07:13:56.189265+0000","last_fullsized":"2024-04-22T07:13:56.189265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187980+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:13:56.187980+0000","last_peered":"2024-04-22T07:13:56.187980+0000","last_clean":"2024-04-22T07:13:56.187980+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:13:56.187980+0000","last_undegraded":"2024-04-22T07:13:56.187980+0000","last_fullsized":"2024-04-22T07:13:56.187980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.431722+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:13:56.431722+0000","last_peered":"2024-04-22T07:13:56.431722+0000","last_clean":"2024-04-22T07:13:56.431722+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:13:56.431722+0000","last_undegraded":"2024-04-22T07:13:56.431722+0000","last_fullsized":"2024-04-22T07:13:56.431722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187874+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:13:56.187874+0000","last_peered":"2024-04-22T07:13:56.187874+0000","last_clean":"2024-04-22T07:13:56.187874+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:13:56.187874+0000","last_undegraded":"2024-04-22T07:13:56.187874+0000","last_fullsized":"2024-04-22T07:13:56.187874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189645+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:13:56.189645+0000","last_peered":"2024-04-22T07:13:56.189645+0000","last_clean":"2024-04-22T07:13:56.189645+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:13:56.189645+0000","last_undegraded":"2024-04-22T07:13:56.189645+0000","last_fullsized":"2024-04-22T07:13:56.189645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188304+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:13:56.188304+0000","last_peered":"2024-04-22T07:13:56.188304+0000","last_clean":"2024-04-22T07:13:56.188304+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:13:56.188304+0000","last_undegraded":"2024-04-22T07:13:56.188304+0000","last_fullsized":"2024-04-22T07:13:56.188304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190614+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:13:56.190614+0000","last_peered":"2024-04-22T07:13:56.190614+0000","last_clean":"2024-04-22T07:13:56.190614+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:13:56.190614+0000","last_undegraded":"2024-04-22T07:13:56.190614+0000","last_fullsized":"2024-04-22T07:13:56.190614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187956+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:13:56.187956+0000","last_peered":"2024-04-22T07:13:56.187956+0000","last_clean":"2024-04-22T07:13:56.187956+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:13:56.187956+0000","last_undegraded":"2024-04-22T07:13:56.187956+0000","last_fullsized":"2024-04-22T07:13:56.187956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427039+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:13:56.427039+0000","last_peered":"2024-04-22T07:13:56.427039+0000","last_clean":"2024-04-22T07:13:56.427039+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:13:56.427039+0000","last_undegraded":"2024-04-22T07:13:56.427039+0000","last_fullsized":"2024-04-22T07:13:56.427039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188140+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:13:56.188140+0000","last_peered":"2024-04-22T07:13:56.188140+0000","last_clean":"2024-04-22T07:13:56.188140+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:13:56.188140+0000","last_undegraded":"2024-04-22T07:13:56.188140+0000","last_fullsized":"2024-04-22T07:13:56.188140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185913+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:13:56.185913+0000","last_peered":"2024-04-22T07:13:56.185913+0000","last_clean":"2024-04-22T07:13:56.185913+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:13:56.185913+0000","last_undegraded":"2024-04-22T07:13:56.185913+0000","last_fullsized":"2024-04-22T07:13:56.185913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.428079+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:13:56.428079+0000","last_peered":"2024-04-22T07:13:56.428079+0000","last_clean":"2024-04-22T07:13:56.428079+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:13:56.428079+0000","last_undegraded":"2024-04-22T07:13:56.428079+0000","last_fullsized":"2024-04-22T07:13:56.428079+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189147+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:13:56.189147+0000","last_peered":"2024-04-22T07:13:56.189147+0000","last_clean":"2024-04-22T07:13:56.189147+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:13:56.189147+0000","last_undegraded":"2024-04-22T07:13:56.189147+0000","last_fullsized":"2024-04-22T07:13:56.189147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426791+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:13:56.426791+0000","last_peered":"2024-04-22T07:13:56.426791+0000","last_clean":"2024-04-22T07:13:56.426791+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:13:56.426791+0000","last_undegraded":"2024-04-22T07:13:56.426791+0000","last_fullsized":"2024-04-22T07:13:56.426791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426686+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:13:56.426686+0000","last_peered":"2024-04-22T07:13:56.426686+0000","last_clean":"2024-04-22T07:13:56.426686+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:13:56.426686+0000","last_undegraded":"2024-04-22T07:13:56.426686+0000","last_fullsized":"2024-04-22T07:13:56.426686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185881+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:13:56.185881+0000","last_peered":"2024-04-22T07:13:56.185881+0000","last_clean":"2024-04-22T07:13:56.185881+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:13:56.185881+0000","last_undegraded":"2024-04-22T07:13:56.185881+0000","last_fullsized":"2024-04-22T07:13:56.185881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190820+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:13:56.190820+0000","last_peered":"2024-04-22T07:13:56.190820+0000","last_clean":"2024-04-22T07:13:56.190820+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:13:56.190820+0000","last_undegraded":"2024-04-22T07:13:56.190820+0000","last_fullsized":"2024-04-22T07:13:56.190820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.428831+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:13:56.428831+0000","last_peered":"2024-04-22T07:13:56.428831+0000","last_clean":"2024-04-22T07:13:56.428831+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:13:56.428831+0000","last_undegraded":"2024-04-22T07:13:56.428831+0000","last_fullsized":"2024-04-22T07:13:56.428831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190513+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:13:56.190513+0000","last_peered":"2024-04-22T07:13:56.190513+0000","last_clean":"2024-04-22T07:13:56.190513+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:13:56.190513+0000","last_undegraded":"2024-04-22T07:13:56.190513+0000","last_fullsized":"2024-04-22T07:13:56.190513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187752+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:13:56.187752+0000","last_peered":"2024-04-22T07:13:56.187752+0000","last_clean":"2024-04-22T07:13:56.187752+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:13:56.187752+0000","last_undegraded":"2024-04-22T07:13:56.187752+0000","last_fullsized":"2024-04-22T07:13:56.187752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188539+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:13:56.188539+0000","last_peered":"2024-04-22T07:13:56.188539+0000","last_clean":"2024-04-22T07:13:56.188539+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:13:56.188539+0000","last_undegraded":"2024-04-22T07:13:56.188539+0000","last_fullsized":"2024-04-22T07:13:56.188539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426810+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:13:56.426810+0000","last_peered":"2024-04-22T07:13:56.426810+0000","last_clean":"2024-04-22T07:13:56.426810+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:13:56.426810+0000","last_undegraded":"2024-04-22T07:13:56.426810+0000","last_fullsized":"2024-04-22T07:13:56.426810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185785+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:13:56.185785+0000","last_peered":"2024-04-22T07:13:56.185785+0000","last_clean":"2024-04-22T07:13:56.185785+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:13:56.185785+0000","last_undegraded":"2024-04-22T07:13:56.185785+0000","last_fullsized":"2024-04-22T07:13:56.185785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64656,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":420,"num_read_kb":278,"num_write":155,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":461597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1114090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":970752,"data_stored":461799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":461137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1564672,"data_stored":1053258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":461188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":522609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":462459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":523343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":461625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1052942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:14:09.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T07:14:09.499 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:09 smithi133 ceph-mon[31198]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:09.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T07:14:09.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:09.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:09.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:09 smithi154 ceph-mon[31862]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.858 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:09.858 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.858 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:09.858 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T07:14:09.858 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:09.859 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:09.859 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:09 smithi113 ceph-mon[27933]: from='mgr.14150 172.21.15.113:0/3343006481' entity='mgr.x' 2024-04-22T07:14:11.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 -- 172.21.15.113:0/3339307980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 msgr2=0x7f023810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:11.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 --2- 172.21.15.113:0/3339307980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f023810f8d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f023000b0a0 tx=0x7f023000b3b0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 -- 172.21.15.113:0/3339307980 shutdown_connections 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 --2- 172.21.15.113:0/3339307980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f023810f8d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 --2- 172.21.15.113:0/3339307980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 0x7f0238108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 --2- 172.21.15.113:0/3339307980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0238102ac0 0x7f0238102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 -- 172.21.15.113:0/3339307980 >> 172.21.15.113:0/3339307980 conn(0x7f0238079bb0 msgr2=0x7f0238079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 -- 172.21.15.113:0/3339307980 shutdown_connections 2024-04-22T07:14:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.028+0000 7f023ce66700 1 -- 172.21.15.113:0/3339307980 wait complete. 2024-04-22T07:14:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 Processor -- start 2024-04-22T07:14:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 -- start start 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0238102ac0 0x7f0238199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f0238199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 0x7f023819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0238112e40 con 0x7f0238103480 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0238112cc0 con 0x7f0238108a70 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f023ce66700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0238112fc0 con 0x7f0238102ac0 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.029+0000 7f0235d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f0238199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f0238199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35760/0 (socket says 172.21.15.113:35760) 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 -- 172.21.15.113:0/2295835495 learned_addr learned my addr 172.21.15.113:0/2295835495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0236d9d700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 0x7f023819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f023659c700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0238102ac0 0x7f0238199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 -- 172.21.15.113:0/2295835495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0238102ac0 msgr2=0x7f0238199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0238102ac0 0x7f0238199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 -- 172.21.15.113:0/2295835495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 msgr2=0x7f023819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 0x7f023819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 -- 172.21.15.113:0/2295835495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023819e2d0 con 0x7f0238103480 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0236d9d700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 0x7f023819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f0235d9b700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f0238199850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f022c00b530 tx=0x7f022c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c015020 con 0x7f0238103480 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023819e5c0 con 0x7f0238103480 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f022c011d00 con 0x7f0238103480 2024-04-22T07:14:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c010e60 con 0x7f0238103480 2024-04-22T07:14:11.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.030+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02381a5e40 con 0x7f0238103480 2024-04-22T07:14:11.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.031+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0238069730 con 0x7f0238103480 2024-04-22T07:14:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.032+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f022c018020 con 0x7f0238103480 2024-04-22T07:14:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.033+0000 7f02277fe700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f02200a3cd0 0x7f02200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.033+0000 7f023659c700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f02200a3cd0 0x7f02200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.033+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f022c0d0230 con 0x7f0238103480 2024-04-22T07:14:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.034+0000 7f023659c700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f02200a3cd0 0x7f02200a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0228006fd0 tx=0x7f0228008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:11.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.037+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f022c01f020 con 0x7f0238103480 2024-04-22T07:14:11.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.232+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0238062e80 con 0x7f02200a3cd0 2024-04-22T07:14:11.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.240+0000 7f02277fe700 1 -- 172.21.15.113:0/2295835495 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258724 (secure 0 0 0) 0x7f0238062e80 con 0x7f02200a3cd0 2024-04-22T07:14:11.244 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f02200a3cd0 msgr2=0x7f02200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f02200a3cd0 0x7f02200a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0228006fd0 tx=0x7f0228008040 comp rx=0 tx=0).stop 2024-04-22T07:14:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 msgr2=0x7f0238199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f0238199850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f022c00b530 tx=0x7f022c00b8f0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 shutdown_connections 2024-04-22T07:14:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f02200a3cd0 0x7f02200a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0238108a70 0x7f023819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0238102ac0 0x7f0238199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 --2- 172.21.15.113:0/2295835495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0238103480 0x7f0238199850 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 >> 172.21.15.113:0/2295835495 conn(0x7f0238079bb0 msgr2=0x7f0238101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 shutdown_connections 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:11.244+0000 7f023ce66700 1 -- 172.21.15.113:0/2295835495 wait complete. 2024-04-22T07:14:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:14:11.835 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-22T07:14:11.090523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655258,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":586,"num_read_kb":412,"num_write":338,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366696,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575128,"statfs":{"total":1151940427776,"available":1151564931072,"internally_reserved":0,"allocated":13627392,"data_stored":7497546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002552"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186071+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:13:56.186071+0000","last_peered":"2024-04-22T07:13:56.186071+0000","last_clean":"2024-04-22T07:13:56.186071+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:13:56.186071+0000","last_undegraded":"2024-04-22T07:13:56.186071+0000","last_fullsized":"2024-04-22T07:13:56.186071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427115+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:13:56.427115+0000","last_peered":"2024-04-22T07:13:56.427115+0000","last_clean":"2024-04-22T07:13:56.427115+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:13:56.427115+0000","last_undegraded":"2024-04-22T07:13:56.427115+0000","last_fullsized":"2024-04-22T07:13:56.427115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426976+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:13:56.426976+0000","last_peered":"2024-04-22T07:13:56.426976+0000","last_clean":"2024-04-22T07:13:56.426976+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:13:56.426976+0000","last_undegraded":"2024-04-22T07:13:56.426976+0000","last_fullsized":"2024-04-22T07:13:56.426976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427176+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:13:56.427176+0000","last_peered":"2024-04-22T07:13:56.427176+0000","last_clean":"2024-04-22T07:13:56.427176+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:13:56.427176+0000","last_undegraded":"2024-04-22T07:13:56.427176+0000","last_fullsized":"2024-04-22T07:13:56.427176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187837+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:13:56.187837+0000","last_peered":"2024-04-22T07:13:56.187837+0000","last_clean":"2024-04-22T07:13:56.187837+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:13:56.187837+0000","last_undegraded":"2024-04-22T07:13:56.187837+0000","last_fullsized":"2024-04-22T07:13:56.187837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186077+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:13:56.186077+0000","last_peered":"2024-04-22T07:13:56.186077+0000","last_clean":"2024-04-22T07:13:56.186077+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:13:56.186077+0000","last_undegraded":"2024-04-22T07:13:56.186077+0000","last_fullsized":"2024-04-22T07:13:56.186077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187387+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:13:56.187387+0000","last_peered":"2024-04-22T07:13:56.187387+0000","last_clean":"2024-04-22T07:13:56.187387+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:13:56.187387+0000","last_undegraded":"2024-04-22T07:13:56.187387+0000","last_fullsized":"2024-04-22T07:13:56.187387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187759+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:13:56.187759+0000","last_peered":"2024-04-22T07:13:56.187759+0000","last_clean":"2024-04-22T07:13:56.187759+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:13:56.187759+0000","last_undegraded":"2024-04-22T07:13:56.187759+0000","last_fullsized":"2024-04-22T07:13:56.187759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189390+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:13:56.189390+0000","last_peered":"2024-04-22T07:13:56.189390+0000","last_clean":"2024-04-22T07:13:56.189390+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:13:56.189390+0000","last_undegraded":"2024-04-22T07:13:56.189390+0000","last_fullsized":"2024-04-22T07:13:56.189390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186260+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:13:56.186260+0000","last_peered":"2024-04-22T07:13:56.186260+0000","last_clean":"2024-04-22T07:13:56.186260+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:13:56.186260+0000","last_undegraded":"2024-04-22T07:13:56.186260+0000","last_fullsized":"2024-04-22T07:13:56.186260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188147+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:13:56.188147+0000","last_peered":"2024-04-22T07:13:56.188147+0000","last_clean":"2024-04-22T07:13:56.188147+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:13:56.188147+0000","last_undegraded":"2024-04-22T07:13:56.188147+0000","last_fullsized":"2024-04-22T07:13:56.188147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427139+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:13:56.427139+0000","last_peered":"2024-04-22T07:13:56.427139+0000","last_clean":"2024-04-22T07:13:56.427139+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:13:56.427139+0000","last_undegraded":"2024-04-22T07:13:56.427139+0000","last_fullsized":"2024-04-22T07:13:56.427139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426907+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:13:56.426907+0000","last_peered":"2024-04-22T07:13:56.426907+0000","last_clean":"2024-04-22T07:13:56.426907+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:13:56.426907+0000","last_undegraded":"2024-04-22T07:13:56.426907+0000","last_fullsized":"2024-04-22T07:13:56.426907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426641+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:13:56.426641+0000","last_peered":"2024-04-22T07:13:56.426641+0000","last_clean":"2024-04-22T07:13:56.426641+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:13:56.426641+0000","last_undegraded":"2024-04-22T07:13:56.426641+0000","last_fullsized":"2024-04-22T07:13:56.426641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426574+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:13:56.426574+0000","last_peered":"2024-04-22T07:13:56.426574+0000","last_clean":"2024-04-22T07:13:56.426574+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:13:56.426574+0000","last_undegraded":"2024-04-22T07:13:56.426574+0000","last_fullsized":"2024-04-22T07:13:56.426574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188769+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:13:56.188769+0000","last_peered":"2024-04-22T07:13:56.188769+0000","last_clean":"2024-04-22T07:13:56.188769+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:13:56.188769+0000","last_undegraded":"2024-04-22T07:13:56.188769+0000","last_fullsized":"2024-04-22T07:13:56.188769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186175+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:13:56.186175+0000","last_peered":"2024-04-22T07:13:56.186175+0000","last_clean":"2024-04-22T07:13:56.186175+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:13:56.186175+0000","last_undegraded":"2024-04-22T07:13:56.186175+0000","last_fullsized":"2024-04-22T07:13:56.186175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185353+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:13:56.185353+0000","last_peered":"2024-04-22T07:13:56.185353+0000","last_clean":"2024-04-22T07:13:56.185353+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:13:56.185353+0000","last_undegraded":"2024-04-22T07:13:56.185353+0000","last_fullsized":"2024-04-22T07:13:56.185353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189809+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:13:56.189809+0000","last_peered":"2024-04-22T07:13:56.189809+0000","last_clean":"2024-04-22T07:13:56.189809+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:13:56.189809+0000","last_undegraded":"2024-04-22T07:13:56.189809+0000","last_fullsized":"2024-04-22T07:13:56.189809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.196930+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:13:56.196930+0000","last_peered":"2024-04-22T07:13:56.196930+0000","last_clean":"2024-04-22T07:13:56.196930+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:13:56.196930+0000","last_undegraded":"2024-04-22T07:13:56.196930+0000","last_fullsized":"2024-04-22T07:13:56.196930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187929+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:13:56.187929+0000","last_peered":"2024-04-22T07:13:56.187929+0000","last_clean":"2024-04-22T07:13:56.187929+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:13:56.187929+0000","last_undegraded":"2024-04-22T07:13:56.187929+0000","last_fullsized":"2024-04-22T07:13:56.187929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188219+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:13:56.188219+0000","last_peered":"2024-04-22T07:13:56.188219+0000","last_clean":"2024-04-22T07:13:56.188219+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:13:56.188219+0000","last_undegraded":"2024-04-22T07:13:56.188219+0000","last_fullsized":"2024-04-22T07:13:56.188219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427193+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:13:56.427193+0000","last_peered":"2024-04-22T07:13:56.427193+0000","last_clean":"2024-04-22T07:13:56.427193+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:13:56.427193+0000","last_undegraded":"2024-04-22T07:13:56.427193+0000","last_fullsized":"2024-04-22T07:13:56.427193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188363+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:13:56.188363+0000","last_peered":"2024-04-22T07:13:56.188363+0000","last_clean":"2024-04-22T07:13:56.188363+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:13:56.188363+0000","last_undegraded":"2024-04-22T07:13:56.188363+0000","last_fullsized":"2024-04-22T07:13:56.188363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188250+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:13:56.188250+0000","last_peered":"2024-04-22T07:13:56.188250+0000","last_clean":"2024-04-22T07:13:56.188250+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:13:56.188250+0000","last_undegraded":"2024-04-22T07:13:56.188250+0000","last_fullsized":"2024-04-22T07:13:56.188250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.200838+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:13:56.200838+0000","last_peered":"2024-04-22T07:13:56.200838+0000","last_clean":"2024-04-22T07:13:56.200838+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:13:56.200838+0000","last_undegraded":"2024-04-22T07:13:56.200838+0000","last_fullsized":"2024-04-22T07:13:56.200838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190726+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:13:56.190726+0000","last_peered":"2024-04-22T07:13:56.190726+0000","last_clean":"2024-04-22T07:13:56.190726+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:13:56.190726+0000","last_undegraded":"2024-04-22T07:13:56.190726+0000","last_fullsized":"2024-04-22T07:13:56.190726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:06.441726+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:06.441726+0000","last_peered":"2024-04-22T07:14:06.441726+0000","last_clean":"2024-04-22T07:14:06.441726+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:06.441726+0000","last_undegraded":"2024-04-22T07:14:06.441726+0000","last_fullsized":"2024-04-22T07:14:06.441726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188083+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:13:56.188083+0000","last_peered":"2024-04-22T07:13:56.188083+0000","last_clean":"2024-04-22T07:13:56.188083+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:13:56.188083+0000","last_undegraded":"2024-04-22T07:13:56.188083+0000","last_fullsized":"2024-04-22T07:13:56.188083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186061+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:13:56.186061+0000","last_peered":"2024-04-22T07:13:56.186061+0000","last_clean":"2024-04-22T07:13:56.186061+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:13:56.186061+0000","last_undegraded":"2024-04-22T07:13:56.186061+0000","last_fullsized":"2024-04-22T07:13:56.186061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189974+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:13:56.189974+0000","last_peered":"2024-04-22T07:13:56.189974+0000","last_clean":"2024-04-22T07:13:56.189974+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:13:56.189974+0000","last_undegraded":"2024-04-22T07:13:56.189974+0000","last_fullsized":"2024-04-22T07:13:56.189974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426878+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:13:56.426878+0000","last_peered":"2024-04-22T07:13:56.426878+0000","last_clean":"2024-04-22T07:13:56.426878+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:13:56.426878+0000","last_undegraded":"2024-04-22T07:13:56.426878+0000","last_fullsized":"2024-04-22T07:13:56.426878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187699+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:13:56.187699+0000","last_peered":"2024-04-22T07:13:56.187699+0000","last_clean":"2024-04-22T07:13:56.187699+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:13:56.187699+0000","last_undegraded":"2024-04-22T07:13:56.187699+0000","last_fullsized":"2024-04-22T07:13:56.187699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.191059+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:13:56.191059+0000","last_peered":"2024-04-22T07:13:56.191059+0000","last_clean":"2024-04-22T07:13:56.191059+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:13:56.191059+0000","last_undegraded":"2024-04-22T07:13:56.191059+0000","last_fullsized":"2024-04-22T07:13:56.191059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185405+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:13:56.185405+0000","last_peered":"2024-04-22T07:13:56.185405+0000","last_clean":"2024-04-22T07:13:56.185405+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:13:56.185405+0000","last_undegraded":"2024-04-22T07:13:56.185405+0000","last_fullsized":"2024-04-22T07:13:56.185405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:07.442490+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:07.442490+0000","last_peered":"2024-04-22T07:14:07.442490+0000","last_clean":"2024-04-22T07:14:07.442490+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:07.442490+0000","last_undegraded":"2024-04-22T07:14:07.442490+0000","last_fullsized":"2024-04-22T07:14:07.442490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188637+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:13:56.188637+0000","last_peered":"2024-04-22T07:13:56.188637+0000","last_clean":"2024-04-22T07:13:56.188637+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:13:56.188637+0000","last_undegraded":"2024-04-22T07:13:56.188637+0000","last_fullsized":"2024-04-22T07:13:56.188637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188409+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:13:56.188409+0000","last_peered":"2024-04-22T07:13:56.188409+0000","last_clean":"2024-04-22T07:13:56.188409+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:13:56.188409+0000","last_undegraded":"2024-04-22T07:13:56.188409+0000","last_fullsized":"2024-04-22T07:13:56.188409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427191+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:13:56.427191+0000","last_peered":"2024-04-22T07:13:56.427191+0000","last_clean":"2024-04-22T07:13:56.427191+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:13:56.427191+0000","last_undegraded":"2024-04-22T07:13:56.427191+0000","last_fullsized":"2024-04-22T07:13:56.427191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190827+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:13:56.190827+0000","last_peered":"2024-04-22T07:13:56.190827+0000","last_clean":"2024-04-22T07:13:56.190827+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:13:56.190827+0000","last_undegraded":"2024-04-22T07:13:56.190827+0000","last_fullsized":"2024-04-22T07:13:56.190827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188362+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:13:56.188362+0000","last_peered":"2024-04-22T07:13:56.188362+0000","last_clean":"2024-04-22T07:13:56.188362+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:13:56.188362+0000","last_undegraded":"2024-04-22T07:13:56.188362+0000","last_fullsized":"2024-04-22T07:13:56.188362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":197,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:08.443894+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:08.443894+0000","last_peered":"2024-04-22T07:14:08.443894+0000","last_clean":"2024-04-22T07:14:08.443894+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:08.443894+0000","last_undegraded":"2024-04-22T07:14:08.443894+0000","last_fullsized":"2024-04-22T07:14:08.443894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":150,"num_read_kb":134,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187758+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:13:56.187758+0000","last_peered":"2024-04-22T07:13:56.187758+0000","last_clean":"2024-04-22T07:13:56.187758+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:13:56.187758+0000","last_undegraded":"2024-04-22T07:13:56.187758+0000","last_fullsized":"2024-04-22T07:13:56.187758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.198776+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:13:56.198776+0000","last_peered":"2024-04-22T07:13:56.198776+0000","last_clean":"2024-04-22T07:13:56.198776+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:13:56.198776+0000","last_undegraded":"2024-04-22T07:13:56.198776+0000","last_fullsized":"2024-04-22T07:13:56.198776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427295+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:13:56.427295+0000","last_peered":"2024-04-22T07:13:56.427295+0000","last_clean":"2024-04-22T07:13:56.427295+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:13:56.427295+0000","last_undegraded":"2024-04-22T07:13:56.427295+0000","last_fullsized":"2024-04-22T07:13:56.427295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189703+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:13:56.189703+0000","last_peered":"2024-04-22T07:13:56.189703+0000","last_clean":"2024-04-22T07:13:56.189703+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:13:56.189703+0000","last_undegraded":"2024-04-22T07:13:56.189703+0000","last_fullsized":"2024-04-22T07:13:56.189703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188014+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:13:56.188014+0000","last_peered":"2024-04-22T07:13:56.188014+0000","last_clean":"2024-04-22T07:13:56.188014+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:13:56.188014+0000","last_undegraded":"2024-04-22T07:13:56.188014+0000","last_fullsized":"2024-04-22T07:13:56.188014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:07.450333+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:07.450333+0000","last_peered":"2024-04-22T07:14:07.450333+0000","last_clean":"2024-04-22T07:14:07.450333+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:07.450333+0000","last_undegraded":"2024-04-22T07:14:07.450333+0000","last_fullsized":"2024-04-22T07:14:07.450333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187398+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:13:56.187398+0000","last_peered":"2024-04-22T07:13:56.187398+0000","last_clean":"2024-04-22T07:13:56.187398+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:13:56.187398+0000","last_undegraded":"2024-04-22T07:13:56.187398+0000","last_fullsized":"2024-04-22T07:13:56.187398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187869+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:13:56.187869+0000","last_peered":"2024-04-22T07:13:56.187869+0000","last_clean":"2024-04-22T07:13:56.187869+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:13:56.187869+0000","last_undegraded":"2024-04-22T07:13:56.187869+0000","last_fullsized":"2024-04-22T07:13:56.187869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190144+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:13:56.190144+0000","last_peered":"2024-04-22T07:13:56.190144+0000","last_clean":"2024-04-22T07:13:56.190144+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:13:56.190144+0000","last_undegraded":"2024-04-22T07:13:56.190144+0000","last_fullsized":"2024-04-22T07:13:56.190144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189378+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:13:56.189378+0000","last_peered":"2024-04-22T07:13:56.189378+0000","last_clean":"2024-04-22T07:13:56.189378+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:13:56.189378+0000","last_undegraded":"2024-04-22T07:13:56.189378+0000","last_fullsized":"2024-04-22T07:13:56.189378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185895+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:13:56.185895+0000","last_peered":"2024-04-22T07:13:56.185895+0000","last_clean":"2024-04-22T07:13:56.185895+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:13:56.185895+0000","last_undegraded":"2024-04-22T07:13:56.185895+0000","last_fullsized":"2024-04-22T07:13:56.185895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185400+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:13:56.185400+0000","last_peered":"2024-04-22T07:13:56.185400+0000","last_clean":"2024-04-22T07:13:56.185400+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:13:56.185400+0000","last_undegraded":"2024-04-22T07:13:56.185400+0000","last_fullsized":"2024-04-22T07:13:56.185400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.186182+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:13:56.186182+0000","last_peered":"2024-04-22T07:13:56.186182+0000","last_clean":"2024-04-22T07:13:56.186182+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:13:56.186182+0000","last_undegraded":"2024-04-22T07:13:56.186182+0000","last_fullsized":"2024-04-22T07:13:56.186182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187814+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:13:56.187814+0000","last_peered":"2024-04-22T07:13:56.187814+0000","last_clean":"2024-04-22T07:13:56.187814+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:13:56.187814+0000","last_undegraded":"2024-04-22T07:13:56.187814+0000","last_fullsized":"2024-04-22T07:13:56.187814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.428324+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:13:56.428324+0000","last_peered":"2024-04-22T07:13:56.428324+0000","last_clean":"2024-04-22T07:13:56.428324+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:13:56.428324+0000","last_undegraded":"2024-04-22T07:13:56.428324+0000","last_fullsized":"2024-04-22T07:13:56.428324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.194115+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:13:56.194115+0000","last_peered":"2024-04-22T07:13:56.194115+0000","last_clean":"2024-04-22T07:13:56.194115+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:13:56.194115+0000","last_undegraded":"2024-04-22T07:13:56.194115+0000","last_fullsized":"2024-04-22T07:13:56.194115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:14:06.447643+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:06.447643+0000","last_peered":"2024-04-22T07:14:06.447643+0000","last_clean":"2024-04-22T07:14:06.447643+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:06.447643+0000","last_undegraded":"2024-04-22T07:14:06.447643+0000","last_fullsized":"2024-04-22T07:14:06.447643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188073+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:13:56.188073+0000","last_peered":"2024-04-22T07:13:56.188073+0000","last_clean":"2024-04-22T07:13:56.188073+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:13:56.188073+0000","last_undegraded":"2024-04-22T07:13:56.188073+0000","last_fullsized":"2024-04-22T07:13:56.188073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427498+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:13:56.427498+0000","last_peered":"2024-04-22T07:13:56.427498+0000","last_clean":"2024-04-22T07:13:56.427498+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:13:56.427498+0000","last_undegraded":"2024-04-22T07:13:56.427498+0000","last_fullsized":"2024-04-22T07:13:56.427498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187565+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:13:56.187565+0000","last_peered":"2024-04-22T07:13:56.187565+0000","last_clean":"2024-04-22T07:13:56.187565+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:13:56.187565+0000","last_undegraded":"2024-04-22T07:13:56.187565+0000","last_fullsized":"2024-04-22T07:13:56.187565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426618+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:13:56.426618+0000","last_peered":"2024-04-22T07:13:56.426618+0000","last_clean":"2024-04-22T07:13:56.426618+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:13:56.426618+0000","last_undegraded":"2024-04-22T07:13:56.426618+0000","last_fullsized":"2024-04-22T07:13:56.426618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187708+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:13:56.187708+0000","last_peered":"2024-04-22T07:13:56.187708+0000","last_clean":"2024-04-22T07:13:56.187708+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:13:56.187708+0000","last_undegraded":"2024-04-22T07:13:56.187708+0000","last_fullsized":"2024-04-22T07:13:56.187708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188179+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:13:56.188179+0000","last_peered":"2024-04-22T07:13:56.188179+0000","last_clean":"2024-04-22T07:13:56.188179+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:13:56.188179+0000","last_undegraded":"2024-04-22T07:13:56.188179+0000","last_fullsized":"2024-04-22T07:13:56.188179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426788+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:13:56.426788+0000","last_peered":"2024-04-22T07:13:56.426788+0000","last_clean":"2024-04-22T07:13:56.426788+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:13:56.426788+0000","last_undegraded":"2024-04-22T07:13:56.426788+0000","last_fullsized":"2024-04-22T07:13:56.426788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185360+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:13:56.185360+0000","last_peered":"2024-04-22T07:13:56.185360+0000","last_clean":"2024-04-22T07:13:56.185360+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:13:56.185360+0000","last_undegraded":"2024-04-22T07:13:56.185360+0000","last_fullsized":"2024-04-22T07:13:56.185360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187867+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:13:56.187867+0000","last_peered":"2024-04-22T07:13:56.187867+0000","last_clean":"2024-04-22T07:13:56.187867+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:13:56.187867+0000","last_undegraded":"2024-04-22T07:13:56.187867+0000","last_fullsized":"2024-04-22T07:13:56.187867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427108+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:13:56.427108+0000","last_peered":"2024-04-22T07:13:56.427108+0000","last_clean":"2024-04-22T07:13:56.427108+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:13:56.427108+0000","last_undegraded":"2024-04-22T07:13:56.427108+0000","last_fullsized":"2024-04-22T07:13:56.427108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427048+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:13:56.427048+0000","last_peered":"2024-04-22T07:13:56.427048+0000","last_clean":"2024-04-22T07:13:56.427048+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:13:56.427048+0000","last_undegraded":"2024-04-22T07:13:56.427048+0000","last_fullsized":"2024-04-22T07:13:56.427048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188070+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:13:56.188070+0000","last_peered":"2024-04-22T07:13:56.188070+0000","last_clean":"2024-04-22T07:13:56.188070+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:13:56.188070+0000","last_undegraded":"2024-04-22T07:13:56.188070+0000","last_fullsized":"2024-04-22T07:13:56.188070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187785+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:13:56.187785+0000","last_peered":"2024-04-22T07:13:56.187785+0000","last_clean":"2024-04-22T07:13:56.187785+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:13:56.187785+0000","last_undegraded":"2024-04-22T07:13:56.187785+0000","last_fullsized":"2024-04-22T07:13:56.187785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187805+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:13:56.187805+0000","last_peered":"2024-04-22T07:13:56.187805+0000","last_clean":"2024-04-22T07:13:56.187805+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:13:56.187805+0000","last_undegraded":"2024-04-22T07:13:56.187805+0000","last_fullsized":"2024-04-22T07:13:56.187805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185667+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:13:56.185667+0000","last_peered":"2024-04-22T07:13:56.185667+0000","last_clean":"2024-04-22T07:13:56.185667+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:13:56.185667+0000","last_undegraded":"2024-04-22T07:13:56.185667+0000","last_fullsized":"2024-04-22T07:13:56.185667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189265+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:13:56.189265+0000","last_peered":"2024-04-22T07:13:56.189265+0000","last_clean":"2024-04-22T07:13:56.189265+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:13:56.189265+0000","last_undegraded":"2024-04-22T07:13:56.189265+0000","last_fullsized":"2024-04-22T07:13:56.189265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187980+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:13:56.187980+0000","last_peered":"2024-04-22T07:13:56.187980+0000","last_clean":"2024-04-22T07:13:56.187980+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:13:56.187980+0000","last_undegraded":"2024-04-22T07:13:56.187980+0000","last_fullsized":"2024-04-22T07:13:56.187980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.431722+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:13:56.431722+0000","last_peered":"2024-04-22T07:13:56.431722+0000","last_clean":"2024-04-22T07:13:56.431722+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:13:56.431722+0000","last_undegraded":"2024-04-22T07:13:56.431722+0000","last_fullsized":"2024-04-22T07:13:56.431722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187874+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:13:56.187874+0000","last_peered":"2024-04-22T07:13:56.187874+0000","last_clean":"2024-04-22T07:13:56.187874+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:13:56.187874+0000","last_undegraded":"2024-04-22T07:13:56.187874+0000","last_fullsized":"2024-04-22T07:13:56.187874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189645+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:13:56.189645+0000","last_peered":"2024-04-22T07:13:56.189645+0000","last_clean":"2024-04-22T07:13:56.189645+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:13:56.189645+0000","last_undegraded":"2024-04-22T07:13:56.189645+0000","last_fullsized":"2024-04-22T07:13:56.189645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188304+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:13:56.188304+0000","last_peered":"2024-04-22T07:13:56.188304+0000","last_clean":"2024-04-22T07:13:56.188304+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:13:56.188304+0000","last_undegraded":"2024-04-22T07:13:56.188304+0000","last_fullsized":"2024-04-22T07:13:56.188304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190614+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:13:56.190614+0000","last_peered":"2024-04-22T07:13:56.190614+0000","last_clean":"2024-04-22T07:13:56.190614+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:13:56.190614+0000","last_undegraded":"2024-04-22T07:13:56.190614+0000","last_fullsized":"2024-04-22T07:13:56.190614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187956+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:13:56.187956+0000","last_peered":"2024-04-22T07:13:56.187956+0000","last_clean":"2024-04-22T07:13:56.187956+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:13:56.187956+0000","last_undegraded":"2024-04-22T07:13:56.187956+0000","last_fullsized":"2024-04-22T07:13:56.187956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.427039+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:13:56.427039+0000","last_peered":"2024-04-22T07:13:56.427039+0000","last_clean":"2024-04-22T07:13:56.427039+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:13:56.427039+0000","last_undegraded":"2024-04-22T07:13:56.427039+0000","last_fullsized":"2024-04-22T07:13:56.427039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188140+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:13:56.188140+0000","last_peered":"2024-04-22T07:13:56.188140+0000","last_clean":"2024-04-22T07:13:56.188140+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:13:56.188140+0000","last_undegraded":"2024-04-22T07:13:56.188140+0000","last_fullsized":"2024-04-22T07:13:56.188140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185913+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:13:56.185913+0000","last_peered":"2024-04-22T07:13:56.185913+0000","last_clean":"2024-04-22T07:13:56.185913+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:13:56.185913+0000","last_undegraded":"2024-04-22T07:13:56.185913+0000","last_fullsized":"2024-04-22T07:13:56.185913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.428079+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:13:56.428079+0000","last_peered":"2024-04-22T07:13:56.428079+0000","last_clean":"2024-04-22T07:13:56.428079+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:13:56.428079+0000","last_undegraded":"2024-04-22T07:13:56.428079+0000","last_fullsized":"2024-04-22T07:13:56.428079+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.189147+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:13:56.189147+0000","last_peered":"2024-04-22T07:13:56.189147+0000","last_clean":"2024-04-22T07:13:56.189147+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:13:56.189147+0000","last_undegraded":"2024-04-22T07:13:56.189147+0000","last_fullsized":"2024-04-22T07:13:56.189147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426791+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:13:56.426791+0000","last_peered":"2024-04-22T07:13:56.426791+0000","last_clean":"2024-04-22T07:13:56.426791+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:13:56.426791+0000","last_undegraded":"2024-04-22T07:13:56.426791+0000","last_fullsized":"2024-04-22T07:13:56.426791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426686+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:13:56.426686+0000","last_peered":"2024-04-22T07:13:56.426686+0000","last_clean":"2024-04-22T07:13:56.426686+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:13:56.426686+0000","last_undegraded":"2024-04-22T07:13:56.426686+0000","last_fullsized":"2024-04-22T07:13:56.426686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185881+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:13:56.185881+0000","last_peered":"2024-04-22T07:13:56.185881+0000","last_clean":"2024-04-22T07:13:56.185881+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:13:56.185881+0000","last_undegraded":"2024-04-22T07:13:56.185881+0000","last_fullsized":"2024-04-22T07:13:56.185881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190820+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:13:56.190820+0000","last_peered":"2024-04-22T07:13:56.190820+0000","last_clean":"2024-04-22T07:13:56.190820+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:13:56.190820+0000","last_undegraded":"2024-04-22T07:13:56.190820+0000","last_fullsized":"2024-04-22T07:13:56.190820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.428831+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:13:56.428831+0000","last_peered":"2024-04-22T07:13:56.428831+0000","last_clean":"2024-04-22T07:13:56.428831+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:13:56.428831+0000","last_undegraded":"2024-04-22T07:13:56.428831+0000","last_fullsized":"2024-04-22T07:13:56.428831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.190513+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:13:56.190513+0000","last_peered":"2024-04-22T07:13:56.190513+0000","last_clean":"2024-04-22T07:13:56.190513+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:13:56.190513+0000","last_undegraded":"2024-04-22T07:13:56.190513+0000","last_fullsized":"2024-04-22T07:13:56.190513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.187752+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:13:56.187752+0000","last_peered":"2024-04-22T07:13:56.187752+0000","last_clean":"2024-04-22T07:13:56.187752+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:13:56.187752+0000","last_undegraded":"2024-04-22T07:13:56.187752+0000","last_fullsized":"2024-04-22T07:13:56.187752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.188539+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:13:56.188539+0000","last_peered":"2024-04-22T07:13:56.188539+0000","last_clean":"2024-04-22T07:13:56.188539+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:13:56.188539+0000","last_undegraded":"2024-04-22T07:13:56.188539+0000","last_fullsized":"2024-04-22T07:13:56.188539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.426810+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:13:56.426810+0000","last_peered":"2024-04-22T07:13:56.426810+0000","last_clean":"2024-04-22T07:13:56.426810+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:13:56.426810+0000","last_undegraded":"2024-04-22T07:13:56.426810+0000","last_fullsized":"2024-04-22T07:13:56.426810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-22T07:13:56.185785+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:13:56.185785+0000","last_peered":"2024-04-22T07:13:56.185785+0000","last_clean":"2024-04-22T07:13:56.185785+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:13:56.185785+0000","last_undegraded":"2024-04-22T07:13:56.185785+0000","last_fullsized":"2024-04-22T07:13:56.185785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64656,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":450,"num_read_kb":293,"num_write":155,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":461597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1114090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":970752,"data_stored":461799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":461137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1564672,"data_stored":1053258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":461188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":522609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":462459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":523343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":461625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1052942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:14:11.839 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-22T07:14:11.839 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T07:14:11.839 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-22T07:14:11.839 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-22T07:14:12.074 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:12.420 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:12 smithi113 ceph-mon[27933]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T07:14:12.420 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:12 smithi113 ceph-mon[27933]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:12 smithi133 ceph-mon[31198]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T07:14:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:12 smithi133 ceph-mon[31198]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:12.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:12 smithi154 ceph-mon[31862]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-22T07:14:12.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:12 smithi154 ceph-mon[31862]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 -- 172.21.15.113:0/239143034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 msgr2=0x7f5ef010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 --2- 172.21.15.113:0/239143034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef010e7f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009990 tx=0x7f5ee0009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 -- 172.21.15.113:0/239143034 shutdown_connections 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 --2- 172.21.15.113:0/239143034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef010e7f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 --2- 172.21.15.113:0/239143034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef0107990 0x7f5ef0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 --2- 172.21.15.113:0/239143034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef0101940 0x7f5ef0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 -- 172.21.15.113:0/239143034 >> 172.21.15.113:0/239143034 conn(0x7f5ef00788f0 msgr2=0x7f5ef0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 -- 172.21.15.113:0/239143034 shutdown_connections 2024-04-22T07:14:13.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.570+0000 7f5ef857d700 1 -- 172.21.15.113:0/239143034 wait complete. 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 Processor -- start 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 -- start start 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef0101940 0x7f5ef01034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef0107990 0x7f5ef0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111c90 con 0x7f5ef0102300 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111b10 con 0x7f5ef0107990 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef857d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111e10 con 0x7f5ef0101940 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef5b18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef5b18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef0103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35780/0 (socket says 172.21.15.113:35780) 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.571+0000 7f5ef5b18700 1 -- 172.21.15.113:0/240095408 learned_addr learned my addr 172.21.15.113:0/240095408 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef6319700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef0101940 0x7f5ef01034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef5b18700 1 -- 172.21.15.113:0/240095408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef0101940 msgr2=0x7f5ef01034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef5b18700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef0101940 0x7f5ef01034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef5b18700 1 -- 172.21.15.113:0/240095408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef0107990 msgr2=0x7f5ef0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef5b18700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef0107990 0x7f5ef0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef5b18700 1 -- 172.21.15.113:0/240095408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef01a77a0 con 0x7f5ef0102300 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef5b18700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef0103a30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b530 tx=0x7f5ee400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4015020 con 0x7f5ef0102300 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef01a7a90 con 0x7f5ef0102300 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5ee4011d00 con 0x7f5ef0102300 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4010e60 con 0x7f5ef0102300 2024-04-22T07:14:13.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.572+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a8020 con 0x7f5ef0102300 2024-04-22T07:14:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.573+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0069730 con 0x7f5ef0102300 2024-04-22T07:14:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.574+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5ee4018020 con 0x7f5ef0102300 2024-04-22T07:14:13.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.575+0000 7f5eef7fe700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5ed80a3cd0 0x7f5ed80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:13.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.575+0000 7f5ef6319700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5ed80a3cd0 0x7f5ed80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:13.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.575+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5ee40d0180 con 0x7f5ef0102300 2024-04-22T07:14:13.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.575+0000 7f5ef6319700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5ed80a3cd0 0x7f5ed80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80060b0 tx=0x7f5ee8008640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:13.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.578+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ee401f020 con 0x7f5ef0102300 2024-04-22T07:14:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.820+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5ef0104a20 con 0x7f5ef0102300 2024-04-22T07:14:13.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.820+0000 7f5eef7fe700 1 -- 172.21.15.113:0/240095408 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5ee409cb20 con 0x7f5ef0102300 2024-04-22T07:14:13.818 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:13.818 INFO:teuthology.orchestra.run.smithi113.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T07:14:13.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5ed80a3cd0 msgr2=0x7f5ed80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:13.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5ed80a3cd0 0x7f5ed80a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80060b0 tx=0x7f5ee8008640 comp rx=0 tx=0).stop 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 msgr2=0x7f5ef0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef0103a30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400b530 tx=0x7f5ee400b8f0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 shutdown_connections 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5ed80a3cd0 0x7f5ed80a6190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef0107990 0x7f5ef0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ef0101940 0x7f5ef01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 --2- 172.21.15.113:0/240095408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef0102300 0x7f5ef0103a30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 >> 172.21.15.113:0/240095408 conn(0x7f5ef00788f0 msgr2=0x7f5ef006aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 shutdown_connections 2024-04-22T07:14:13.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:13.823+0000 7f5ef857d700 1 -- 172.21.15.113:0/240095408 wait complete. 2024-04-22T07:14:14.431 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:14 smithi113 ceph-mon[27933]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:14.431 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:14 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/240095408' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T07:14:14.458 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-22T07:14:14.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:14.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 -- 172.21.15.113:0/1889884666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160101fc0 msgr2=0x7f7160102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 --2- 172.21.15.113:0/1889884666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160101fc0 0x7f7160102440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7154009900 tx=0x7f715402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 -- 172.21.15.113:0/1889884666 shutdown_connections 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 --2- 172.21.15.113:0/1889884666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160102980 0x7f71601092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 --2- 172.21.15.113:0/1889884666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160101fc0 0x7f7160102440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 --2- 172.21.15.113:0/1889884666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7160100e70 0x7f7160101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 -- 172.21.15.113:0/1889884666 >> 172.21.15.113:0/1889884666 conn(0x7f71600fc3e0 msgr2=0x7f71600fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 -- 172.21.15.113:0/1889884666 shutdown_connections 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.561+0000 7f7164893640 1 -- 172.21.15.113:0/1889884666 wait complete. 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 Processor -- start 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 -- start start 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 0x7f7160069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160101fc0 0x7f716019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7160102980 0x7f716019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f716010cb10 con 0x7f7160100e70 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f716010c990 con 0x7f7160101fc0 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f7164893640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f716010cc90 con 0x7f7160102980 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f715e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 0x7f7160069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f715e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 0x7f7160069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35794/0 (socket says 172.21.15.113:35794) 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.562+0000 7f715e59c640 1 -- 172.21.15.113:0/1449863926 learned_addr learned my addr 172.21.15.113:0/1449863926 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715ed9d640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7160102980 0x7f716019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715dd9b640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160101fc0 0x7f716019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715e59c640 1 -- 172.21.15.113:0/1449863926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7160102980 msgr2=0x7f716019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715e59c640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7160102980 0x7f716019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715e59c640 1 -- 172.21.15.113:0/1449863926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160101fc0 msgr2=0x7f716019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715e59c640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160101fc0 0x7f716019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715e59c640 1 -- 172.21.15.113:0/1449863926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716019ff30 con 0x7f7160100e70 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715dd9b640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160101fc0 0x7f716019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f715e59c640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 0x7f7160069c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f715000d770 tx=0x7f715000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150017070 con 0x7f7160100e70 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f71500054f0 con 0x7f7160100e70 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150004de0 con 0x7f7160100e70 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601a01c0 con 0x7f7160100e70 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.563+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601a0710 con 0x7f7160100e70 2024-04-22T07:14:14.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.564+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7160108180 con 0x7f7160100e70 2024-04-22T07:14:14.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:14 smithi133 ceph-mon[31198]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:14.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:14 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/240095408' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T07:14:14.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.565+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7150004f80 con 0x7f7160100e70 2024-04-22T07:14:14.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.566+0000 7f714f7fe640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f71480a3cc0 0x7f71480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.566+0000 7f715dd9b640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f71480a3cc0 0x7f71480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.566+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f71500d04d0 con 0x7f7160100e70 2024-04-22T07:14:14.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.566+0000 7f715dd9b640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f71480a3cc0 0x7f71480a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7154036040 tx=0x7f7154000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:14.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:14 smithi154 ceph-mon[31862]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:14.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:14 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/240095408' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T07:14:14.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.569+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f715009cde0 con 0x7f7160100e70 2024-04-22T07:14:14.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.746+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f716002d790 con 0x7f7160100e70 2024-04-22T07:14:14.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.747+0000 7f714f7fe640 1 -- 172.21.15.113:0/1449863926 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22700 (secure 0 0 0) 0x7f715009c6e0 con 0x7f7160100e70 2024-04-22T07:14:14.745 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:14.745 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:13:56.180466+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:14.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f71480a3cc0 msgr2=0x7f71480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f71480a3cc0 0x7f71480a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7154036040 tx=0x7f7154000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 msgr2=0x7f7160069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 0x7f7160069c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f715000d770 tx=0x7f715000dc30 comp rx=0 tx=0).stop 2024-04-22T07:14:14.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 shutdown_connections 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f71480a3cc0 0x7f71480a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7160102980 0x7f716019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7160101fc0 0x7f716019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 --2- 172.21.15.113:0/1449863926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7160100e70 0x7f7160069c50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 >> 172.21.15.113:0/1449863926 conn(0x7f71600fc3e0 msgr2=0x7f7160105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 shutdown_connections 2024-04-22T07:14:14.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.749+0000 7f7164893640 1 -- 172.21.15.113:0/1449863926 wait complete. 2024-04-22T07:14:14.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:14.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:14.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 -- 172.21.15.113:0/803151344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064101fa0 msgr2=0x7f70641023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 --2- 172.21.15.113:0/803151344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064101fa0 0x7f70641023c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7054009900 tx=0x7f705402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 -- 172.21.15.113:0/803151344 shutdown_connections 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 --2- 172.21.15.113:0/803151344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064103b50 0x7f706410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 --2- 172.21.15.113:0/803151344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7064103190 0x7f7064103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 --2- 172.21.15.113:0/803151344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064101fa0 0x7f70641023c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 -- 172.21.15.113:0/803151344 >> 172.21.15.113:0/803151344 conn(0x7f70640fd530 msgr2=0x7f70640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 -- 172.21.15.113:0/803151344 shutdown_connections 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 -- 172.21.15.113:0/803151344 wait complete. 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.862+0000 7f7069806640 1 Processor -- start 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 -- start start 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7064101fa0 0x7f7064075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 0x7f7064074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064103b50 0x7f7064074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f706410dbe0 con 0x7f7064103190 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f706410da60 con 0x7f7064103b50 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f706410dd60 con 0x7f7064101fa0 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 0x7f7064074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 0x7f7064074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35820/0 (socket says 172.21.15.113:35820) 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 -- 172.21.15.113:0/1166692057 learned_addr learned my addr 172.21.15.113:0/1166692057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70637fe640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064103b50 0x7f7064074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7062ffd640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7064101fa0 0x7f7064075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 -- 172.21.15.113:0/1166692057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7064101fa0 msgr2=0x7f7064075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7064101fa0 0x7f7064075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 -- 172.21.15.113:0/1166692057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064103b50 msgr2=0x7f7064074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:14.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064103b50 0x7f7064074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 -- 172.21.15.113:0/1166692057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7064075110 con 0x7f7064103190 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f70627fc640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 0x7f7064074340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f705800d950 tx=0x7f705800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058017070 con 0x7f7064103190 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7058009c30 con 0x7f7064103190 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641a23d0 con 0x7f7064103190 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.863+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705800f6b0 con 0x7f7064103190 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.864+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641a2920 con 0x7f7064103190 2024-04-22T07:14:14.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.865+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70641092c0 con 0x7f7064103190 2024-04-22T07:14:14.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.866+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f70580109c0 con 0x7f7064103190 2024-04-22T07:14:14.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.866+0000 7f7068804640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f704c0a3c40 0x7f704c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:14.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.866+0000 7f7062ffd640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f704c0a3c40 0x7f704c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:14.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.866+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f70580d0100 con 0x7f7064103190 2024-04-22T07:14:14.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.867+0000 7f7062ffd640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f704c0a3c40 0x7f704c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7054009230 tx=0x7f7054005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:14.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:14.871+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7058009de0 con 0x7f7064103190 2024-04-22T07:14:15.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.045+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f706402d810 con 0x7f7064103190 2024-04-22T07:14:15.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.045+0000 7f7068804640 1 -- 172.21.15.113:0/1166692057 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f705809c340 con 0x7f7064103190 2024-04-22T07:14:15.043 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:15.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f704c0a3c40 msgr2=0x7f704c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f704c0a3c40 0x7f704c0a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7054009230 tx=0x7f7054005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 msgr2=0x7f7064074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 0x7f7064074340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f705800d950 tx=0x7f705800de10 comp rx=0 tx=0).stop 2024-04-22T07:14:15.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 shutdown_connections 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f704c0a3c40 0x7f704c0a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064103b50 0x7f7064074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7064103190 0x7f7064074340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 --2- 172.21.15.113:0/1166692057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7064101fa0 0x7f7064075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 >> 172.21.15.113:0/1166692057 conn(0x7f70640fd530 msgr2=0x7f7064106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 shutdown_connections 2024-04-22T07:14:15.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.047+0000 7f7069806640 1 -- 172.21.15.113:0/1166692057 wait complete. 2024-04-22T07:14:15.056 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 -- 172.21.15.113:0/1740783776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8101980 msgr2=0x7faaa8106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 --2- 172.21.15.113:0/1740783776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8101980 0x7faaa8106220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009ca0 tx=0x7faaa002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 -- 172.21.15.113:0/1740783776 shutdown_connections 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 --2- 172.21.15.113:0/1740783776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8101980 0x7faaa8106220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 --2- 172.21.15.113:0/1740783776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaa8100fc0 0x7faaa8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 --2- 172.21.15.113:0/1740783776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaa8100260 0x7faaa8100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.161+0000 7faaafe55640 1 -- 172.21.15.113:0/1740783776 >> 172.21.15.113:0/1740783776 conn(0x7faaa80fbb90 msgr2=0x7faaa80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 -- 172.21.15.113:0/1740783776 shutdown_connections 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 -- 172.21.15.113:0/1740783776 wait complete. 2024-04-22T07:14:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 Processor -- start 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 -- start start 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 0x7faaa819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaa8100fc0 0x7faaa819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaa8101980 0x7faaa81a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8114240 con 0x7faaa8100260 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81140c0 con 0x7faaa8100fc0 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaafe55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81143c0 con 0x7faaa8101980 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaadbf1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 0x7faaa819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaadbf1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 0x7faaa819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35830/0 (socket says 172.21.15.113:35830) 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaadbf1640 1 -- 172.21.15.113:0/3489033876 learned_addr learned my addr 172.21.15.113:0/3489033876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaad3f0640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaa8100fc0 0x7faaa819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.162+0000 7faaae3f2640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaa8101980 0x7faaa81a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaadbf1640 1 -- 172.21.15.113:0/3489033876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaa8101980 msgr2=0x7faaa81a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaadbf1640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaa8101980 0x7faaa81a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaadbf1640 1 -- 172.21.15.113:0/3489033876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaa8100fc0 msgr2=0x7faaa819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaadbf1640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaa8100fc0 0x7faaa819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaadbf1640 1 -- 172.21.15.113:0/3489033876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa81a64a0 con 0x7faaa8100260 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaadbf1640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 0x7faaa819e920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e9f0 tx=0x7faaa400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa400cd80 con 0x7faaa8100260 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa81a6790 con 0x7faaa8100260 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faaa40054f0 con 0x7faaa8100260 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4010690 con 0x7faaa8100260 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.163+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa81a6ce0 con 0x7faaa8100260 2024-04-22T07:14:15.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.164+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa8105030 con 0x7faaa8100260 2024-04-22T07:14:15.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.165+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7faaa40046c0 con 0x7faaa8100260 2024-04-22T07:14:15.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.166+0000 7faa9effd640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7faa940a3c40 0x7faa940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.166+0000 7faaad3f0640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7faa940a3c40 0x7faa940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.166+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faaa40d08f0 con 0x7faaa8100260 2024-04-22T07:14:15.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.167+0000 7faaad3f0640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7faa940a3c40 0x7faa940a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faaa819fd90 tx=0x7faa98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:15.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.171+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faaa409d180 con 0x7faaa8100260 2024-04-22T07:14:15.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:15 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1449863926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:15.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:15 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1166692057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:15.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.349+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faaa8062b50 con 0x7faaa8100260 2024-04-22T07:14:15.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.350+0000 7faa9effd640 1 -- 172.21.15.113:0/3489033876 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7faaa40bd0c0 con 0x7faaa8100260 2024-04-22T07:14:15.347 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:15.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7faa940a3c40 msgr2=0x7faa940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7faa940a3c40 0x7faa940a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faaa819fd90 tx=0x7faa98005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 msgr2=0x7faaa819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 0x7faaa819e920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e9f0 tx=0x7faaa400edf0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 shutdown_connections 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7faa940a3c40 0x7faa940a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faaa8101980 0x7faaa81a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaa8100fc0 0x7faaa819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 --2- 172.21.15.113:0/3489033876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa8100260 0x7faaa819e920 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 >> 172.21.15.113:0/3489033876 conn(0x7faaa80fbb90 msgr2=0x7faaa8102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 shutdown_connections 2024-04-22T07:14:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.352+0000 7faaafe55640 1 -- 172.21.15.113:0/3489033876 wait complete. 2024-04-22T07:14:15.361 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:15.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 -- 172.21.15.113:0/590675225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf08102a90 msgr2=0x7fdf08104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/590675225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf08102a90 0x7fdf08104e80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009900 tx=0x7fdf0402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:15.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 -- 172.21.15.113:0/590675225 shutdown_connections 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/590675225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 0x7fdf0810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/590675225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf081053c0 0x7fdf081077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/590675225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf08102a90 0x7fdf08104e80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 -- 172.21.15.113:0/590675225 >> 172.21.15.113:0/590675225 conn(0x7fdf080fc440 msgr2=0x7fdf080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 -- 172.21.15.113:0/590675225 shutdown_connections 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.463+0000 7fdf0fac7640 1 -- 172.21.15.113:0/590675225 wait complete. 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 Processor -- start 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 -- start start 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf08102a90 0x7fdf08198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 0x7fdf08198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 0x7fdf0819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810dc30 con 0x7fdf081053c0 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810dab0 con 0x7fdf08107cf0 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0fac7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0810ddb0 con 0x7fdf08102a90 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 0x7fdf08198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 0x7fdf08198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35848/0 (socket says 172.21.15.113:35848) 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 -- 172.21.15.113:0/2502069315 learned_addr learned my addr 172.21.15.113:0/2502069315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d863640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf08102a90 0x7fdf08198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0e064640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 0x7fdf0819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 -- 172.21.15.113:0/2502069315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf08102a90 msgr2=0x7fdf08198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf08102a90 0x7fdf08198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 -- 172.21.15.113:0/2502069315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 msgr2=0x7fdf0819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 0x7fdf0819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 -- 172.21.15.113:0/2502069315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0819ff50 con 0x7fdf081053c0 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0e064640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 0x7fdf0819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.464+0000 7fdf0d062640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 0x7fdf08198870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdef800e9f0 tx=0x7fdef800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.465+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef800ccc0 con 0x7fdf081053c0 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.465+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081a0240 con 0x7fdf081053c0 2024-04-22T07:14:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.465+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdef80054f0 con 0x7fdf081053c0 2024-04-22T07:14:15.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.465+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8010690 con 0x7fdf081053c0 2024-04-22T07:14:15.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.465+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a0740 con 0x7fdf081053c0 2024-04-22T07:14:15.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.465+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf080fdf30 con 0x7fdf081053c0 2024-04-22T07:14:15.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.467+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdef80046c0 con 0x7fdf081053c0 2024-04-22T07:14:15.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.467+0000 7fdefeffd640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdef40a3c40 0x7fdef40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.467+0000 7fdf0d863640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdef40a3c40 0x7fdef40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.467+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdef80d0c00 con 0x7fdf081053c0 2024-04-22T07:14:15.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.468+0000 7fdf0d863640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdef40a3c40 0x7fdef40a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006010 tx=0x7fdf04005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:15.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.471+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdef8053ff0 con 0x7fdf081053c0 2024-04-22T07:14:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:15 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1449863926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:15 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1166692057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:15.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:15 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1449863926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:15.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:15 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1166692057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:15.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.646+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf0802d810 con 0x7fdf081053c0 2024-04-22T07:14:15.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.647+0000 7fdefeffd640 1 -- 172.21.15.113:0/2502069315 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fdef809d490 con 0x7fdf081053c0 2024-04-22T07:14:15.645 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdef40a3c40 msgr2=0x7fdef40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdef40a3c40 0x7fdef40a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006010 tx=0x7fdf04005b50 comp rx=0 tx=0).stop 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 msgr2=0x7fdf08198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 0x7fdf08198870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdef800e9f0 tx=0x7fdef800eeb0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 shutdown_connections 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fdef40a3c40 0x7fdef40a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf08107cf0 0x7fdf0819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf081053c0 0x7fdf08198870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 --2- 172.21.15.113:0/2502069315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf08102a90 0x7fdf08198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 >> 172.21.15.113:0/2502069315 conn(0x7fdf080fc440 msgr2=0x7fdf08103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 shutdown_connections 2024-04-22T07:14:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.649+0000 7fdf0fac7640 1 -- 172.21.15.113:0/2502069315 wait complete. 2024-04-22T07:14:15.657 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 -- 172.21.15.113:0/1426313937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 msgr2=0x7f2e40102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 --2- 172.21.15.113:0/1426313937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e40102450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009900 tx=0x7f2e3002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 -- 172.21.15.113:0/1426313937 shutdown_connections 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 --2- 172.21.15.113:0/1426313937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 0x7f2e401092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 --2- 172.21.15.113:0/1426313937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e40102450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 --2- 172.21.15.113:0/1426313937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 0x7f2e401012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 -- 172.21.15.113:0/1426313937 >> 172.21.15.113:0/1426313937 conn(0x7f2e400fc410 msgr2=0x7f2e400fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 -- 172.21.15.113:0/1426313937 shutdown_connections 2024-04-22T07:14:15.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.757+0000 7f2e47311640 1 -- 172.21.15.113:0/1426313937 wait complete. 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 Processor -- start 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 -- start start 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 0x7f2e4006b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e4019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 0x7f2e4019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4010cb10 con 0x7f2e40101fd0 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4010c990 con 0x7f2e40100e80 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e47311640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4010cc90 con 0x7f2e40102990 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e4019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e4019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46210/0 (socket says 172.21.15.113:46210) 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 -- 172.21.15.113:0/2496548539 learned_addr learned my addr 172.21.15.113:0/2496548539 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e458ae640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 0x7f2e4019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e450ad640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 0x7f2e4006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 -- 172.21.15.113:0/2496548539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 msgr2=0x7f2e4019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 0x7f2e4019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 -- 172.21.15.113:0/2496548539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 msgr2=0x7f2e4006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 0x7f2e4006b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 -- 172.21.15.113:0/2496548539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019fe80 con 0x7f2e40101fd0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e458ae640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 0x7f2e4019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e450ad640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 0x7f2e4006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.758+0000 7f2e448ac640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e4019cf80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e30036040 tx=0x7f2e30005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.759+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30041070 con 0x7f2e40101fd0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.759+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2e300044c0 con 0x7f2e40101fd0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.759+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e401a0110 con 0x7f2e40101fd0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.759+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30034440 con 0x7f2e40101fd0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.759+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401a0660 con 0x7f2e40101fd0 2024-04-22T07:14:15.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.760+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e40108190 con 0x7f2e40101fd0 2024-04-22T07:14:15.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.761+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2e30044070 con 0x7f2e40101fd0 2024-04-22T07:14:15.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.761+0000 7f2e367fc640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2e2c0a3c40 0x7f2e2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:15.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.761+0000 7f2e450ad640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2e2c0a3c40 0x7f2e2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:15.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.761+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2e300f21a0 con 0x7f2e40101fd0 2024-04-22T07:14:15.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.762+0000 7f2e450ad640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2e2c0a3c40 0x7f2e2c0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009b40 tx=0x7f2e3c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:15.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.766+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e300beb40 con 0x7f2e40101fd0 2024-04-22T07:14:15.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.980+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e4002d5c0 con 0x7f2e40101fd0 2024-04-22T07:14:15.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.981+0000 7f2e367fc640 1 -- 172.21.15.113:0/2496548539 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11875 (secure 0 0 0) 0x7f2e300be440 con 0x7f2e40101fd0 2024-04-22T07:14:15.978 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:15.978 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:12:58.522890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24473,24506]},"id":1}]} 2024-04-22T07:14:15.979 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:14:15.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.982+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2e2c0a3c40 msgr2=0x7f2e2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.982+0000 7f2e47311640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2e2c0a3c40 0x7f2e2c0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009b40 tx=0x7f2e3c005e50 comp rx=0 tx=0).stop 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.982+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 msgr2=0x7f2e4019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.982+0000 7f2e47311640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e4019cf80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e30036040 tx=0x7f2e30005e60 comp rx=0 tx=0).stop 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 shutdown_connections 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2e2c0a3c40 0x7f2e2c0a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e40102990 0x7f2e4019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e40101fd0 0x7f2e4019cf80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 --2- 172.21.15.113:0/2496548539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e40100e80 0x7f2e4006b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 >> 172.21.15.113:0/2496548539 conn(0x7f2e400fc410 msgr2=0x7f2e40105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 shutdown_connections 2024-04-22T07:14:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:15.983+0000 7f2e47311640 1 -- 172.21.15.113:0/2496548539 wait complete. 2024-04-22T07:14:15.992 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 -- 172.21.15.113:0/4027513303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 msgr2=0x7f7c54102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 --2- 172.21.15.113:0/4027513303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c54102470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009900 tx=0x7f7c4802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 -- 172.21.15.113:0/4027513303 shutdown_connections 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 --2- 172.21.15.113:0/4027513303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c541029b0 0x7f7c541092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 --2- 172.21.15.113:0/4027513303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c54102470 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 --2- 172.21.15.113:0/4027513303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c54100ea0 0x7f7c541012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 -- 172.21.15.113:0/4027513303 >> 172.21.15.113:0/4027513303 conn(0x7f7c540fc410 msgr2=0x7f7c540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 -- 172.21.15.113:0/4027513303 shutdown_connections 2024-04-22T07:14:16.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.091+0000 7f7c59111640 1 -- 172.21.15.113:0/4027513303 wait complete. 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 Processor -- start 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 -- start start 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c54100ea0 0x7f7c54198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c541987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c541029b0 0x7f7c5419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410cb20 con 0x7f7c54101ff0 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410c9a0 con 0x7f7c54100ea0 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c59111640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5410cca0 con 0x7f7c541029b0 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c5259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c541987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c5259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c541987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46242/0 (socket says 172.21.15.113:46242) 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.092+0000 7f7c5259c640 1 -- 172.21.15.113:0/1913050635 learned_addr learned my addr 172.21.15.113:0/1913050635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:16.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5359e640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c541029b0 0x7f7c5419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5259c640 1 -- 172.21.15.113:0/1913050635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c541029b0 msgr2=0x7f7c5419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5259c640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c541029b0 0x7f7c5419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5259c640 1 -- 172.21.15.113:0/1913050635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c54100ea0 msgr2=0x7f7c54198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c52d9d640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c54100ea0 0x7f7c54198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5259c640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c54100ea0 0x7f7c54198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5259c640 1 -- 172.21.15.113:0/1913050635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5419fe90 con 0x7f7c54101ff0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c5259c640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c541987b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c48036040 tx=0x7f7c48005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48041070 con 0x7f7c54101ff0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c541a0120 con 0x7f7c54101ff0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7c480044c0 con 0x7f7c54101ff0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48034440 con 0x7f7c54101ff0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.093+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541a06c0 con 0x7f7c54101ff0 2024-04-22T07:14:16.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.094+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c541a0920 con 0x7f7c54101ff0 2024-04-22T07:14:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.095+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7c48044070 con 0x7f7c54101ff0 2024-04-22T07:14:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.096+0000 7f7c3bfff640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f7c3c0a3c40 0x7f7c3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.096+0000 7f7c52d9d640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f7c3c0a3c40 0x7f7c3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.096+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7c480f25c0 con 0x7f7c54101ff0 2024-04-22T07:14:16.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.096+0000 7f7c52d9d640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f7c3c0a3c40 0x7f7c3c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005c40 tx=0x7f7c44005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.099+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c480bef60 con 0x7f7c54101ff0 2024-04-22T07:14:16.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.282+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c541a0af0 con 0x7f7c54101ff0 2024-04-22T07:14:16.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.283+0000 7f7c3bfff640 1 -- 172.21.15.113:0/1913050635 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22700 (secure 0 0 0) 0x7f7c480be860 con 0x7f7c54101ff0 2024-04-22T07:14:16.281 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:16.281 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:13:56.180466+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:16.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f7c3c0a3c40 msgr2=0x7f7c3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f7c3c0a3c40 0x7f7c3c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005c40 tx=0x7f7c44005bd0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 msgr2=0x7f7c541987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c541987b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c48036040 tx=0x7f7c48005e60 comp rx=0 tx=0).stop 2024-04-22T07:14:16.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 shutdown_connections 2024-04-22T07:14:16.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f7c3c0a3c40 0x7f7c3c0a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c541029b0 0x7f7c5419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c54101ff0 0x7f7c541987b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 --2- 172.21.15.113:0/1913050635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c54100ea0 0x7f7c54198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 >> 172.21.15.113:0/1913050635 conn(0x7f7c540fc410 msgr2=0x7f7c5406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 shutdown_connections 2024-04-22T07:14:16.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.285+0000 7f7c59111640 1 -- 172.21.15.113:0/1913050635 wait complete. 2024-04-22T07:14:16.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:16 smithi113 ceph-mon[27933]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:16.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:16 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3489033876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:16.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:16 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2502069315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:16.293 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:16 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2496548539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:16.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:16.295 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:16.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 -- 172.21.15.113:0/1814350283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe970101810 msgr2=0x7fe970101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 --2- 172.21.15.113:0/1814350283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe970101810 0x7fe970101c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe960009900 tx=0x7fe96002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:16.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 -- 172.21.15.113:0/1814350283 shutdown_connections 2024-04-22T07:14:16.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 --2- 172.21.15.113:0/1814350283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9701021d0 0x7fe970108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 --2- 172.21.15.113:0/1814350283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe970101810 0x7fe970101c90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 --2- 172.21.15.113:0/1814350283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 0x7fe970100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 -- 172.21.15.113:0/1814350283 >> 172.21.15.113:0/1814350283 conn(0x7fe9700fbb90 msgr2=0x7fe9700fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 -- 172.21.15.113:0/1814350283 shutdown_connections 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 -- 172.21.15.113:0/1814350283 wait complete. 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 Processor -- start 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.398+0000 7fe977952640 1 -- start start 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 0x7fe970109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe970101810 0x7fe97019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 0x7fe97019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010cb10 con 0x7fe9701021d0 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010c990 con 0x7fe970101810 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010cc90 con 0x7fe970100620 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 0x7fe97019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 0x7fe97019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46254/0 (socket says 172.21.15.113:46254) 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 -- 172.21.15.113:0/3971140031 learned_addr learned my addr 172.21.15.113:0/3971140031 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe9756ee640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 0x7fe970109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe974eed640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe970101810 0x7fe97019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 -- 172.21.15.113:0/3971140031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 msgr2=0x7fe970109cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 0x7fe970109cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 -- 172.21.15.113:0/3971140031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe970101810 msgr2=0x7fe97019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe970101810 0x7fe97019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 -- 172.21.15.113:0/3971140031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97019fee0 con 0x7fe9701021d0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe9756ee640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 0x7fe970109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe975eef640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 0x7fe97019f620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00d750 tx=0x7fe96c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c017070 con 0x7fe9701021d0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701a01d0 con 0x7fe9701021d0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe96c0054f0 con 0x7fe9701021d0 2024-04-22T07:14:16.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c004cd0 con 0x7fe9701021d0 2024-04-22T07:14:16.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.399+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701a06d0 con 0x7fe9701021d0 2024-04-22T07:14:16.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.400+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970107940 con 0x7fe9701021d0 2024-04-22T07:14:16.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.401+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe96c009bd0 con 0x7fe9701021d0 2024-04-22T07:14:16.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.402+0000 7fe9667fc640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe95c0a3c40 0x7fe95c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.402+0000 7fe9756ee640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe95c0a3c40 0x7fe95c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.402+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe96c0d0580 con 0x7fe9701021d0 2024-04-22T07:14:16.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.402+0000 7fe9756ee640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe95c0a3c40 0x7fe95c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe970101670 tx=0x7fe968009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.405+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe96c09ce10 con 0x7fe9701021d0 2024-04-22T07:14:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:16 smithi133 ceph-mon[31198]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:16 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3489033876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:16 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2502069315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:16.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:16 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2496548539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:16 smithi154 ceph-mon[31862]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:16 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3489033876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:16 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2502069315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:16.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:16 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2496548539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:16.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.580+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe97002d7d0 con 0x7fe9701021d0 2024-04-22T07:14:16.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.580+0000 7fe9667fc640 1 -- 172.21.15.113:0/3971140031 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe96c09c710 con 0x7fe9701021d0 2024-04-22T07:14:16.578 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe95c0a3c40 msgr2=0x7fe95c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe95c0a3c40 0x7fe95c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe970101670 tx=0x7fe968009520 comp rx=0 tx=0).stop 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 msgr2=0x7fe97019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 0x7fe97019f620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00d750 tx=0x7fe96c00dc10 comp rx=0 tx=0).stop 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 shutdown_connections 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fe95c0a3c40 0x7fe95c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9701021d0 0x7fe97019f620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe970101810 0x7fe97019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 --2- 172.21.15.113:0/3971140031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe970100620 0x7fe970109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 >> 172.21.15.113:0/3971140031 conn(0x7fe9700fbb90 msgr2=0x7fe970104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 shutdown_connections 2024-04-22T07:14:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.582+0000 7fe977952640 1 -- 172.21.15.113:0/3971140031 wait complete. 2024-04-22T07:14:16.591 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:16.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 -- 172.21.15.113:0/721389221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 msgr2=0x7fc79410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 --2- 172.21.15.113:0/721389221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79410a0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009ca0 tx=0x7fc78c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 -- 172.21.15.113:0/721389221 shutdown_connections 2024-04-22T07:14:16.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 --2- 172.21.15.113:0/721389221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79410a0e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 --2- 172.21.15.113:0/721389221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 0x7fc794107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 --2- 172.21.15.113:0/721389221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 0x7fc794104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 -- 172.21.15.113:0/721389221 >> 172.21.15.113:0/721389221 conn(0x7fc7940fc420 msgr2=0x7fc7940fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 -- 172.21.15.113:0/721389221 shutdown_connections 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.693+0000 7fc79bba4640 1 -- 172.21.15.113:0/721389221 wait complete. 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 Processor -- start 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 -- start start 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 0x7fc7941982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 0x7fc794198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410dc30 con 0x7fc794107c90 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410dab0 con 0x7fc794105360 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79bba4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc79410ddb0 con 0x7fc794102a30 2024-04-22T07:14:16.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46264/0 (socket says 172.21.15.113:46264) 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 -- 172.21.15.113:0/3114002662 learned_addr learned my addr 172.21.15.113:0/3114002662 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79913f640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 0x7fc794198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 -- 172.21.15.113:0/3114002662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 msgr2=0x7fc7941982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc799940640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 0x7fc7941982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 0x7fc7941982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 -- 172.21.15.113:0/3114002662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 msgr2=0x7fc794198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 0x7fc794198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79a141640 1 -- 172.21.15.113:0/3114002662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79419fef0 con 0x7fc794107c90 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc799940640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 0x7fc7941982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.694+0000 7fc79913f640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 0x7fc794198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.695+0000 7fc79a141640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79419f880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005cc0 tx=0x7fc78c032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.695+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c041020 con 0x7fc794107c90 2024-04-22T07:14:16.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.695+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc78c003e40 con 0x7fc794107c90 2024-04-22T07:14:16.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.695+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941a0180 con 0x7fc794107c90 2024-04-22T07:14:16.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.695+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c039040 con 0x7fc794107c90 2024-04-22T07:14:16.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.695+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941a0640 con 0x7fc794107c90 2024-04-22T07:14:16.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.696+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7940fdf10 con 0x7fc794107c90 2024-04-22T07:14:16.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.697+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc78c044020 con 0x7fc794107c90 2024-04-22T07:14:16.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.698+0000 7fc78affd640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc7800a3c40 0x7fc7800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.698+0000 7fc799940640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc7800a3c40 0x7fc7800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.698+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc78c0f2e20 con 0x7fc794107c90 2024-04-22T07:14:16.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.698+0000 7fc799940640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc7800a3c40 0x7fc7800a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7900058c0 tx=0x7fc790005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.702+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc78c04b020 con 0x7fc794107c90 2024-04-22T07:14:16.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.879+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc79402d5c0 con 0x7fc794107c90 2024-04-22T07:14:16.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.880+0000 7fc78affd640 1 -- 172.21.15.113:0/3114002662 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fc78c0bf7c0 con 0x7fc794107c90 2024-04-22T07:14:16.878 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc7800a3c40 msgr2=0x7fc7800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc7800a3c40 0x7fc7800a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7900058c0 tx=0x7fc790005d30 comp rx=0 tx=0).stop 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 msgr2=0x7fc79419f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79419f880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005cc0 tx=0x7fc78c032950 comp rx=0 tx=0).stop 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 shutdown_connections 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fc7800a3c40 0x7fc7800a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc794107c90 0x7fc79419f880 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc794105360 0x7fc794198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 --2- 172.21.15.113:0/3114002662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc794102a30 0x7fc7941982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 >> 172.21.15.113:0/3114002662 conn(0x7fc7940fc420 msgr2=0x7fc7941032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 shutdown_connections 2024-04-22T07:14:16.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.882+0000 7fc79bba4640 1 -- 172.21.15.113:0/3114002662 wait complete. 2024-04-22T07:14:16.891 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:16.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 -- 172.21.15.113:0/3989608714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44100e90 msgr2=0x7efc441012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 --2- 172.21.15.113:0/3989608714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44100e90 0x7efc441012b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc40009900 tx=0x7efc4002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 -- 172.21.15.113:0/3989608714 shutdown_connections 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 --2- 172.21.15.113:0/3989608714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 0x7efc441092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 --2- 172.21.15.113:0/3989608714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc44101fe0 0x7efc44102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 --2- 172.21.15.113:0/3989608714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44100e90 0x7efc441012b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 -- 172.21.15.113:0/3989608714 >> 172.21.15.113:0/3989608714 conn(0x7efc440fc400 msgr2=0x7efc440fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 -- 172.21.15.113:0/3989608714 shutdown_connections 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 -- 172.21.15.113:0/3989608714 wait complete. 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 Processor -- start 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.994+0000 7efc4b843640 1 -- start start 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc44100e90 0x7efc441983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 0x7efc441988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 0x7efc4419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410dcf0 con 0x7efc44101fe0 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410db70 con 0x7efc441029a0 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efc4410de70 con 0x7efc44100e90 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 0x7efc441988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 0x7efc441988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46272/0 (socket says 172.21.15.113:46272) 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 -- 172.21.15.113:0/944897839 learned_addr learned my addr 172.21.15.113:0/944897839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc495df640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc44100e90 0x7efc441983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 -- 172.21.15.113:0/944897839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc44100e90 msgr2=0x7efc441983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc44100e90 0x7efc441983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc49de0640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 0x7efc4419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 -- 172.21.15.113:0/944897839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 msgr2=0x7efc4419f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 0x7efc4419f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:16.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 -- 172.21.15.113:0/944897839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4419ffc0 con 0x7efc44101fe0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc49de0640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 0x7efc4419f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc48dde640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 0x7efc441988e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efc3400b450 tx=0x7efc3400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34015020 con 0x7efc44101fe0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efc34011d60 con 0x7efc44101fe0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc441a02b0 con 0x7efc44101fe0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3400f340 con 0x7efc44101fe0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.995+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc441a0880 con 0x7efc44101fe0 2024-04-22T07:14:16.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.996+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc441081a0 con 0x7efc44101fe0 2024-04-22T07:14:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.998+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efc34018020 con 0x7efc44101fe0 2024-04-22T07:14:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.998+0000 7efc3a7fc640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7efc300a3c40 0x7efc300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.998+0000 7efc495df640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7efc300a3c40 0x7efc300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.998+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7efc340d04c0 con 0x7efc44101fe0 2024-04-22T07:14:16.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:16.999+0000 7efc495df640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7efc300a3c40 0x7efc300a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c00 tx=0x7efc4002c850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:16.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.002+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc3401f020 con 0x7efc44101fe0 2024-04-22T07:14:17.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.181+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc4402d810 con 0x7efc44101fe0 2024-04-22T07:14:17.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.182+0000 7efc3a7fc640 1 -- 172.21.15.113:0/944897839 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7efc3400d040 con 0x7efc44101fe0 2024-04-22T07:14:17.179 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:17.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7efc300a3c40 msgr2=0x7efc300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7efc300a3c40 0x7efc300a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c00 tx=0x7efc4002c850 comp rx=0 tx=0).stop 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 msgr2=0x7efc441988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 0x7efc441988e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efc3400b450 tx=0x7efc3400b910 comp rx=0 tx=0).stop 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 shutdown_connections 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7efc300a3c40 0x7efc300a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc441029a0 0x7efc4419f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc44101fe0 0x7efc441988e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 --2- 172.21.15.113:0/944897839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc44100e90 0x7efc441983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 >> 172.21.15.113:0/944897839 conn(0x7efc440fc400 msgr2=0x7efc44105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 shutdown_connections 2024-04-22T07:14:17.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.184+0000 7efc4b843640 1 -- 172.21.15.113:0/944897839 wait complete. 2024-04-22T07:14:17.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:17.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 -- 172.21.15.113:0/1032210507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a0107cd0 msgr2=0x7f85a010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 --2- 172.21.15.113:0/1032210507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a0107cd0 0x7f85a010a0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8598009ca0 tx=0x7f859802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 -- 172.21.15.113:0/1032210507 shutdown_connections 2024-04-22T07:14:17.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 --2- 172.21.15.113:0/1032210507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a0107cd0 0x7f85a010a0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 --2- 172.21.15.113:0/1032210507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a01053a0 0x7f85a0107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 --2- 172.21.15.113:0/1032210507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0102a70 0x7f85a0104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 -- 172.21.15.113:0/1032210507 >> 172.21.15.113:0/1032210507 conn(0x7f85a00fc400 msgr2=0x7f85a00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 -- 172.21.15.113:0/1032210507 shutdown_connections 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.293+0000 7f85a6481640 1 -- 172.21.15.113:0/1032210507 wait complete. 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 Processor -- start 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 -- start start 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a0102a70 0x7f85a0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 0x7f85a01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0107cd0 0x7f85a019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010dc50 con 0x7f85a01053a0 2024-04-22T07:14:17.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010dad0 con 0x7f85a0107cd0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a6481640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010ddd0 con 0x7f85a0102a70 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 0x7f85a01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 0x7f85a01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46288/0 (socket says 172.21.15.113:46288) 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 -- 172.21.15.113:0/3109594193 learned_addr learned my addr 172.21.15.113:0/3109594193 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f85a4a1e640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0107cd0 0x7f85a019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859ffff640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a0102a70 0x7f85a0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 -- 172.21.15.113:0/3109594193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a0102a70 msgr2=0x7f85a0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a0102a70 0x7f85a0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 -- 172.21.15.113:0/3109594193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0107cd0 msgr2=0x7f85a019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0107cd0 0x7f85a019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859f7fe640 1 -- 172.21.15.113:0/3109594193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a019feb0 con 0x7f85a01053a0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.294+0000 7f859ffff640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a0102a70 0x7f85a0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.295+0000 7f859f7fe640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 0x7f85a01987d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f859400b450 tx=0x7f859400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.295+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594015020 con 0x7f85a01053a0 2024-04-22T07:14:17.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.295+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8594011d60 con 0x7f85a01053a0 2024-04-22T07:14:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.295+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a01a01a0 con 0x7f85a01053a0 2024-04-22T07:14:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.295+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859400f340 con 0x7f85a01053a0 2024-04-22T07:14:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.295+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a0720 con 0x7f85a01053a0 2024-04-22T07:14:17.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.296+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a00fdef0 con 0x7f85a01053a0 2024-04-22T07:14:17.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.297+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8594018020 con 0x7f85a01053a0 2024-04-22T07:14:17.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.297+0000 7f859d7fa640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85880a3c40 0x7f85880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.298+0000 7f859ffff640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85880a3c40 0x7f85880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.298+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f85940d0520 con 0x7f85a01053a0 2024-04-22T07:14:17.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.298+0000 7f859ffff640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85880a3c40 0x7f85880a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85900058c0 tx=0x7f8590005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:17.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.301+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f859401f020 con 0x7f85a01053a0 2024-04-22T07:14:17.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.512+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85a002d810 con 0x7f85a01053a0 2024-04-22T07:14:17.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.513+0000 7f859d7fa640 1 -- 172.21.15.113:0/3109594193 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11875 (secure 0 0 0) 0x7f859400d040 con 0x7f85a01053a0 2024-04-22T07:14:17.511 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:17.511 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:12:58.522890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24473,24506]},"id":1}]} 2024-04-22T07:14:17.511 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:14:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85880a3c40 msgr2=0x7f85880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85880a3c40 0x7f85880a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f85900058c0 tx=0x7f8590005ec0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 msgr2=0x7f85a01987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 0x7f85a01987d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f859400b450 tx=0x7f859400b910 comp rx=0 tx=0).stop 2024-04-22T07:14:17.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 shutdown_connections 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85880a3c40 0x7f85880a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85a0107cd0 0x7f85a019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a01053a0 0x7f85a01987d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 --2- 172.21.15.113:0/3109594193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85a0102a70 0x7f85a0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 >> 172.21.15.113:0/3109594193 conn(0x7f85a00fc400 msgr2=0x7f85a01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 shutdown_connections 2024-04-22T07:14:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.515+0000 7f85a6481640 1 -- 172.21.15.113:0/3109594193 wait complete. 2024-04-22T07:14:17.523 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:17 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1913050635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:17.523 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:17 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3971140031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:17.523 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:17 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3114002662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:17.524 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:17 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1913050635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:17 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3971140031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:17 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3114002662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:17.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:17 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1913050635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:17.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:17 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3971140031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:17.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:17 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3114002662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:17.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.624+0000 7f441b3f4640 1 -- 172.21.15.113:0/70529763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414101fc0 msgr2=0x7f4414102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.624+0000 7f441b3f4640 1 --2- 172.21.15.113:0/70529763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414101fc0 0x7f4414102440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4404009900 tx=0x7f440402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.624+0000 7f441b3f4640 1 -- 172.21.15.113:0/70529763 shutdown_connections 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.624+0000 7f441b3f4640 1 --2- 172.21.15.113:0/70529763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414102980 0x7f44141092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 --2- 172.21.15.113:0/70529763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414101fc0 0x7f4414102440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 --2- 172.21.15.113:0/70529763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414100e70 0x7f4414101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- 172.21.15.113:0/70529763 >> 172.21.15.113:0/70529763 conn(0x7f44140fc3e0 msgr2=0x7f44140fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- 172.21.15.113:0/70529763 shutdown_connections 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- 172.21.15.113:0/70529763 wait complete. 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 Processor -- start 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- start start 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414100e70 0x7f44141982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414101fc0 0x7f4414198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 0x7f441419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f441410cb10 con 0x7f4414102980 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f441410c990 con 0x7f4414100e70 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441b3f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f441410cc90 con 0x7f4414101fc0 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f4419991640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 0x7f441419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f4419991640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 0x7f441419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46308/0 (socket says 172.21.15.113:46308) 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f4419991640 1 -- 172.21.15.113:0/3832707888 learned_addr learned my addr 172.21.15.113:0/3832707888 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.625+0000 7f441898f640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414101fc0 0x7f4414198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419190640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414100e70 0x7f44141982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419991640 1 -- 172.21.15.113:0/3832707888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414101fc0 msgr2=0x7f4414198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419991640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414101fc0 0x7f4414198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419991640 1 -- 172.21.15.113:0/3832707888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414100e70 msgr2=0x7f44141982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419991640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414100e70 0x7f44141982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419991640 1 -- 172.21.15.113:0/3832707888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441419fef0 con 0x7f4414102980 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f441898f640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414101fc0 0x7f4414198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f4419991640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 0x7f441419f880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f440c00eaa0 tx=0x7f440c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c00cdb0 con 0x7f4414102980 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f440c005020 con 0x7f4414102980 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141a01e0 con 0x7f4414102980 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0116d0 con 0x7f4414102980 2024-04-22T07:14:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.626+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a06b0 con 0x7f4414102980 2024-04-22T07:14:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.627+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44141a0970 con 0x7f4414102980 2024-04-22T07:14:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.628+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f440c0051d0 con 0x7f4414102980 2024-04-22T07:14:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.629+0000 7f440a7fc640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f44000a3c40 0x7f44000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.629+0000 7f4419190640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f44000a3c40 0x7f44000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.629+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f440c0d0c50 con 0x7f4414102980 2024-04-22T07:14:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.629+0000 7f4419190640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f44000a3c40 0x7f44000a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4410005fd0 tx=0x7f4410005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.632+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f440c016030 con 0x7f4414102980 2024-04-22T07:14:17.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.811+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44141a0b40 con 0x7f4414102980 2024-04-22T07:14:17.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.813+0000 7f440a7fc640 1 -- 172.21.15.113:0/3832707888 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22700 (secure 0 0 0) 0x7f440c09d4e0 con 0x7f4414102980 2024-04-22T07:14:17.811 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:17.811 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:13:56.180466+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:17.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f44000a3c40 msgr2=0x7f44000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f44000a3c40 0x7f44000a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4410005fd0 tx=0x7f4410005850 comp rx=0 tx=0).stop 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 msgr2=0x7f441419f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 0x7f441419f880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f440c00eaa0 tx=0x7f440c00ef60 comp rx=0 tx=0).stop 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 shutdown_connections 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f44000a3c40 0x7f44000a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4414102980 0x7f441419f880 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4414101fc0 0x7f4414198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 --2- 172.21.15.113:0/3832707888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414100e70 0x7f44141982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 >> 172.21.15.113:0/3832707888 conn(0x7f44140fc3e0 msgr2=0x7f44140690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 shutdown_connections 2024-04-22T07:14:17.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.815+0000 7f441b3f4640 1 -- 172.21.15.113:0/3832707888 wait complete. 2024-04-22T07:14:17.824 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:17.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 -- 172.21.15.113:0/138137357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102990 msgr2=0x7fba581092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/138137357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102990 0x7fba581092b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fba48009ca0 tx=0x7fba4802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 -- 172.21.15.113:0/138137357 shutdown_connections 2024-04-22T07:14:17.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/138137357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102990 0x7fba581092b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/138137357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58101fd0 0x7fba58102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/138137357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 0x7fba581012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 -- 172.21.15.113:0/138137357 >> 172.21.15.113:0/138137357 conn(0x7fba580fc410 msgr2=0x7fba580fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 -- 172.21.15.113:0/138137357 shutdown_connections 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.924+0000 7fba5cf2c640 1 -- 172.21.15.113:0/138137357 wait complete. 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 Processor -- start 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 -- start start 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 0x7fba58198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 0x7fba581986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58102990 0x7fba5819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba5810dc00 con 0x7fba58101fd0 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba5810da80 con 0x7fba58102990 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5cf2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba5810dd80 con 0x7fba58100e80 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 0x7fba581986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 0x7fba581986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46328/0 (socket says 172.21.15.113:46328) 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 -- 172.21.15.113:0/784107963 learned_addr learned my addr 172.21.15.113:0/784107963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5659c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 0x7fba58198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba56d9d640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58102990 0x7fba5819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 -- 172.21.15.113:0/784107963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 msgr2=0x7fba58198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 0x7fba58198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 -- 172.21.15.113:0/784107963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58102990 msgr2=0x7fba5819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58102990 0x7fba5819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 -- 172.21.15.113:0/784107963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5819fda0 con 0x7fba58101fd0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba5659c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 0x7fba58198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba55d9b640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 0x7fba581986c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba4000d770 tx=0x7fba4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.925+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40017070 con 0x7fba58101fd0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.926+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba581a0090 con 0x7fba58101fd0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.926+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fba40009c30 con 0x7fba58101fd0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.926+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40003d70 con 0x7fba58101fd0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.926+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba581a05e0 con 0x7fba58101fd0 2024-04-22T07:14:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.927+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba581080c0 con 0x7fba58101fd0 2024-04-22T07:14:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.928+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fba40003f10 con 0x7fba58101fd0 2024-04-22T07:14:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.928+0000 7fba4f7fe640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fba440a3c40 0x7fba440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.928+0000 7fba5659c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fba440a3c40 0x7fba440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.928+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fba400cffa0 con 0x7fba58101fd0 2024-04-22T07:14:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.929+0000 7fba5659c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fba440a3c40 0x7fba440a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba5000abe0 tx=0x7fba5000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:17.933+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba4009c830 con 0x7fba58101fd0 2024-04-22T07:14:18.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.146+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba5802d660 con 0x7fba58101fd0 2024-04-22T07:14:18.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.147+0000 7fba4f7fe640 1 -- 172.21.15.113:0/784107963 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11875 (secure 0 0 0) 0x7fba40009de0 con 0x7fba58101fd0 2024-04-22T07:14:18.145 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:18.145 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:12:58.522890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24473,24506]},"id":1}]} 2024-04-22T07:14:18.145 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fba440a3c40 msgr2=0x7fba440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fba440a3c40 0x7fba440a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba5000abe0 tx=0x7fba5000a5c0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 msgr2=0x7fba581986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 0x7fba581986c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba4000d770 tx=0x7fba4000dc30 comp rx=0 tx=0).stop 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 shutdown_connections 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fba440a3c40 0x7fba440a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58102990 0x7fba5819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58101fd0 0x7fba581986c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 --2- 172.21.15.113:0/784107963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58100e80 0x7fba58198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 >> 172.21.15.113:0/784107963 conn(0x7fba580fc410 msgr2=0x7fba58105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 shutdown_connections 2024-04-22T07:14:18.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.149+0000 7fba5cf2c640 1 -- 172.21.15.113:0/784107963 wait complete. 2024-04-22T07:14:18.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T07:12:54.514841+0000', 'modified': '2024-04-22T07:12:58.522890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24473, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/217490889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 217490889}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 217490889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/1615298521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1615298521}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1615298521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/3935683574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3935683574}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3935683574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.154:6839/3404626603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 3404626603}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 3404626603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6837/65907218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 65907218}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 65907218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6839/982201245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 982201245}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 982201245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24469, 24473, 24506]} 2024-04-22T07:14:18.159 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-22T07:14:18.159 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T07:14:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 -- 172.21.15.113:0/2507817561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24102a00 msgr2=0x7eff24109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/2507817561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24102a00 0x7eff24109320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009ca0 tx=0x7eff1c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 -- 172.21.15.113:0/2507817561 shutdown_connections 2024-04-22T07:14:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/2507817561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24102a00 0x7eff24109320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/2507817561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 0x7eff241024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/2507817561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24100e60 0x7eff24101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 -- 172.21.15.113:0/2507817561 >> 172.21.15.113:0/2507817561 conn(0x7eff240fc420 msgr2=0x7eff240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 -- 172.21.15.113:0/2507817561 shutdown_connections 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.260+0000 7eff2c0d5640 1 -- 172.21.15.113:0/2507817561 wait complete. 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 Processor -- start 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 -- start start 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 0x7eff2406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 0x7eff2419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24102a00 0x7eff2419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff2410cc30 con 0x7eff24100e60 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff2410cab0 con 0x7eff24102a00 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff2410cdb0 con 0x7eff24102040 2024-04-22T07:14:18.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 0x7eff2406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 0x7eff2406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46346/0 (socket says 172.21.15.113:46346) 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 -- 172.21.15.113:0/3596083238 learned_addr learned my addr 172.21.15.113:0/3596083238 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29670640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 0x7eff2419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2a672640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24102a00 0x7eff2419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 -- 172.21.15.113:0/3596083238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 msgr2=0x7eff2419d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 0x7eff2419d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 -- 172.21.15.113:0/3596083238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24102a00 msgr2=0x7eff2419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24102a00 0x7eff2419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 -- 172.21.15.113:0/3596083238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff241a0080 con 0x7eff24100e60 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29670640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 0x7eff2419d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2a672640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24102a00 0x7eff2419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff29e71640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 0x7eff2406b0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff2000b450 tx=0x7eff2000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff1affd640 1 -- 172.21.15.113:0/3596083238 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20015020 con 0x7eff24100e60 2024-04-22T07:14:18.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff241a0310 con 0x7eff24100e60 2024-04-22T07:14:18.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff1affd640 1 -- 172.21.15.113:0/3596083238 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff20011d60 con 0x7eff24100e60 2024-04-22T07:14:18.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff1affd640 1 -- 172.21.15.113:0/3596083238 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2000f340 con 0x7eff24100e60 2024-04-22T07:14:18.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.261+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff241a0930 con 0x7eff24100e60 2024-04-22T07:14:18.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.263+0000 7eff1affd640 1 -- 172.21.15.113:0/3596083238 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7eff20018020 con 0x7eff24100e60 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 Processor -- start 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 -- start start 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee47805260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 0x55ee47805cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee4780ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ee47769b10 con 0x55ee47805830 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ee47707240 con 0x55ee478062d0 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ee47765ad0 con 0x55ee47804e40 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 0x55ee47805cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 0x55ee47805cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46354/0 (socket says 172.21.15.113:46354) 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 -- 172.21.15.113:0/46864174 learned_addr learned my addr 172.21.15.113:0/46864174 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff1affd640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff100a3c40 0x7eff100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29670640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee4780ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2a672640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee47805260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2a672640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff100a3c40 0x7eff100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 -- 172.21.15.113:0/46864174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 msgr2=0x55ee47805260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee47805260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 -- 172.21.15.113:0/46864174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 msgr2=0x55ee4780ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee4780ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29e71640 1 -- 172.21.15.113:0/46864174 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4747c2d0 con 0x55ee47805830 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff29670640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee4780ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.264+0000 7eff2a672640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee47805260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.265+0000 7eff1affd640 1 -- 172.21.15.113:0/3596083238 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7eff200cffa0 con 0x7eff24100e60 2024-04-22T07:14:18.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.265+0000 7eff29e71640 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 0x55ee47805cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7eff20098c10 tx=0x7eff20021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a8c5296daef64d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.265+0000 7eff0ffff640 1 -- 172.21.15.113:0/46864174 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20091790 con 0x55ee47805830 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.265+0000 7eff2a672640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff100a3c40 0x7eff100a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55ee47800200 tx=0x7eff1c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.265+0000 7eff0ffff640 1 -- 172.21.15.113:0/46864174 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff2009c940 con 0x55ee47805830 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.265+0000 7eff0ffff640 1 -- 172.21.15.113:0/46864174 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20091580 con 0x55ee47805830 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- 172.21.15.113:0/46864174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 msgr2=0x55ee47805cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 0x55ee47805cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7eff20098c10 tx=0x7eff20021000 comp rx=0 tx=0).stop 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- 172.21.15.113:0/46864174 shutdown_connections 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee4780ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47805830 0x55ee47805cb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/46864174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee47805260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- 172.21.15.113:0/46864174 >> 172.21.15.113:0/46864174 conn(0x55ee47800f40 msgr2=0x55ee477ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- 172.21.15.113:0/46864174 shutdown_connections 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- 172.21.15.113:0/46864174 wait complete. 2024-04-22T07:14:18.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- start start 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 Processor -- start 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee4780ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47805830 0x55ee4780f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47816400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ee4780d4b0 con 0x55ee478062d0 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ee476f2190 con 0x55ee47805830 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.266+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ee4780d630 con 0x55ee47804e40 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2ed9c740 10 client.0 _ll_register_callbacks cb 0x55ee476ed8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47816400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47816400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46370/0 (socket says 172.21.15.113:46370) 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 -- 172.21.15.113:0/1013193341 learned_addr learned my addr 172.21.15.113:0/1013193341 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff29e71640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee4780ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff29670640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47805830 0x55ee4780f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 -- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 msgr2=0x55ee4780ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee4780ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 -- 172.21.15.113:0/1013193341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47805830 msgr2=0x55ee4780f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47805830 0x55ee4780f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 -- 172.21.15.113:0/1013193341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee475466b0 con 0x55ee478062d0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff29670640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47805830 0x55ee4780f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2a672640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47816400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff1c036000 tx=0x7eff1c0325a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8495b013ec6133ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff0d7fa640 1 -- 172.21.15.113:0/1013193341 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c045050 con 0x55ee478062d0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff0d7fa640 1 -- 172.21.15.113:0/1013193341 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7eff1c0047c0 con 0x55ee478062d0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2009c940 con 0x55ee478062d0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff0d7fa640 1 -- 172.21.15.113:0/1013193341 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7eff1c039040 con 0x55ee478062d0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2ed9c740 10 client.15051 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ee47819750 con 0x55ee478062d0 2024-04-22T07:14:18.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.267+0000 7eff0d7fa640 1 -- 172.21.15.113:0/1013193341 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff2009c940 con 0x55ee478062d0 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff0d7fa640 1 -- 172.21.15.113:0/1013193341 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7eff1c04a440 con 0x55ee478062d0 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff2ed9c740 10 client.15051 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff2ed9c740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff2ed9c740 20 client.15051 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee4781be80 0x55ee4781d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff2ed9c740 4 client.15051 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee47777530 con 0x55ee4781be80 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff29e71640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee4781be80 0x55ee4781d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff0d7fa640 10 client.15051 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.268+0000 7eff29e71640 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee4781be80 0x55ee4781d2f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37475fa18094d76 server_cookie=4ea99f9f89bb6f6 in_seq=0 out_seq=0 2024-04-22T07:14:18.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff0d7fa640 1 -- 172.21.15.113:0/1013193341 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee47777530 con 0x55ee4781be80 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff0d7fa640 10 client.15051 handle_command_reply: tid=0 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 client.15051 shutdown 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 msgr2=0x55ee47816400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47816400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7eff1c036000 tx=0x7eff1c0325a0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 shutdown_connections 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee4781be80 0x55ee4781d2f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47816400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47805830 0x55ee4780f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff0d7fa640 0 client.15051 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1013193341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47804e40 0x55ee4780ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 >> 172.21.15.113:0/1013193341 conn(0x55ee47800f40 msgr2=0x55ee477fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 shutdown_connections 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1013193341 wait complete. 2024-04-22T07:14:18.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.269+0000 7eff2ed9c740 20 client.15051 trim_cache size 0 max 16384 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 Processor -- start 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 -- start start 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 0x55ee47808c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee477ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ee4781b630 con 0x55ee47804e40 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ee4781bd00 con 0x55ee478062d0 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.287+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ee4781c3d0 con 0x55ee47805830 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 0x55ee47808c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 0x55ee47808c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46384/0 (socket says 172.21.15.113:46384) 2024-04-22T07:14:18.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 -- 172.21.15.113:0/1203683116 learned_addr learned my addr 172.21.15.113:0/1203683116 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 -- 172.21.15.113:0/1203683116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 msgr2=0x55ee478163c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 -- 172.21.15.113:0/1203683116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 msgr2=0x55ee477ff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee477ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 -- 172.21.15.113:0/1203683116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee47800230 con 0x55ee47804e40 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff29e71640 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 0x55ee47808c70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff200986a0 tx=0x7eff200d3bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1425dbb9349d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff0f7fe640 1 -- 172.21.15.113:0/1203683116 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200a4eb0 con 0x55ee47804e40 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff0f7fe640 1 -- 172.21.15.113:0/1203683116 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff200b09a0 con 0x55ee47804e40 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.288+0000 7eff0f7fe640 1 -- 172.21.15.113:0/1203683116 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200d3860 con 0x55ee47804e40 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1203683116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 msgr2=0x55ee47808c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 0x55ee47808c70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7eff200986a0 tx=0x7eff200d3bd0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1203683116 shutdown_connections 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee478062d0 0x55ee477ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478163c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/1203683116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee47804e40 0x55ee47808c70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1203683116 >> 172.21.15.113:0/1203683116 conn(0x55ee47800f40 msgr2=0x55ee4780ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1203683116 shutdown_connections 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- 172.21.15.113:0/1203683116 wait complete. 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- start start 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 Processor -- start 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47804e40 0x55ee478123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47814e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ee4781d170 con 0x55ee478062d0 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ee4781caa0 con 0x55ee47804e40 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ee4781d840 con 0x55ee47805830 2024-04-22T07:14:18.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2ed9c740 10 client.0 _ll_register_callbacks cb 0x7eff1c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2a672640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47814e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2a672640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47814e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46388/0 (socket says 172.21.15.113:46388) 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.289+0000 7eff2a672640 1 -- 172.21.15.113:0/2426330824 learned_addr learned my addr 172.21.15.113:0/2426330824 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff29670640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2a672640 1 -- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 msgr2=0x55ee478148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2a672640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2a672640 1 -- 172.21.15.113:0/2426330824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47804e40 msgr2=0x55ee478123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2a672640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47804e40 0x55ee478123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2a672640 1 -- 172.21.15.113:0/2426330824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee47815370 con 0x55ee478062d0 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff29670640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2a672640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47814e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff1c04a900 tx=0x7eff1c032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7700191376367f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:18.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff0d7fa640 1 -- 172.21.15.113:0/2426330824 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c045040 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff0d7fa640 1 -- 172.21.15.113:0/2426330824 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7eff1c004610 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff200b09a0 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff0d7fa640 1 -- 172.21.15.113:0/2426330824 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7eff1c004810 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 10 client.15057 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ee47816650 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff0d7fa640 1 -- 172.21.15.113:0/2426330824 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff200b09a0 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff0d7fa640 1 -- 172.21.15.113:0/2426330824 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7eff1c03f590 con 0x55ee478062d0 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 10 client.15057 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 20 client.15057 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee47886730 0x55ee47885440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 4 client.15057 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.290+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "status"}) -- 0x55ee477713a0 con 0x55ee47886730 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff29e71640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee47886730 0x55ee47885440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff0d7fa640 10 client.15057 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff29e71640 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee47886730 0x55ee47885440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a8cfa0430b2aad1 server_cookie=d01954d79dc032bd in_seq=0 out_seq=0 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff0d7fa640 1 -- 172.21.15.113:0/2426330824 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ee477713a0 con 0x55ee47886730 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff0d7fa640 10 client.15057 handle_command_reply: tid=0 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 client.15057 shutdown 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 msgr2=0x55ee47814e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47814e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7eff1c04a900 tx=0x7eff1c032fc0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 shutdown_connections 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x55ee47886730 0x55ee47885440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ee478062d0 0x55ee47814e30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ee47805830 0x55ee478148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 --2- 172.21.15.113:0/2426330824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ee47804e40 0x55ee478123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff0d7fa640 0 client.15057 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.291+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 >> 172.21.15.113:0/2426330824 conn(0x55ee47800f40 msgr2=0x55ee4780cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 shutdown_connections 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2ed9c740 1 -- 172.21.15.113:0/2426330824 wait complete. 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2ed9c740 20 client.15057 trim_cache size 0 max 16384 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 2, 2024-04-22T07:14:18.296 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24469, 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 80.117439617000002, 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 88, 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 86, 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 96.885975918 2024-04-22T07:14:18.297 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff100a3c40 msgr2=0x7eff100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff100a3c40 0x7eff100a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55ee47800200 tx=0x7eff1c0095c0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 msgr2=0x7eff2406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 0x7eff2406b0c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff2000b450 tx=0x7eff2000b910 comp rx=0 tx=0).stop 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 shutdown_connections 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7eff100a3c40 0x7eff100a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff24102a00 0x7eff2419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff24102040 0x7eff2419d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 --2- 172.21.15.113:0/3596083238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff24100e60 0x7eff2406b0c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 >> 172.21.15.113:0/3596083238 conn(0x7eff240fc420 msgr2=0x7eff24105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 shutdown_connections 2024-04-22T07:14:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.292+0000 7eff2c0d5640 1 -- 172.21.15.113:0/3596083238 wait complete. 2024-04-22T07:14:18.301 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-22T07:14:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 -- 172.21.15.113:0/2761187436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 msgr2=0x7f5654101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 --2- 172.21.15.113:0/2761187436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654101280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5650009900 tx=0x7f565002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 -- 172.21.15.113:0/2761187436 shutdown_connections 2024-04-22T07:14:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 --2- 172.21.15.113:0/2761187436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 0x7f5654109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 --2- 172.21.15.113:0/2761187436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5654102040 0x7f56541024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 --2- 172.21.15.113:0/2761187436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654101280 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 -- 172.21.15.113:0/2761187436 >> 172.21.15.113:0/2761187436 conn(0x7f56540fc420 msgr2=0x7f56540fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 -- 172.21.15.113:0/2761187436 shutdown_connections 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.401+0000 7f565b9e2640 1 -- 172.21.15.113:0/2761187436 wait complete. 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 Processor -- start 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 -- start start 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5654102040 0x7f5654198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 0x7f565419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f565410dc40 con 0x7f5654100e60 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f565410dac0 con 0x7f5654102a00 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f565410ddc0 con 0x7f5654102040 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46416/0 (socket says 172.21.15.113:46416) 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 -- 172.21.15.113:0/1998062667 learned_addr learned my addr 172.21.15.113:0/1998062667 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f5658f7d640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5654102040 0x7f5654198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 -- 172.21.15.113:0/1998062667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5654102040 msgr2=0x7f5654198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f5659f7f640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 0x7f565419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5654102040 0x7f5654198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 -- 172.21.15.113:0/1998062667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 msgr2=0x7f565419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 0x7f565419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 -- 172.21.15.113:0/1998062667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565419ff30 con 0x7f5654100e60 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f5659f7f640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 0x7f565419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565977e640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654198310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5650036040 tx=0x7f5650005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f564a7fc640 1 -- 172.21.15.113:0/1998062667 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650041070 con 0x7f5654100e60 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56541a01c0 con 0x7f5654100e60 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f564a7fc640 1 -- 172.21.15.113:0/1998062667 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f56500044c0 con 0x7f5654100e60 2024-04-22T07:14:18.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f564a7fc640 1 -- 172.21.15.113:0/1998062667 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650034440 con 0x7f5654100e60 2024-04-22T07:14:18.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.402+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56541a0710 con 0x7f5654100e60 2024-04-22T07:14:18.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.404+0000 7f564a7fc640 1 -- 172.21.15.113:0/1998062667 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5650044070 con 0x7f5654100e60 2024-04-22T07:14:18.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 Processor -- start 2024-04-22T07:14:18.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 -- start start 2024-04-22T07:14:18.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba7b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba7bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba82eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56260b9dfb10 con 0x56260ba7c2d0 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56260b97d240 con 0x56260ba7ae40 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565e5cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56260b9dbad0 con 0x56260ba7b830 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba82eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba82eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46428/0 (socket says 172.21.15.113:46428) 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 -- 172.21.15.113:0/3663220289 learned_addr learned my addr 172.21.15.113:0/3663220289 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f564a7fc640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56400a3c40 0x7f56400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5659f7f640 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba7b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5659f7f640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56400a3c40 0x7f56400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f565977e640 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba7bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 -- 172.21.15.113:0/3663220289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 msgr2=0x56260ba7bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba7bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f564a7fc640 1 -- 172.21.15.113:0/1998062667 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f56500f2110 con 0x7f5654100e60 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 -- 172.21.15.113:0/3663220289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 msgr2=0x56260ba7b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.405+0000 7f5658f7d640 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba7b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f5658f7d640 1 -- 172.21.15.113:0/3663220289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260b6f22d0 con 0x56260ba7c2d0 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f565977e640 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba7bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f5658f7d640 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba82eb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f564400b740 tx=0x7f564400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf6a5d07629c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f5659f7f640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56400a3c40 0x7f56400a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f564c009270 tx=0x7f564c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f563f7fe640 1 -- 172.21.15.113:0/3663220289 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644003dd0 con 0x56260ba7c2d0 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f563f7fe640 1 -- 172.21.15.113:0/3663220289 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5644015070 con 0x56260ba7c2d0 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.406+0000 7f563f7fe640 1 -- 172.21.15.113:0/3663220289 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644004dd0 con 0x56260ba7c2d0 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- 172.21.15.113:0/3663220289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 msgr2=0x56260ba82eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba82eb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f564400b740 tx=0x7f564400bc00 comp rx=0 tx=0).stop 2024-04-22T07:14:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- 172.21.15.113:0/3663220289 shutdown_connections 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba82eb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba7bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- 172.21.15.113:0/3663220289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba7b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- 172.21.15.113:0/3663220289 >> 172.21.15.113:0/3663220289 conn(0x56260ba76f40 msgr2=0x56260ba75490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- 172.21.15.113:0/3663220289 shutdown_connections 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- 172.21.15.113:0/3663220289 wait complete. 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- start start 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 Processor -- start 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba90860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56260ba834b0 con 0x56260ba7c2d0 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56260b968190 con 0x56260ba7ae40 2024-04-22T07:14:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56260ba83630 con 0x56260ba7b830 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f565e5cb740 10 client.0 _ll_register_callbacks cb 0x56260b9638b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.407+0000 7f5659f7f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba90860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba90860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46430/0 (socket says 172.21.15.113:46430) 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 -- 172.21.15.113:0/883108811 learned_addr learned my addr 172.21.15.113:0/883108811 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f565977e640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5658f7d640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 -- 172.21.15.113:0/883108811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 msgr2=0x56260ba89820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba89820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 -- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 msgr2=0x56260ba892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 -- 172.21.15.113:0/883108811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260b7bc6b0 con 0x56260ba7c2d0 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f565977e640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f5659f7f640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba90860 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f564c009d50 tx=0x7f564c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e87d0b2334831e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f563cff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f563cff9640 1 -- 172.21.15.113:0/883108811 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c00fd70 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644015070 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f563cff9640 1 -- 172.21.15.113:0/883108811 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f564c004b20 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f563cff9640 1 -- 172.21.15.113:0/883108811 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f564c004d20 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f565e5cb740 10 client.15069 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56260ba93c00 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.408+0000 7f563cff9640 1 -- 172.21.15.113:0/883108811 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5644015070 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f563cff9640 1 -- 172.21.15.113:0/883108811 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f564c01d040 con 0x56260ba7c2d0 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565e5cb740 10 client.15069 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565e5cb740 10 client.15069 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565e5cb740 20 client.15069 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565e5cb740 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260ba962e0 0x56260ba97750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565e5cb740 4 client.15069 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260b9ed530 con 0x56260ba962e0 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565977e640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260ba962e0 0x56260ba97750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f563cff9640 10 client.15069 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.409+0000 7f565977e640 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260ba962e0 0x56260ba97750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f4a62e21c1cebe9 server_cookie=c92bb5c004a72857 in_seq=0 out_seq=0 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f563cff9640 1 -- 172.21.15.113:0/883108811 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56260b9ed530 con 0x56260ba962e0 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f563cff9640 10 client.15069 handle_command_reply: tid=0 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 client.15069 shutdown 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 msgr2=0x56260ba90860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba90860 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f564c009d50 tx=0x7f564c004a40 comp rx=0 tx=0).stop 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 shutdown_connections 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260ba962e0 0x56260ba97750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba90860 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f563cff9640 0 client.15069 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7b830 0x56260ba89820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 --2- 172.21.15.113:0/883108811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7ae40 0x56260ba892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.410+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 >> 172.21.15.113:0/883108811 conn(0x56260ba76f40 msgr2=0x56260ba74a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.411+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 shutdown_connections 2024-04-22T07:14:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.411+0000 7f565e5cb740 1 -- 172.21.15.113:0/883108811 wait complete. 2024-04-22T07:14:18.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.411+0000 7f565e5cb740 20 client.15069 trim_cache size 0 max 16384 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 Processor -- start 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 -- start start 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba87940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 0x56260ba90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7c2d0 0x56260ba87e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56260ba95a90 con 0x56260ba7b830 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56260ba96160 con 0x56260ba7c2d0 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565e5cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56260ba96830 con 0x56260ba7ae40 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 0x56260ba90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 0x56260ba90820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46432/0 (socket says 172.21.15.113:46432) 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 -- 172.21.15.113:0/803614273 learned_addr learned my addr 172.21.15.113:0/803614273 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5659f7f640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7c2d0 0x56260ba87e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565977e640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba87940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 -- 172.21.15.113:0/803614273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 msgr2=0x56260ba87940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba87940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 -- 172.21.15.113:0/803614273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7c2d0 msgr2=0x56260ba87e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7c2d0 0x56260ba87e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 -- 172.21.15.113:0/803614273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260ba76630 con 0x56260ba7b830 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f565977e640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba87940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5659f7f640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7c2d0 0x56260ba87e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.429+0000 7f5658f7d640 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 0x56260ba90820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5644005f50 tx=0x7f5644005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdd593b52a8afb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f563effd640 1 -- 172.21.15.113:0/803614273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644027070 con 0x56260ba7b830 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f563effd640 1 -- 172.21.15.113:0/803614273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5644015040 con 0x56260ba7b830 2024-04-22T07:14:18.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f563effd640 1 -- 172.21.15.113:0/803614273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56440049e0 con 0x56260ba7b830 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 -- 172.21.15.113:0/803614273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 msgr2=0x56260ba90820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 0x56260ba90820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5644005f50 tx=0x7f5644005f80 comp rx=0 tx=0).stop 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 -- 172.21.15.113:0/803614273 shutdown_connections 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7c2d0 0x56260ba87e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7b830 0x56260ba90820 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 --2- 172.21.15.113:0/803614273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba87940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 -- 172.21.15.113:0/803614273 >> 172.21.15.113:0/803614273 conn(0x56260ba76f40 msgr2=0x56260ba842d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 -- 172.21.15.113:0/803614273 shutdown_connections 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 -- 172.21.15.113:0/803614273 wait complete. 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.430+0000 7f565e5cb740 1 -- start start 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 Processor -- start 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba8ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7b830 0x56260ba8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba8efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56260ba975d0 con 0x56260ba7c2d0 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56260ba96f00 con 0x56260ba7b830 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56260ba97ca0 con 0x56260ba7ae40 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565e5cb740 10 client.0 _ll_register_callbacks cb 0x7f564c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba8efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba8efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46438/0 (socket says 172.21.15.113:46438) 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 -- 172.21.15.113:0/4016319468 learned_addr learned my addr 172.21.15.113:0/4016319468 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f565977e640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba8ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5658f7d640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7b830 0x56260ba8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 -- 172.21.15.113:0/4016319468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 msgr2=0x56260ba8ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba8ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 -- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7b830 msgr2=0x56260ba8bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7b830 0x56260ba8bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 -- 172.21.15.113:0/4016319468 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260ba8f5b0 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f5659f7f640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba8efc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f564c006010 tx=0x7f564c011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b775c88051d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.431+0000 7f563cff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f563cff9640 1 -- 172.21.15.113:0/4016319468 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c00fb70 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f563cff9640 1 -- 172.21.15.113:0/4016319468 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f564c00fd10 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644015040 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f563cff9640 1 -- 172.21.15.113:0/4016319468 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f564c0299e0 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 10 client.15075 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56260ba90820 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f563cff9640 1 -- 172.21.15.113:0/4016319468 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5644015040 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f563cff9640 1 -- 172.21.15.113:0/4016319468 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f564c034a40 con 0x56260ba7c2d0 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 10 client.15075 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 10 client.15075 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 20 client.15075 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:18.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260bb008f0 0x56260bafcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 4 client.15075 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "status"}) -- 0x56260b9e73a0 con 0x56260bb008f0 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.432+0000 7f565977e640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260bb008f0 0x56260bafcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f563cff9640 10 client.15075 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565977e640 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260bb008f0 0x56260bafcd30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e699ed0b6ccf2680 server_cookie=cc01e470f0753d6a in_seq=0 out_seq=0 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f563cff9640 1 -- 172.21.15.113:0/4016319468 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56260b9e73a0 con 0x56260bb008f0 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f563cff9640 10 client.15075 handle_command_reply: tid=0 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 client.15075 shutdown 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 msgr2=0x56260ba8efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba8efc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f564c006010 tx=0x7f564c011f40 comp rx=0 tx=0).stop 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 shutdown_connections 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x56260bb008f0 0x56260bafcd30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56260ba7c2d0 0x56260ba8efc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56260ba7b830 0x56260ba8bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f563cff9640 0 client.15075 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 --2- 172.21.15.113:0/4016319468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56260ba7ae40 0x56260ba8ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 >> 172.21.15.113:0/4016319468 conn(0x56260ba76f40 msgr2=0x56260ba74170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 shutdown_connections 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 1 -- 172.21.15.113:0/4016319468 wait complete. 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.433+0000 7f565e5cb740 20 client.15075 trim_cache size 0 max 16384 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:14:18.438 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 1, 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24473, 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 81.265935889000005, 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 88, 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 86, 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 99.751486731 2024-04-22T07:14:18.439 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56400a3c40 msgr2=0x7f56400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56400a3c40 0x7f56400a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f564c009270 tx=0x7f564c005bd0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 msgr2=0x7f5654198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654198310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5650036040 tx=0x7f5650005e60 comp rx=0 tx=0).stop 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 shutdown_connections 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f56400a3c40 0x7f56400a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5654102a00 0x7f565419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5654102040 0x7f5654198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 --2- 172.21.15.113:0/1998062667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5654100e60 0x7f5654198310 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 >> 172.21.15.113:0/1998062667 conn(0x7f56540fc420 msgr2=0x7f5654105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 shutdown_connections 2024-04-22T07:14:18.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.434+0000 7f565b9e2640 1 -- 172.21.15.113:0/1998062667 wait complete. 2024-04-22T07:14:18.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:18 smithi113 ceph-mon[27933]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:18.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/944897839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:18.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3109594193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.441 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3832707888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.442 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:18 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/784107963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:14:18.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 -- 172.21.15.113:0/4069911617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 msgr2=0x7f4be810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 --2- 172.21.15.113:0/4069911617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be810a5a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009ca0 tx=0x7f4be002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 -- 172.21.15.113:0/4069911617 shutdown_connections 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 --2- 172.21.15.113:0/4069911617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be810a5a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 --2- 172.21.15.113:0/4069911617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be8100030 0x7f4be81004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 --2- 172.21.15.113:0/4069911617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be81035c0 0x7f4be81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 -- 172.21.15.113:0/4069911617 >> 172.21.15.113:0/4069911617 conn(0x7f4be80fbb90 msgr2=0x7f4be80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 -- 172.21.15.113:0/4069911617 shutdown_connections 2024-04-22T07:14:18.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.544+0000 7f4beea55640 1 -- 172.21.15.113:0/4069911617 wait complete. 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 Processor -- start 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 -- start start 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be8100030 0x7f4be8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be81035c0 0x7f4be819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4be810dc20 con 0x7f4be81009f0 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4be810daa0 con 0x7f4be8100030 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4beea55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4be810dda0 con 0x7f4be81035c0 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46458/0 (socket says 172.21.15.113:46458) 2024-04-22T07:14:18.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 -- 172.21.15.113:0/3486978928 learned_addr learned my addr 172.21.15.113:0/3486978928 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be8100030 0x7f4be8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4becff2640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be81035c0 0x7f4be819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 -- 172.21.15.113:0/3486978928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be81035c0 msgr2=0x7f4be819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be81035c0 0x7f4be819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 -- 172.21.15.113:0/3486978928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be8100030 msgr2=0x7f4be8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be8100030 0x7f4be8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.545+0000 7f4be77fe640 1 -- 172.21.15.113:0/3486978928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be819fe30 con 0x7f4be81009f0 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.546+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be8198750 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00d770 tx=0x7f4bdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.546+0000 7f4be57fa640 1 -- 172.21.15.113:0/3486978928 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc017070 con 0x7f4be81009f0 2024-04-22T07:14:18.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.546+0000 7f4be57fa640 1 -- 172.21.15.113:0/3486978928 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4bdc0054f0 con 0x7f4be81009f0 2024-04-22T07:14:18.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.546+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be81a0120 con 0x7f4be81009f0 2024-04-22T07:14:18.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.546+0000 7f4be57fa640 1 -- 172.21.15.113:0/3486978928 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc004de0 con 0x7f4be81009f0 2024-04-22T07:14:18.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.546+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be81a0620 con 0x7f4be81009f0 2024-04-22T07:14:18.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.548+0000 7f4be57fa640 1 -- 172.21.15.113:0/3486978928 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4bdc004f80 con 0x7f4be81009f0 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.548+0000 7f4bf164a740 1 Processor -- start 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.548+0000 7f4bf164a740 1 -- start start 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.548+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a57670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.548+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.548+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a5f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4bf164a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b029b7b10 con 0x560b02a586e0 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4bf164a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b02955240 con 0x560b02a57c40 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be57fa640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4bd00a3c40 0x7f4bd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4bf164a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b029b3ad0 con 0x560b02a57250 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a5f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a5f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46470/0 (socket says 172.21.15.113:46470) 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 -- 172.21.15.113:0/157602752 learned_addr learned my addr 172.21.15.113:0/157602752 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4becff2640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4bd00a3c40 0x7f4bd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4becff2640 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a57670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be7fff640 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be57fa640 1 -- 172.21.15.113:0/3486978928 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4bdc0cffe0 con 0x7f4be81009f0 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 -- 172.21.15.113:0/157602752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 msgr2=0x560b02a57670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a57670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 -- 172.21.15.113:0/157602752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 msgr2=0x560b02a580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 -- 172.21.15.113:0/157602752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b026ca2d0 con 0x560b02a586e0 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be7fff640 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4becff2640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4bd00a3c40 0x7f4bd00a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4be0006010 tx=0x7f4be00093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4be77fe640 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a5f2c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0cde70 tx=0x7f4bdc0c7320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75245be177c32bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4bce7fc640 1 -- 172.21.15.113:0/157602752 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc020070 con 0x560b02a586e0 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4bce7fc640 1 -- 172.21.15.113:0/157602752 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4bdc09c210 con 0x560b02a586e0 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.549+0000 7f4bce7fc640 1 -- 172.21.15.113:0/157602752 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc005270 con 0x560b02a586e0 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 -- 172.21.15.113:0/157602752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 msgr2=0x560b02a5f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a5f2c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0cde70 tx=0x7f4bdc0c7320 comp rx=0 tx=0).stop 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 -- 172.21.15.113:0/157602752 shutdown_connections 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a5f2c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 --2- 172.21.15.113:0/157602752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a57670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 -- 172.21.15.113:0/157602752 >> 172.21.15.113:0/157602752 conn(0x560b02a4ef40 msgr2=0x560b02a4d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 -- 172.21.15.113:0/157602752 shutdown_connections 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 -- 172.21.15.113:0/157602752 wait complete. 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.550+0000 7f4bf164a740 1 -- start start 2024-04-22T07:14:18.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 Processor -- start 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b02a5f8c0 con 0x560b02a586e0 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b02940190 con 0x560b02a57c40 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b02a5fa40 con 0x560b02a57250 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 10 client.0 _ll_register_callbacks cb 0x560b0293b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46472/0 (socket says 172.21.15.113:46472) 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 -- 172.21.15.113:0/3109939859 learned_addr learned my addr 172.21.15.113:0/3109939859 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 -- 172.21.15.113:0/3109939859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 msgr2=0x560b02a612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 -- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 msgr2=0x560b02a617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 -- 172.21.15.113:0/3109939859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b027946b0 con 0x560b02a586e0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4becff2640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4be0036000 tx=0x7f4be0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaa289c27faf177 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3109939859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0049040 con 0x560b02a586e0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3109939859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4be0004240 con 0x560b02a586e0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.551+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bdc09c210 con 0x560b02a586e0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3109939859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4be0004440 con 0x560b02a586e0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 10 client.15087 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b02a6bc50 con 0x560b02a586e0 2024-04-22T07:14:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3109939859 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4bdc09c210 con 0x560b02a586e0 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3109939859 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4be0034a30 con 0x560b02a586e0 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 10 client.15087 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 20 client.15087 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02a6e360 0x560b02a6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 4 client.15087 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b029c5530 con 0x560b02a6e360 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.552+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02a6e360 0x560b02a6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.553+0000 7f4bc77fe640 10 client.15087 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.553+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02a6e360 0x560b02a6f7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374994ed227b0f84 server_cookie=91c80d05827ff52b in_seq=0 out_seq=0 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.553+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3109939859 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b029c5530 con 0x560b02a6e360 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.553+0000 7f4bc77fe640 10 client.15087 handle_command_reply: tid=0 2024-04-22T07:14:18.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 client.15087 shutdown 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 msgr2=0x560b02a68820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4be0036000 tx=0x7f4be0044000 comp rx=0 tx=0).stop 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 shutdown_connections 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02a6e360 0x560b02a6f7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3109939859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bc77fe640 0 client.15087 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 >> 172.21.15.113:0/3109939859 conn(0x560b02a4ef40 msgr2=0x560b02a4ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 shutdown_connections 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 1 -- 172.21.15.113:0/3109939859 wait complete. 2024-04-22T07:14:18.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.554+0000 7f4bf164a740 20 client.15087 trim_cache size 0 max 16384 2024-04-22T07:14:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:18 smithi133 ceph-mon[31198]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/944897839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3109594193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3832707888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:18 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/784107963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:18 smithi154 ceph-mon[31862]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/944897839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3109594193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3832707888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:18 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/784107963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 Processor -- start 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 -- start start 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b02a6db10 con 0x560b02a586e0 2024-04-22T07:14:18.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b02a6e1e0 con 0x560b02a57c40 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bf164a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b02a6e8b0 con 0x560b02a57250 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46480/0 (socket says 172.21.15.113:46480) 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 -- 172.21.15.113:0/2008505323 learned_addr learned my addr 172.21.15.113:0/2008505323 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 -- 172.21.15.113:0/2008505323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 msgr2=0x560b02a4d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 -- 172.21.15.113:0/2008505323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 msgr2=0x560b02a4d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 -- 172.21.15.113:0/2008505323 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b02a4e230 con 0x560b02a586e0 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4becff2640 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4be0034e20 tx=0x7f4be0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e74618f5bbc9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bcdffb640 1 -- 172.21.15.113:0/2008505323 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0049050 con 0x560b02a586e0 2024-04-22T07:14:18.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bcdffb640 1 -- 172.21.15.113:0/2008505323 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4be0052070 con 0x560b02a586e0 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.572+0000 7f4bcdffb640 1 -- 172.21.15.113:0/2008505323 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be00349f0 con 0x560b02a586e0 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 -- 172.21.15.113:0/2008505323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 msgr2=0x560b02a68740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4be0034e20 tx=0x7f4be0009920 comp rx=0 tx=0).stop 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 -- 172.21.15.113:0/2008505323 shutdown_connections 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a586e0 0x560b02a68740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 --2- 172.21.15.113:0/2008505323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a57250 0x560b02a4d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 -- 172.21.15.113:0/2008505323 >> 172.21.15.113:0/2008505323 conn(0x560b02a4ef40 msgr2=0x560b02a5f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 -- 172.21.15.113:0/2008505323 shutdown_connections 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 -- 172.21.15.113:0/2008505323 wait complete. 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.573+0000 7f4bf164a740 1 -- start start 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 Processor -- start 2024-04-22T07:14:18.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 0x560b02a647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a64d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a586e0 0x560b02a67d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b02a6f650 con 0x560b02a57250 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b02a6ef80 con 0x560b02a57c40 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b02a6fd20 con 0x560b02a586e0 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 10 client.0 _ll_register_callbacks cb 0x7f4be0046040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 0x560b02a647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 0x560b02a647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46484/0 (socket says 172.21.15.113:46484) 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 -- 172.21.15.113:0/3292040826 learned_addr learned my addr 172.21.15.113:0/3292040826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4becff2640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a586e0 0x560b02a67d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a64d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 -- 172.21.15.113:0/3292040826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a586e0 msgr2=0x560b02a67d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a586e0 0x560b02a67d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 -- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 msgr2=0x560b02a64d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a64d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 -- 172.21.15.113:0/3292040826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b02a682f0 con 0x560b02a57250 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4becff2640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a586e0 0x560b02a67d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a64d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4be7fff640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 0x560b02a647f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x560b02a6ff10 tx=0x7f4bd8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfe1475bbeabae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:18.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3292040826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80200f0 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3292040826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4bd8020290 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be0052070 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3292040826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4bd8020490 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 10 client.15093 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.574+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b02ad4bc0 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3292040826 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4be0052070 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3292040826 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4bd8010c60 con 0x560b02a57250 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bf164a740 10 client.15093 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bf164a740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bf164a740 20 client.15093 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02ad9830 0x560b02ad8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bf164a740 4 client.15093 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:14:18.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "status"}) -- 0x560b029bf3a0 con 0x560b02ad9830 2024-04-22T07:14:18.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02ad9830 0x560b02ad8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:18.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4bc77fe640 10 client.15093 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:18.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.575+0000 7f4be77fe640 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02ad9830 0x560b02ad8540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953d96a0a7e41085 server_cookie=5aeb534433a38329 in_seq=0 out_seq=0 2024-04-22T07:14:18.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bc77fe640 1 -- 172.21.15.113:0/3292040826 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x560b029bf3a0 con 0x560b02ad9830 2024-04-22T07:14:18.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bc77fe640 10 client.15093 handle_command_reply: tid=0 2024-04-22T07:14:18.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 client.15093 shutdown 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 msgr2=0x560b02a647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 0x560b02a647f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x560b02a6ff10 tx=0x7f4bd8004410 comp rx=0 tx=0).stop 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 shutdown_connections 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b02a586e0 0x560b02a67d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b02a57c40 0x560b02a64d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b02ad9830 0x560b02ad8540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 --2- 172.21.15.113:0/3292040826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b02a57250 0x560b02a647f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bc77fe640 0 client.15093 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 >> 172.21.15.113:0/3292040826 conn(0x560b02a4ef40 msgr2=0x560b02a5bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 shutdown_connections 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 1 -- 172.21.15.113:0/3292040826 wait complete. 2024-04-22T07:14:18.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4bf164a740 20 client.15093 trim_cache size 0 max 16384 2024-04-22T07:14:18.583 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:14:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-22T07:14:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24506, 2024-04-22T07:14:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:14:18.583 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 83.419385745, 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 88, 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 86, 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 89.064827180999998 2024-04-22T07:14:18.584 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:18.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4bd00a3c40 msgr2=0x7f4bd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4beea55640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4bd00a3c40 0x7f4bd00a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4be0006010 tx=0x7f4be00093f0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 msgr2=0x7f4be8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.576+0000 7f4beea55640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be8198750 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00d770 tx=0x7f4bdc00dc30 comp rx=0 tx=0).stop 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 shutdown_connections 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f4bd00a3c40 0x7f4bd00a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be81035c0 0x7f4be819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be81009f0 0x7f4be8198750 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 --2- 172.21.15.113:0/3486978928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be8100030 0x7f4be8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 >> 172.21.15.113:0/3486978928 conn(0x7f4be80fbb90 msgr2=0x7f4be81065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 shutdown_connections 2024-04-22T07:14:18.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:18.577+0000 7f4beea55640 1 -- 172.21.15.113:0/3486978928 wait complete. 2024-04-22T07:14:18.587 INFO:tasks.cephadm:Setup complete, yielding 2024-04-22T07:14:18.587 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T07:14:18.599 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:14:18.599 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-22T07:14:18.835 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 -- 172.21.15.113:0/421226127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4102ab0 msgr2=0x7f66b4102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/421226127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4102ab0 0x7f66b4102ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f669c0099c0 tx=0x7f669c009cd0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 -- 172.21.15.113:0/421226127 shutdown_connections 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/421226127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b4104650 0x7f66b410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/421226127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4102ab0 0x7f66b4102ed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/421226127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66b4103cb0 0x7f66b4104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 -- 172.21.15.113:0/421226127 >> 172.21.15.113:0/421226127 conn(0x7f66b40fe050 msgr2=0x7f66b4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:20.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 -- 172.21.15.113:0/421226127 shutdown_connections 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.264+0000 7f66ba5b8700 1 -- 172.21.15.113:0/421226127 wait complete. 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 Processor -- start 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 -- start start 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66b4102ab0 0x7f66b4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b4103cb0 0x7f66b4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 0x7f66b41a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f66b410e700 con 0x7f66b4104650 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66b410e580 con 0x7f66b4102ab0 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66ba5b8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f66b410e880 con 0x7f66b4103cb0 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 0x7f66b41a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 0x7f66b41a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46494/0 (socket says 172.21.15.113:46494) 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 -- 172.21.15.113:0/1258278958 learned_addr learned my addr 172.21.15.113:0/1258278958 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b37fe700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b4103cb0 0x7f66b4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:20.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b3fff700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66b4102ab0 0x7f66b4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 -- 172.21.15.113:0/1258278958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b4103cb0 msgr2=0x7f66b4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b4103cb0 0x7f66b4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 -- 172.21.15.113:0/1258278958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66b4102ab0 msgr2=0x7f66b4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66b4102ab0 0x7f66b4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.265+0000 7f66b8b55700 1 -- 172.21.15.113:0/1258278958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b41a0990 con 0x7f66b4104650 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.266+0000 7f66b8b55700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 0x7f66b41a02c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f66ac007ee0 tx=0x7f66ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.266+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac004bb0 con 0x7f66b4104650 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.266+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f66ac004d50 con 0x7f66b4104650 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.266+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b41a0c80 con 0x7f66b4104650 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.266+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0059e0 con 0x7f66b4104650 2024-04-22T07:14:20.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.266+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41a1190 con 0x7f66b4104650 2024-04-22T07:14:20.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.267+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b4069730 con 0x7f66b4104650 2024-04-22T07:14:20.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.268+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f66ac01a020 con 0x7f66b4104650 2024-04-22T07:14:20.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.268+0000 7f66b17fa700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f66a00a3cd0 0x7f66a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:20.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.269+0000 7f66b3fff700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f66a00a3cd0 0x7f66a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:20.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.269+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f66ac0d0270 con 0x7f66b4104650 2024-04-22T07:14:20.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.269+0000 7f66b3fff700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f66a00a3cd0 0x7f66a00a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:20.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.272+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66ac010070 con 0x7f66b4104650 2024-04-22T07:14:20.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:20 smithi133 ceph-mon[31198]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.487+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f66b4062e80 con 0x7f66a00a3cd0 2024-04-22T07:14:20.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:20 smithi113 ceph-mon[27933]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.488+0000 7f66b17fa700 1 -- 172.21.15.113:0/1258278958 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f66b4062e80 con 0x7f66a00a3cd0 2024-04-22T07:14:20.486 INFO:teuthology.orchestra.run.smithi113.stdout:Backend: cephadm 2024-04-22T07:14:20.487 INFO:teuthology.orchestra.run.smithi113.stdout:Available: Yes 2024-04-22T07:14:20.487 INFO:teuthology.orchestra.run.smithi113.stdout:Paused: No 2024-04-22T07:14:20.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.490+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f66a00a3cd0 msgr2=0x7f66a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:20.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.490+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f66a00a3cd0 0x7f66a00a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c009f10 comp rx=0 tx=0).stop 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 msgr2=0x7f66b41a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 0x7f66b41a02c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f66ac007ee0 tx=0x7f66ac00d3b0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 shutdown_connections 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f66a00a3cd0 0x7f66a00a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66b4104650 0x7f66b41a02c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66b4102ab0 0x7f66b4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 --2- 172.21.15.113:0/1258278958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66b4103cb0 0x7f66b4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 >> 172.21.15.113:0/1258278958 conn(0x7f66b40fe050 msgr2=0x7f66b4107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 shutdown_connections 2024-04-22T07:14:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:20.491+0000 7f66ba5b8700 1 -- 172.21.15.113:0/1258278958 wait complete. 2024-04-22T07:14:20.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:20 smithi154 ceph-mon[31862]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:21.091 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-22T07:14:21.341 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:22.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:22 smithi113 ceph-mon[27933]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:22.495 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:22 smithi113 ceph-mon[27933]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:22.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:22 smithi133 ceph-mon[31198]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:22.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:22 smithi133 ceph-mon[31198]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:22.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:22 smithi154 ceph-mon[31862]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:22.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:22 smithi154 ceph-mon[31862]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:22.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 -- 172.21.15.113:0/864255241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8103540 msgr2=0x7f2ba8105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 --2- 172.21.15.113:0/864255241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8103540 0x7f2ba8105930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009990 tx=0x7f2ba4009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 -- 172.21.15.113:0/864255241 shutdown_connections 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 --2- 172.21.15.113:0/864255241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba81087a0 0x7f2ba810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 --2- 172.21.15.113:0/864255241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8103540 0x7f2ba8105930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 --2- 172.21.15.113:0/864255241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba8105e70 0x7f2ba8108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 -- 172.21.15.113:0/864255241 >> 172.21.15.113:0/864255241 conn(0x7f2ba80fcf70 msgr2=0x7f2ba80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 -- 172.21.15.113:0/864255241 shutdown_connections 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 -- 172.21.15.113:0/864255241 wait complete. 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.995+0000 7f2bb055c700 1 Processor -- start 2024-04-22T07:14:22.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 -- start start 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8103540 0x7f2ba8102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 0x7f2ba81014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba81087a0 0x7f2ba8101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba810e770 con 0x7f2ba8105e70 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba810e5f0 con 0x7f2ba81087a0 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba810e8f0 con 0x7f2ba8103540 2024-04-22T07:14:22.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 0x7f2ba81014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 0x7f2ba81014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46522/0 (socket says 172.21.15.113:46522) 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 -- 172.21.15.113:0/3270386843 learned_addr learned my addr 172.21.15.113:0/3270386843 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bae2f8700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8103540 0x7f2ba8102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2baeaf9700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba81087a0 0x7f2ba8101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 -- 172.21.15.113:0/3270386843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8103540 msgr2=0x7f2ba8102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8103540 0x7f2ba8102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 -- 172.21.15.113:0/3270386843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba81087a0 msgr2=0x7f2ba8101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba81087a0 0x7f2ba8101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 -- 172.21.15.113:0/3270386843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba81022e0 con 0x7f2ba8105e70 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2baeaf9700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba81087a0 0x7f2ba8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2badaf7700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 0x7f2ba81014f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800b530 tx=0x7f2b9800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:22.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98015020 con 0x7f2ba8105e70 2024-04-22T07:14:22.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba81a2ff0 con 0x7f2ba8105e70 2024-04-22T07:14:22.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2b98011d00 con 0x7f2ba8105e70 2024-04-22T07:14:22.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.996+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800f2e0 con 0x7f2ba8105e70 2024-04-22T07:14:22.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.997+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba81a3530 con 0x7f2ba8105e70 2024-04-22T07:14:22.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.998+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba8069730 con 0x7f2ba8105e70 2024-04-22T07:14:22.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.999+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2b98018020 con 0x7f2ba8105e70 2024-04-22T07:14:22.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.999+0000 7f2b9f7fe700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b940a3cd0 0x7f2b940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:22.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.999+0000 7f2bae2f8700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b940a3cd0 0x7f2b940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:22.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:22.999+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2b980cfc70 con 0x7f2ba8105e70 2024-04-22T07:14:22.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.000+0000 7f2bae2f8700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b940a3cd0 0x7f2b940a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b5c0 tx=0x7f2ba4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:23.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.004+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b9801f020 con 0x7f2ba8105e70 2024-04-22T07:14:23.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.217+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2ba8062e80 con 0x7f2b940a3cd0 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.231+0000 7f2b9f7fe700 1 -- 172.21.15.113:0/3270386843 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f2ba8062e80 con 0x7f2b940a3cd0 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.a smithi113 running (96s) 14s ago 96s 13.9M - 19.0.0-3156-g86f7587a 1b5e03f260bd 332be3bd9c70 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.b smithi133 running (94s) 27s ago 93s 28.6M - 19.0.0-3156-g86f7587a 1b5e03f260bd b19aca13ef75 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.c smithi154 running (107s) 27s ago 107s 19.2M - 19.0.0-3156-g86f7587a 1b5e03f260bd 3b72115b888e 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.d smithi113 running (115s) 14s ago 115s 13.1M - 19.0.0-3156-g86f7587a 1b5e03f260bd 7ca41c63c4d6 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.e smithi133 running (112s) 27s ago 112s 20.4M - 19.0.0-3156-g86f7587a 1b5e03f260bd f9cf8bfae3a6 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.f smithi154 running (110s) 27s ago 109s 16.9M - 19.0.0-3156-g86f7587a 1b5e03f260bd 1dd73fe23a49 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.g smithi113 running (99s) 14s ago 98s 16.1M - 19.0.0-3156-g86f7587a 1b5e03f260bd 674baabee996 2024-04-22T07:14:23.229 INFO:teuthology.orchestra.run.smithi113.stdout:mds.h smithi133 running (104s) 27s ago 104s 20.4M - 19.0.0-3156-g86f7587a 1b5e03f260bd 9a81b14fc52e 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mds.i smithi154 running (102s) 27s ago 101s 19.1M - 19.0.0-3156-g86f7587a 1b5e03f260bd 2cb0ad707540 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mds.j smithi113 running (118s) 14s ago 118s 14.0M - 19.0.0-3156-g86f7587a 1b5e03f260bd 66dd16f85579 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mds.k smithi133 running (2m) 27s ago 2m 13.7M - 19.0.0-3156-g86f7587a 1b5e03f260bd 5d68cde14284 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mds.l smithi154 running (2m) 27s ago 2m 16.4M - 19.0.0-3156-g86f7587a 1b5e03f260bd da1eb41d991c 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mgr.x smithi113 *:9283,8765 running (9m) 14s ago 9m 514M - 19.0.0-3156-g86f7587a 1b5e03f260bd 8768c6742f43 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mgr.y smithi133 *:8443,8765 running (7m) 27s ago 7m 453M - 19.0.0-3156-g86f7587a 1b5e03f260bd 2e7396d59a13 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mgr.z smithi154 *:8443,8765 running (7m) 27s ago 7m 452M - 19.0.0-3156-g86f7587a 1b5e03f260bd 45a3c77739e9 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mon.a smithi113 running (9m) 14s ago 9m 52.0M 2048M 19.0.0-3156-g86f7587a 1b5e03f260bd b65780c4ca1b 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mon.b smithi133 running (7m) 27s ago 7m 38.9M 2048M 19.0.0-3156-g86f7587a 1b5e03f260bd ba8f14fe8628 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:mon.c smithi154 running (7m) 27s ago 7m 38.5M 2048M 19.0.0-3156-g86f7587a 1b5e03f260bd 83e618809653 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:osd.0 smithi113 running (7m) 14s ago 7m 48.3M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd c2f278c3b5da 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:osd.1 smithi133 running (6m) 27s ago 6m 60.5M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd 7df7f5889aa0 2024-04-22T07:14:23.230 INFO:teuthology.orchestra.run.smithi113.stdout:osd.2 smithi154 running (6m) 27s ago 6m 51.9M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd de7c281fd593 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.3 smithi113 running (5m) 14s ago 5m 46.6M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd b10e4c7aa2ea 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.4 smithi133 running (5m) 27s ago 5m 45.4M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd 65bb9d160d6d 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.5 smithi154 running (4m) 27s ago 4m 46.2M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd e07172a6ec87 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.6 smithi113 running (4m) 14s ago 4m 55.0M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd 2fe84bf5c718 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.7 smithi133 running (4m) 27s ago 4m 57.6M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd a798f39d6140 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.8 smithi154 running (3m) 27s ago 3m 46.6M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd 71855abbbcde 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.9 smithi113 running (3m) 14s ago 3m 49.6M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd e10700e6f1a4 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.10 smithi133 running (2m) 27s ago 2m 45.8M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd 10895c805196 2024-04-22T07:14:23.231 INFO:teuthology.orchestra.run.smithi113.stdout:osd.11 smithi154 running (2m) 27s ago 2m 55.0M 4272M 19.0.0-3156-g86f7587a 1b5e03f260bd 378a4e9451a4 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b940a3cd0 msgr2=0x7f2b940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b940a3cd0 0x7f2b940a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b5c0 tx=0x7f2ba4005fb0 comp rx=0 tx=0).stop 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 msgr2=0x7f2ba81014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 0x7f2ba81014f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800b530 tx=0x7f2b9800b840 comp rx=0 tx=0).stop 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 shutdown_connections 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f2b940a3cd0 0x7f2b940a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ba81087a0 0x7f2ba8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8103540 0x7f2ba8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 --2- 172.21.15.113:0/3270386843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba8105e70 0x7f2ba81014f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 >> 172.21.15.113:0/3270386843 conn(0x7f2ba80fcf70 msgr2=0x7f2ba8103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 shutdown_connections 2024-04-22T07:14:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:23.233+0000 7f2bb055c700 1 -- 172.21.15.113:0/3270386843 wait complete. 2024-04-22T07:14:23.920 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-22T07:14:24.159 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:24.481 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:24 smithi113 ceph-mon[27933]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:24.481 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:24 smithi113 ceph-mon[27933]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:24.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:24 smithi133 ceph-mon[31198]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:24.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:24 smithi133 ceph-mon[31198]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:24.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:24 smithi154 ceph-mon[31862]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:24.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:24 smithi154 ceph-mon[31862]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:25.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 -- 172.21.15.113:0/4035401480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c1087f0 msgr2=0x7f857c10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 --2- 172.21.15.113:0/4035401480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c1087f0 0x7f857c10ac00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f857400b0a0 tx=0x7f857400b3b0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 -- 172.21.15.113:0/4035401480 shutdown_connections 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 --2- 172.21.15.113:0/4035401480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c1087f0 0x7f857c10ac00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 --2- 172.21.15.113:0/4035401480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f857c103590 0x7f857c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 --2- 172.21.15.113:0/4035401480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c105ec0 0x7f857c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 -- 172.21.15.113:0/4035401480 >> 172.21.15.113:0/4035401480 conn(0x7f857c0fcf40 msgr2=0x7f857c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 -- 172.21.15.113:0/4035401480 shutdown_connections 2024-04-22T07:14:25.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 -- 172.21.15.113:0/4035401480 wait complete. 2024-04-22T07:14:25.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 Processor -- start 2024-04-22T07:14:25.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.718+0000 7f8583a41700 1 -- start start 2024-04-22T07:14:25.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c103590 0x7f857c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:25.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 0x7f857c101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:25.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f857c1087f0 0x7f857c101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10e680 con 0x7f857c105ec0 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10e500 con 0x7f857c103590 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f857c10e800 con 0x7f857c1087f0 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 0x7f857c101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 0x7f857c101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32852/0 (socket says 172.21.15.113:32852) 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 -- 172.21.15.113:0/986323453 learned_addr learned my addr 172.21.15.113:0/986323453 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8581fde700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f857c1087f0 0x7f857c101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f85817dd700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c103590 0x7f857c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 -- 172.21.15.113:0/986323453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f857c1087f0 msgr2=0x7f857c101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f857c1087f0 0x7f857c101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 -- 172.21.15.113:0/986323453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c103590 msgr2=0x7f857c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c103590 0x7f857c102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 -- 172.21.15.113:0/986323453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c102330 con 0x7f857c105ec0 2024-04-22T07:14:25.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f85817dd700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c103590 0x7f857c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8580fdc700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 0x7f857c101540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c9b0 tx=0x7f856c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c007b00 con 0x7f857c105ec0 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857c1a2dc0 con 0x7f857c105ec0 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f856c007ca0 con 0x7f857c105ec0 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c018640 con 0x7f857c105ec0 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.719+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857c1a3270 con 0x7f857c105ec0 2024-04-22T07:14:25.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.720+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c02df80 con 0x7f857c105ec0 2024-04-22T07:14:25.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.721+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f856c0188a0 con 0x7f857c105ec0 2024-04-22T07:14:25.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.722+0000 7f85727fc700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85680a3c40 0x7f85680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:25.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.722+0000 7f85817dd700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85680a3c40 0x7f85680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:25.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.722+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f856c0d03b0 con 0x7f857c105ec0 2024-04-22T07:14:25.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.722+0000 7f85817dd700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85680a3c40 0x7f85680a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8578005fd0 tx=0x7f8578005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:25.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.727+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f856c014030 con 0x7f857c105ec0 2024-04-22T07:14:25.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.935+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f857c062e80 con 0x7f85680a3c40 2024-04-22T07:14:25.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.941+0000 7f85727fc700 1 -- 172.21.15.113:0/986323453 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f857c062e80 con 0x7f85680a3c40 2024-04-22T07:14:25.938 INFO:teuthology.orchestra.run.smithi113.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-22T07:14:25.939 INFO:teuthology.orchestra.run.smithi113.stdout:mds.all 12/12 30s ago 2m smithi113=a;smithi113=d;smithi113=g;smithi113=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi154=c;smithi154=f;smithi154=i;smithi154=l;count:12 2024-04-22T07:14:25.939 INFO:teuthology.orchestra.run.smithi113.stdout:mgr 3/3 30s ago 7m smithi113=x;smithi133=y;smithi154=z;count:3 2024-04-22T07:14:25.939 INFO:teuthology.orchestra.run.smithi113.stdout:mon 3/3 30s ago 7m smithi113:172.21.15.113=a;smithi133:172.21.15.133=b;smithi154:172.21.15.154=c;count:3 2024-04-22T07:14:25.939 INFO:teuthology.orchestra.run.smithi113.stdout:osd 12 30s ago - 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85680a3c40 msgr2=0x7f85680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85680a3c40 0x7f85680a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8578005fd0 tx=0x7f8578005dc0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 msgr2=0x7f857c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 0x7f857c101540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c9b0 tx=0x7f856c00ccc0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 shutdown_connections 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f85680a3c40 0x7f85680a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f857c1087f0 0x7f857c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f857c103590 0x7f857c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 --2- 172.21.15.113:0/986323453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f857c105ec0 0x7f857c101540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 >> 172.21.15.113:0/986323453 conn(0x7f857c0fcf40 msgr2=0x7f857c103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 shutdown_connections 2024-04-22T07:14:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:25.943+0000 7f8583a41700 1 -- 172.21.15.113:0/986323453 wait complete. 2024-04-22T07:14:26.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:26 smithi113 ceph-mon[27933]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:26.292 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:26 smithi113 ceph-mon[27933]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T07:14:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:26 smithi133 ceph-mon[31198]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:26.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:26 smithi133 ceph-mon[31198]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T07:14:26.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:26 smithi154 ceph-mon[31862]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:14:26.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:26 smithi154 ceph-mon[31862]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T07:14:26.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-22T07:14:26.887 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:27.258 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:27 smithi113 ceph-mon[27933]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:27.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:27 smithi133 ceph-mon[31198]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:27.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:27 smithi154 ceph-mon[31862]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:28.278 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:28 smithi113 ceph-mon[27933]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 -- 172.21.15.113:0/2315133885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 msgr2=0x7fbe84103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 --2- 172.21.15.113:0/2315133885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe84103010 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe780099c0 tx=0x7fbe78009cd0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 -- 172.21.15.113:0/2315133885 shutdown_connections 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 --2- 172.21.15.113:0/2315133885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe84103550 0x7fbe84109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 --2- 172.21.15.113:0/2315133885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe841019b0 0x7fbe84101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 --2- 172.21.15.113:0/2315133885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe84103010 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 -- 172.21.15.113:0/2315133885 >> 172.21.15.113:0/2315133885 conn(0x7fbe840fcf70 msgr2=0x7fbe840ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 -- 172.21.15.113:0/2315133885 shutdown_connections 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.504+0000 7fbe8a799700 1 -- 172.21.15.113:0/2315133885 wait complete. 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 Processor -- start 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 -- start start 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe841019b0 0x7fbe8406bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe8419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe84103550 0x7fbe841a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410d560 con 0x7fbe84102bb0 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410d3e0 con 0x7fbe84103550 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe8a799700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410d6e0 con 0x7fbe841019b0 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe8419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe8419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32868/0 (socket says 172.21.15.113:32868) 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 -- 172.21.15.113:0/1168254265 learned_addr learned my addr 172.21.15.113:0/1168254265 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe83fff700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe841019b0 0x7fbe8406bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe88d36700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe84103550 0x7fbe841a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 -- 172.21.15.113:0/1168254265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe841019b0 msgr2=0x7fbe8406bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe841019b0 0x7fbe8406bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 -- 172.21.15.113:0/1168254265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe84103550 msgr2=0x7fbe841a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe84103550 0x7fbe841a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe837fe700 1 -- 172.21.15.113:0/1168254265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe841a0980 con 0x7fbe84102bb0 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.505+0000 7fbe83fff700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe841019b0 0x7fbe8406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.506+0000 7fbe837fe700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe8419da20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b580 tx=0x7fbe7800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.506+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78009e60 con 0x7fbe84102bb0 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.506+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe841a0c10 con 0x7fbe84102bb0 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.506+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbe7800f460 con 0x7fbe84102bb0 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.506+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78017650 con 0x7fbe84102bb0 2024-04-22T07:14:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.506+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe841a10c0 con 0x7fbe84102bb0 2024-04-22T07:14:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.507+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe84069730 con 0x7fbe84102bb0 2024-04-22T07:14:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.508+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fbe7800f610 con 0x7fbe84102bb0 2024-04-22T07:14:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.508+0000 7fbe817fa700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.508+0000 7fbe83fff700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.508+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbe780d1370 con 0x7fbe84102bb0 2024-04-22T07:14:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.509+0000 7fbe83fff700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe84102a10 tx=0x7fbe74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.513+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe78013070 con 0x7fbe84102bb0 2024-04-22T07:14:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:28 smithi133 ceph-mon[31198]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:28.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:28 smithi154 ceph-mon[31862]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:14:28.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.722+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbe84062e80 con 0x7fbe6c0a3c40 2024-04-22T07:14:28.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.724+0000 7fbe817fa700 1 -- 172.21.15.113:0/1168254265 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fbe84062e80 con 0x7fbe6c0a3c40 2024-04-22T07:14:28.721 INFO:teuthology.orchestra.run.smithi113.stdout:HOST ADDR LABELS STATUS 2024-04-22T07:14:28.722 INFO:teuthology.orchestra.run.smithi113.stdout:smithi113 172.21.15.113 2024-04-22T07:14:28.722 INFO:teuthology.orchestra.run.smithi113.stdout:smithi133 172.21.15.133 2024-04-22T07:14:28.722 INFO:teuthology.orchestra.run.smithi113.stdout:smithi154 172.21.15.154 2024-04-22T07:14:28.722 INFO:teuthology.orchestra.run.smithi113.stdout:3 hosts in cluster 2024-04-22T07:14:28.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbe6c0a3c40 msgr2=0x7fbe6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe84102a10 tx=0x7fbe74009260 comp rx=0 tx=0).stop 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 msgr2=0x7fbe8419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe8419da20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b580 tx=0x7fbe7800bf90 comp rx=0 tx=0).stop 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 shutdown_connections 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe84103550 0x7fbe841a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe841019b0 0x7fbe8406bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 --2- 172.21.15.113:0/1168254265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe84102bb0 0x7fbe8419da20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 >> 172.21.15.113:0/1168254265 conn(0x7fbe840fcf70 msgr2=0x7fbe84105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 shutdown_connections 2024-04-22T07:14:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:28.726+0000 7fbe8a799700 1 -- 172.21.15.113:0/1168254265 wait complete. 2024-04-22T07:14:29.321 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-22T07:14:29.561 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:30.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:30 smithi113 ceph-mon[27933]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:30.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:30 smithi113 ceph-mon[27933]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:30 smithi133 ceph-mon[31198]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:30 smithi133 ceph-mon[31198]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:30.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:30 smithi154 ceph-mon[31862]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:30 smithi154 ceph-mon[31862]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/507532728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8102aa0 msgr2=0x7f6ec8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/507532728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8102aa0 0x7f6ec8102ec0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80099c0 tx=0x7f6eb8009cd0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/507532728 shutdown_connections 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/507532728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec8104640 0x7f6ec810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/507532728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8102aa0 0x7f6ec8102ec0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/507532728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/507532728 >> 172.21.15.113:0/507532728 conn(0x7f6ec80fe040 msgr2=0x7f6ec8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:31.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/507532728 shutdown_connections 2024-04-22T07:14:31.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.228+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/507532728 wait complete. 2024-04-22T07:14:31.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.229+0000 7f6ecd2d8700 1 Processor -- start 2024-04-22T07:14:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.229+0000 7f6ecd2d8700 1 -- start start 2024-04-22T07:14:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.229+0000 7f6ecd2d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec8102aa0 0x7f6ec8198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ec8104640 0x7f6ec81a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec810e650 con 0x7f6ec8103ca0 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec810e4d0 con 0x7f6ec8104640 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec810e7d0 con 0x7f6ec8102aa0 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32892/0 (socket says 172.21.15.113:32892) 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 -- 172.21.15.113:0/4051205690 learned_addr learned my addr 172.21.15.113:0/4051205690 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec6ffd700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec8102aa0 0x7f6ec8198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec77fe700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ec8104640 0x7f6ec81a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 -- 172.21.15.113:0/4051205690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec8102aa0 msgr2=0x7f6ec8198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec8102aa0 0x7f6ec8198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 -- 172.21.15.113:0/4051205690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ec8104640 msgr2=0x7f6ec81a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ec8104640 0x7f6ec81a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 -- 172.21.15.113:0/4051205690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec81a0960 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ec67fc700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8199250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c920 tx=0x7f6ebc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc007b00 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec81a0c50 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6ebc004d00 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc005560 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.230+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec81a11b0 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.231+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec8069730 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.232+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6ebc004750 con 0x7f6ec8103ca0 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.233+0000 7f6eaffff700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6eb00a3c40 0x7f6eb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:31.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.233+0000 7f6ec6ffd700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6eb00a3c40 0x7f6eb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:31.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.233+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6ebc0cffa0 con 0x7f6ec8103ca0 2024-04-22T07:14:31.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.234+0000 7f6ec6ffd700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6eb00a3c40 0x7f6eb00a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8006010 tx=0x7f6eb8009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:31.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.238+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ebc014030 con 0x7f6ec8103ca0 2024-04-22T07:14:31.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.445+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6ec8062e80 con 0x7f6eb00a3c40 2024-04-22T07:14:31.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.446+0000 7f6eaffff700 1 -- 172.21.15.113:0/4051205690 <== mgr.14150 v2:172.21.15.113:6800/610623186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6ec8062e80 con 0x7f6eb00a3c40 2024-04-22T07:14:31.446 INFO:teuthology.orchestra.run.smithi113.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-22T07:14:31.446 INFO:teuthology.orchestra.run.smithi113.stdout:smithi113 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD5171000M2P0SGN 1863G No 3m ago Has a FileSystem, LVM detected 2024-04-22T07:14:31.447 INFO:teuthology.orchestra.run.smithi113.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T07:14:31.447 INFO:teuthology.orchestra.run.smithi113.stdout:smithi154 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233003B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T07:14:31.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.448+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6eb00a3c40 msgr2=0x7f6eb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:31.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.448+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6eb00a3c40 0x7f6eb00a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8006010 tx=0x7f6eb8009f10 comp rx=0 tx=0).stop 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.448+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 msgr2=0x7f6ec8199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.448+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8199250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c920 tx=0x7f6ebc00cc30 comp rx=0 tx=0).stop 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 shutdown_connections 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f6eb00a3c40 0x7f6eb00a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ec8104640 0x7f6ec81a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ec8102aa0 0x7f6ec8198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 --2- 172.21.15.113:0/4051205690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec8103ca0 0x7f6ec8199250 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 >> 172.21.15.113:0/4051205690 conn(0x7f6ec80fe040 msgr2=0x7f6ec81070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 shutdown_connections 2024-04-22T07:14:31.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:31.449+0000 7f6ecd2d8700 1 -- 172.21.15.113:0/4051205690 wait complete. 2024-04-22T07:14:32.093 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T07:14:32.160 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:14:32.160 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T07:14:32.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:32 smithi113 ceph-mon[27933]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:32.401 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:32 smithi133 ceph-mon[31198]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:32 smithi154 ceph-mon[31862]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:33.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:33 smithi113 ceph-mon[27933]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:33 smithi133 ceph-mon[31198]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:33.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:33 smithi154 ceph-mon[31862]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:33.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 -- 172.21.15.113:0/1448300515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af8103010 msgr2=0x7f0af8103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/1448300515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af8103010 0x7f0af8103430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009990 tx=0x7f0ae8009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 -- 172.21.15.113:0/1448300515 shutdown_connections 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/1448300515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af81025a0 0x7f0af8109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/1448300515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af8103010 0x7f0af8103430 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/1448300515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8101be0 0x7f0af8102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 -- 172.21.15.113:0/1448300515 >> 172.21.15.113:0/1448300515 conn(0x7f0af80fc6d0 msgr2=0x7f0af80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 -- 172.21.15.113:0/1448300515 shutdown_connections 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.937+0000 7f0aff8c3700 1 -- 172.21.15.113:0/1448300515 wait complete. 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 Processor -- start 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 -- start start 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af8101be0 0x7f0af8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 0x7f0af8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:33.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8103010 0x7f0af81a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0af810d5c0 con 0x7f0af81025a0 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0af810d440 con 0x7f0af8101be0 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0aff8c3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0af810d740 con 0x7f0af8103010 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 0x7f0af8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 0x7f0af8199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32920/0 (socket says 172.21.15.113:32920) 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 -- 172.21.15.113:0/4161238596 learned_addr learned my addr 172.21.15.113:0/4161238596 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afd65f700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af8101be0 0x7f0af8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afde60700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8103010 0x7f0af81a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 -- 172.21.15.113:0/4161238596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8103010 msgr2=0x7f0af81a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8103010 0x7f0af81a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 -- 172.21.15.113:0/4161238596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af8101be0 msgr2=0x7f0af8198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af8101be0 0x7f0af8198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 -- 172.21.15.113:0/4161238596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af81a0880 con 0x7f0af81025a0 2024-04-22T07:14:33.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afd65f700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af8101be0 0x7f0af8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afde60700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8103010 0x7f0af81a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.938+0000 7f0afce5e700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 0x7f0af8199170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0af400c9b0 tx=0x7f0af400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.939+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4007b00 con 0x7f0af81025a0 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.939+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af81a0b70 con 0x7f0af81025a0 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.939+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0af4007ca0 con 0x7f0af81025a0 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.939+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40187c0 con 0x7f0af81025a0 2024-04-22T07:14:33.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.939+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af81a10b0 con 0x7f0af81025a0 2024-04-22T07:14:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.940+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af8069730 con 0x7f0af81025a0 2024-04-22T07:14:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.941+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0af400f450 con 0x7f0af81025a0 2024-04-22T07:14:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.942+0000 7f0aee7fc700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0ae40a3c40 0x7f0ae40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.942+0000 7f0afd65f700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0ae40a3c40 0x7f0ae40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.942+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0af40d0780 con 0x7f0af81025a0 2024-04-22T07:14:33.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.942+0000 7f0afd65f700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0ae40a3c40 0x7f0ae40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8003910 tx=0x7f0ae800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:33.946+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0af4014030 con 0x7f0af81025a0 2024-04-22T07:14:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.187+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0af81a13f0 con 0x7f0af81025a0 2024-04-22T07:14:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.188+0000 7f0aee7fc700 1 -- 172.21.15.113:0/4161238596 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3052 (secure 0 0 0) 0x7f0af81a13f0 con 0x7f0af81025a0 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:e21 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:legacy client fscid: 1 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:Filesystem 'cephfs' (1) 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:fs_name cephfs 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:epoch 21 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:created 2024-04-22T07:12:54.514841+0000 2024-04-22T07:14:34.186 INFO:teuthology.orchestra.run.smithi113.stdout:modified 2024-04-22T07:12:58.522890+0000 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:tableserver 0 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:root 0 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:session_timeout 300 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:session_autoclose 300 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:max_file_size 1099511627776 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:max_xattr_size 65536 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:required_client_features {} 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:last_failure 0 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:last_failure_osd_epoch 0 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:max_mds 3 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:in 0,1,2 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:up {0=24506,1=24473,2=24469} 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:failed 2024-04-22T07:14:34.187 INFO:teuthology.orchestra.run.smithi113.stdout:damaged 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:stopped 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:data_pools [3] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:metadata_pool 2 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:inline_data disabled 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:balancer 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:bal_rank_mask -1 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:standby_count_wanted 0 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:qdb_cluster leader: 24506 members: 24469,24473,24506 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b{0:24506} state up:active seq 3 addr [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.e{0:24446} state up:standby-replay seq 1 addr [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.h{1:24473} state up:active seq 6 addr [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c{1:24445} state up:standby-replay seq 1 addr [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.i{2:24469} state up:active seq 6 addr [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.f{2:24433} state up:standby-replay seq 1 addr [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:34.188 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:Standby daemons: 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.113:6834/127167986,v1:172.21.15.113:6835/127167986] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.113:6836/918732402,v1:172.21.15.113:6837/918732402] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.g{-1:14625} state up:standby seq 1 addr [v2:172.21.15.113:6838/1495688995,v1:172.21.15.113:6839/1495688995] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a{-1:14637} state up:standby seq 1 addr [v2:172.21.15.113:6840/1868487444,v1:172.21.15.113:6841/1868487444] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.l{-1:24403} state up:standby seq 1 addr [v2:172.21.15.154:6832/819873552,v1:172.21.15.154:6833/819873552] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.133:6832/4012646459,v1:172.21.15.133:6833/4012646459] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0ae40a3c40 msgr2=0x7f0ae40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0ae40a3c40 0x7f0ae40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8003910 tx=0x7f0ae800b540 comp rx=0 tx=0).stop 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 msgr2=0x7f0af8199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 0x7f0af8199170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0af400c9b0 tx=0x7f0af400cd70 comp rx=0 tx=0).stop 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 shutdown_connections 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f0ae40a3c40 0x7f0ae40a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af8103010 0x7f0af81a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0af8101be0 0x7f0af8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 --2- 172.21.15.113:0/4161238596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0af81025a0 0x7f0af8199170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 >> 172.21.15.113:0/4161238596 conn(0x7f0af80fc6d0 msgr2=0x7f0af8100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 shutdown_connections 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:34.190+0000 7f0aff8c3700 1 -- 172.21.15.113:0/4161238596 wait complete. 2024-04-22T07:14:34.190 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:14:34.488 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:34 smithi113 ceph-mon[27933]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:34.488 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:34 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4161238596' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T07:14:34.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:34 smithi133 ceph-mon[31198]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:34.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:34 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4161238596' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T07:14:34.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:34 smithi154 ceph-mon[31862]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:34.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:34 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4161238596' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T07:14:34.898 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81035712-0076-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-22T07:14:35.138 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:36 smithi133 ceph-mon[31198]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:36.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:36 smithi154 ceph-mon[31862]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:36.568 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:36 smithi113 ceph-mon[27933]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:36.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 -- 172.21.15.113:0/3331776998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42581019b0 msgr2=0x7f4258101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:36.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 --2- 172.21.15.113:0/3331776998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42581019b0 0x7f4258101dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42480099c0 tx=0x7f4248009cd0 comp rx=0 tx=0).stop 2024-04-22T07:14:36.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 -- 172.21.15.113:0/3331776998 shutdown_connections 2024-04-22T07:14:36.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 --2- 172.21.15.113:0/3331776998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258103550 0x7f4258109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 --2- 172.21.15.113:0/3331776998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42581019b0 0x7f4258101dd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 --2- 172.21.15.113:0/3331776998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4258102bb0 0x7f4258103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 -- 172.21.15.113:0/3331776998 >> 172.21.15.113:0/3331776998 conn(0x7f42580fcf70 msgr2=0x7f42580ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 -- 172.21.15.113:0/3331776998 shutdown_connections 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.903+0000 7f425db3b700 1 -- 172.21.15.113:0/3331776998 wait complete. 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 Processor -- start 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 -- start start 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42581019b0 0x7f4258105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 0x7f4258103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258103550 0x7f42581044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:36.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f425810d660 con 0x7f4258102bb0 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f425810d4e0 con 0x7f4258103550 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f425db3b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f425810d7e0 con 0x7f42581019b0 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f4256ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 0x7f4258103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f4256ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 0x7f4258103f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34494/0 (socket says 172.21.15.113:34494) 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.904+0000 7f4256ffd700 1 -- 172.21.15.113:0/2074695607 learned_addr learned my addr 172.21.15.113:0/2074695607 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4257fff700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258103550 0x7f42581044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f42577fe700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42581019b0 0x7f4258105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4256ffd700 1 -- 172.21.15.113:0/2074695607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42581019b0 msgr2=0x7f4258105940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4256ffd700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42581019b0 0x7f4258105940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4256ffd700 1 -- 172.21.15.113:0/2074695607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258103550 msgr2=0x7f42581044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4256ffd700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258103550 0x7f42581044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:36.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4256ffd700 1 -- 172.21.15.113:0/2074695607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4258104d80 con 0x7f4258102bb0 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f42577fe700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42581019b0 0x7f4258105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4256ffd700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 0x7f4258103f90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f424c00b530 tx=0x7f424c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c015020 con 0x7f4258102bb0 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42581a2fa0 con 0x7f4258102bb0 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f424c011d00 con 0x7f4258102bb0 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c010e60 con 0x7f4258102bb0 2024-04-22T07:14:36.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.905+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42581a3480 con 0x7f4258102bb0 2024-04-22T07:14:36.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.906+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4258069730 con 0x7f4258102bb0 2024-04-22T07:14:36.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.907+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f424c018020 con 0x7f4258102bb0 2024-04-22T07:14:36.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.908+0000 7f4254ff9700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f42400a3c40 0x7f42400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:36.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.908+0000 7f42577fe700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f42400a3c40 0x7f42400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:36.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.908+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f424c0d0190 con 0x7f4258102bb0 2024-04-22T07:14:36.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.908+0000 7f42577fe700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f42400a3c40 0x7f42400a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4248003800 tx=0x7f42480038d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:36.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:36.911+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f424c01f020 con 0x7f4258102bb0 2024-04-22T07:14:37.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.110+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f42581a37c0 con 0x7f4258102bb0 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.111+0000 7f4254ff9700 1 -- 172.21.15.113:0/2074695607 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5307 (secure 0 0 0) 0x7f42581a37c0 con 0x7f4258102bb0 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:epoch 88 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:fsid 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:created 2024-04-22T07:04:58.384709+0000 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:modified 2024-04-22T07:13:56.180466+0000 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:crush_version 26 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:full_ratio 0.95 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:backfillfull_ratio 0.9 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:nearfull_ratio 0.85 2024-04-22T07:14:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:require_min_compat_client luminous 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:min_compat_client luminous 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:require_osd_release squid 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:stretch_mode_enabled false 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:max_osd 12 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6802/1926824887,v1:172.21.15.113:6803/1926824887] [v2:172.21.15.113:6804/1926824887,v1:172.21.15.113:6805/1926824887] exists,up 99b292ee-908e-46a7-8b87-733e426fe61b 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6800/283944628,v1:172.21.15.133:6801/283944628] [v2:172.21.15.133:6802/283944628,v1:172.21.15.133:6803/283944628] exists,up fed6e4a7-b083-483a-b8ec-2b567c9f6e92 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/343492984,v1:172.21.15.154:6801/343492984] [v2:172.21.15.154:6802/343492984,v1:172.21.15.154:6803/343492984] exists,up ed142201-490b-4bd4-8fe4-5aeaa334c18a 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6810/1273649373,v1:172.21.15.113:6811/1273649373] [v2:172.21.15.113:6812/1273649373,v1:172.21.15.113:6813/1273649373] exists,up 06e87876-83f8-46b3-8daf-3ebf810d8b99 2024-04-22T07:14:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6808/2027989696,v1:172.21.15.133:6809/2027989696] [v2:172.21.15.133:6810/2027989696,v1:172.21.15.133:6811/2027989696] exists,up 35753be4-5e96-47bf-8527-b7c4baeb873e 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/3828843548,v1:172.21.15.154:6809/3828843548] [v2:172.21.15.154:6810/3828843548,v1:172.21.15.154:6811/3828843548] exists,up ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6818/499388993,v1:172.21.15.113:6819/499388993] [v2:172.21.15.113:6820/499388993,v1:172.21.15.113:6821/499388993] exists,up 6196d5c6-a3a0-407c-af4a-3ca38ca997fc 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6816/3422116555,v1:172.21.15.133:6817/3422116555] [v2:172.21.15.133:6818/3422116555,v1:172.21.15.133:6819/3422116555] exists,up da26f96f-5fe9-4081-8152-7396392c2754 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/3748797627,v1:172.21.15.154:6817/3748797627] [v2:172.21.15.154:6818/3748797627,v1:172.21.15.154:6819/3748797627] exists,up c4f5d70d-b3b8-447c-8d18-60e6ac682dc4 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6826/744209950,v1:172.21.15.113:6827/744209950] [v2:172.21.15.113:6828/744209950,v1:172.21.15.113:6829/744209950] exists,up c9a6649c-b736-4f95-986a-2ff0463c0995 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6824/1753665247,v1:172.21.15.133:6825/1753665247] [v2:172.21.15.133:6826/1753665247,v1:172.21.15.133:6827/1753665247] exists,up 513726aa-29df-4ae7-bebc-db08c423daf4 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/3813220428,v1:172.21.15.154:6825/3813220428] [v2:172.21.15.154:6826/3813220428,v1:172.21.15.154:6827/3813220428] exists,up 0998e208-1905-409f-8e8d-7e8b7d1703dd 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 2.b [2,4] 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 2.13 [2,11] 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 2.1c [8,11] 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 2.1f [8,1] 2024-04-22T07:14:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 3.0 [4,8] 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 3.3 [4,9] 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 3.21 [5,9] 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:pg_upmap_items 3.27 [3,9] 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:6801/1807162111 expires 2024-04-23T07:05:54.991164+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:6800/1807162111 expires 2024-04-23T07:05:54.991164+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:0/1679614079 expires 2024-04-23T07:05:28.459936+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:0/4206830927 expires 2024-04-23T07:05:28.459936+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:0/3528402889 expires 2024-04-23T07:05:54.991164+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:0/2185314274 expires 2024-04-23T07:05:28.459936+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:6801/1303444723 expires 2024-04-23T07:05:28.459936+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:0/2266395984 expires 2024-04-23T07:05:54.991164+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:6800/1303444723 expires 2024-04-23T07:05:28.459936+0000 2024-04-22T07:14:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:blocklist 172.21.15.113:0/2368945695 expires 2024-04-23T07:05:54.991164+0000 2024-04-22T07:14:37.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f42400a3c40 msgr2=0x7f42400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:37.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f42400a3c40 0x7f42400a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4248003800 tx=0x7f42480038d0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 msgr2=0x7f4258103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:37.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 0x7f4258103f90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f424c00b530 tx=0x7f424c00b8f0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 shutdown_connections 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f42400a3c40 0x7f42400a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258103550 0x7f42581044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f42581019b0 0x7f4258105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 --2- 172.21.15.113:0/2074695607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4258102bb0 0x7f4258103f90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 >> 172.21.15.113:0/2074695607 conn(0x7f42580fcf70 msgr2=0x7f425806b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 shutdown_connections 2024-04-22T07:14:37.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.114+0000 7f425db3b700 1 -- 172.21.15.113:0/2074695607 wait complete. 2024-04-22T07:14:37.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:37 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2074695607' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T07:14:37.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:37 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2074695607' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T07:14:37.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:37 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2074695607' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T07:14:37.703 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-22T07:14:37.715 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:37.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 -- 172.21.15.113:0/557551471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 msgr2=0x7f83f41092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:37.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 --2- 172.21.15.113:0/557551471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f41092a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009ca0 tx=0x7f83ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 -- 172.21.15.113:0/557551471 shutdown_connections 2024-04-22T07:14:37.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 --2- 172.21.15.113:0/557551471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f41092a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 --2- 172.21.15.113:0/557551471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f4101fc0 0x7f83f4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 --2- 172.21.15.113:0/557551471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f4100e70 0x7f83f4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 -- 172.21.15.113:0/557551471 >> 172.21.15.113:0/557551471 conn(0x7f83f40fc3e0 msgr2=0x7f83f40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 -- 172.21.15.113:0/557551471 shutdown_connections 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.820+0000 7f83fa338640 1 -- 172.21.15.113:0/557551471 wait complete. 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 Processor -- start 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 -- start start 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f4100e70 0x7f83f4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f4101fc0 0x7f83f419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83f410cc00 con 0x7f83f4102980 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83f410ca80 con 0x7f83f4100e70 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83fa338640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83f410cd80 con 0x7f83f4101fc0 2024-04-22T07:14:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34504/0 (socket says 172.21.15.113:34504) 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 -- 172.21.15.113:0/2806321363 learned_addr learned my addr 172.21.15.113:0/2806321363 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f3fff640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f4100e70 0x7f83f4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 -- 172.21.15.113:0/2806321363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f4101fc0 msgr2=0x7f83f419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f4101fc0 0x7f83f419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 -- 172.21.15.113:0/2806321363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f4100e70 msgr2=0x7f83f4069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f4100e70 0x7f83f4069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 -- 172.21.15.113:0/2806321363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f41a0000 con 0x7f83f4102980 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.821+0000 7f83f88d5640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f419f6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005cc0 tx=0x7f83ec032a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.822+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec039040 con 0x7f83f4102980 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.822+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f83ec044070 con 0x7f83f4102980 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.822+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec004750 con 0x7f83f4102980 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.822+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f41a0230 con 0x7f83f4102980 2024-04-22T07:14:37.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.822+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f41a0770 con 0x7f83f4102980 2024-04-22T07:14:37.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.823+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f4108180 con 0x7f83f4102980 2024-04-22T07:14:37.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.824+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f83ec003970 con 0x7f83f4102980 2024-04-22T07:14:37.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.824+0000 7f83f17fa640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f83dc0a3bb0 0x7f83dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.824+0000 7f83f3fff640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f83dc0a3bb0 0x7f83dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.824+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f83ec0f2d70 con 0x7f83f4102980 2024-04-22T07:14:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.825+0000 7f83f3fff640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f83dc0a3bb0 0x7f83dc0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009990 tx=0x7f83e8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:37.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:37.829+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83ec0bf7f0 con 0x7f83f4102980 2024-04-22T07:14:38.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.008+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83f4062b50 con 0x7f83f4102980 2024-04-22T07:14:38.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.009+0000 7f83f17fa640 1 -- 172.21.15.113:0/2806321363 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22700 (secure 0 0 0) 0x7f83ec0bf0f0 con 0x7f83f4102980 2024-04-22T07:14:38.007 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:13:56.180466+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f83dc0a3bb0 msgr2=0x7f83dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f83dc0a3bb0 0x7f83dc0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009990 tx=0x7f83e8009110 comp rx=0 tx=0).stop 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 msgr2=0x7f83f419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f419f6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005cc0 tx=0x7f83ec032a90 comp rx=0 tx=0).stop 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 shutdown_connections 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f83dc0a3bb0 0x7f83dc0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83f4102980 0x7f83f419f6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f4101fc0 0x7f83f419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 --2- 172.21.15.113:0/2806321363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f4100e70 0x7f83f4069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 >> 172.21.15.113:0/2806321363 conn(0x7f83f40fc3e0 msgr2=0x7f83f4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 shutdown_connections 2024-04-22T07:14:38.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.011+0000 7f83fa338640 1 -- 172.21.15.113:0/2806321363 wait complete. 2024-04-22T07:14:38.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:38.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 -- 172.21.15.113:0/662591961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924105b10 msgr2=0x7ff924105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 --2- 172.21.15.113:0/662591961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924105b10 0x7ff924105f90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009ca0 tx=0x7ff91c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 -- 172.21.15.113:0/662591961 shutdown_connections 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 --2- 172.21.15.113:0/662591961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924105b10 0x7ff924105f90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 --2- 172.21.15.113:0/662591961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff924103160 0x7ff9241055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 --2- 172.21.15.113:0/662591961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9241007f0 0x7ff924102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 -- 172.21.15.113:0/662591961 >> 172.21.15.113:0/662591961 conn(0x7ff9240fc3f0 msgr2=0x7ff9240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 -- 172.21.15.113:0/662591961 shutdown_connections 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.124+0000 7ff92b806640 1 -- 172.21.15.113:0/662591961 wait complete. 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 Processor -- start 2024-04-22T07:14:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 -- start start 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9241007f0 0x7ff92419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 0x7ff92419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff924105b10 0x7ff9241a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241141f0 con 0x7ff924103160 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff924114070 con 0x7ff924105b10 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff92b806640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff924114370 con 0x7ff9241007f0 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 0x7ff92419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 0x7ff92419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34516/0 (socket says 172.21.15.113:34516) 2024-04-22T07:14:38.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 -- 172.21.15.113:0/570821342 learned_addr learned my addr 172.21.15.113:0/570821342 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff929da3640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff924105b10 0x7ff9241a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 -- 172.21.15.113:0/570821342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9241007f0 msgr2=0x7ff92419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9241007f0 0x7ff92419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 -- 172.21.15.113:0/570821342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff924105b10 msgr2=0x7ff9241a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff924105b10 0x7ff9241a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 -- 172.21.15.113:0/570821342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9241a64b0 con 0x7ff924103160 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.125+0000 7ff928da1640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 0x7ff92419ee20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff91400d950 tx=0x7ff91400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.126+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914017070 con 0x7ff924103160 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.126+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff914009c30 con 0x7ff924103160 2024-04-22T07:14:38.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.126+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9241a67a0 con 0x7ff924103160 2024-04-22T07:14:38.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.126+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91400f6b0 con 0x7ff924103160 2024-04-22T07:14:38.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.126+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241a6cf0 con 0x7ff924103160 2024-04-22T07:14:38.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.127+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9240fdee0 con 0x7ff924103160 2024-04-22T07:14:38.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.128+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7ff9140109c0 con 0x7ff924103160 2024-04-22T07:14:38.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.128+0000 7ff91a7fc640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9100a3bb0 0x7ff9100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.128+0000 7ff9295a2640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9100a3bb0 0x7ff9100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.128+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff9140d0020 con 0x7ff924103160 2024-04-22T07:14:38.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.129+0000 7ff9295a2640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9100a3bb0 0x7ff9100a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff920005fd0 tx=0x7ff920005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:38.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.133+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff914009de0 con 0x7ff924103160 2024-04-22T07:14:38.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.313+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff924062b80 con 0x7ff924103160 2024-04-22T07:14:38.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.313+0000 7ff91a7fc640 1 -- 172.21.15.113:0/570821342 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7ff91409ca10 con 0x7ff924103160 2024-04-22T07:14:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:38.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.315+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9100a3bb0 msgr2=0x7ff9100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.315+0000 7ff92b806640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9100a3bb0 0x7ff9100a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff920005fd0 tx=0x7ff920005850 comp rx=0 tx=0).stop 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.315+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 msgr2=0x7ff92419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.315+0000 7ff92b806640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 0x7ff92419ee20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff91400d950 tx=0x7ff91400de10 comp rx=0 tx=0).stop 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 shutdown_connections 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7ff9100a3bb0 0x7ff9100a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff924105b10 0x7ff9241a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff924103160 0x7ff92419ee20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 --2- 172.21.15.113:0/570821342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9241007f0 0x7ff92419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 >> 172.21.15.113:0/570821342 conn(0x7ff9240fc3f0 msgr2=0x7ff924103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 shutdown_connections 2024-04-22T07:14:38.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.316+0000 7ff92b806640 1 -- 172.21.15.113:0/570821342 wait complete. 2024-04-22T07:14:38.323 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:38 smithi113 ceph-mon[27933]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:38.323 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:38 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2806321363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:38.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:38.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 -- 172.21.15.113:0/3437207627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c101031a0 msgr2=0x7f3c10103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 --2- 172.21.15.113:0/3437207627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c101031a0 0x7f3c10103620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009900 tx=0x7f3c0402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:38.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 -- 172.21.15.113:0/3437207627 shutdown_connections 2024-04-22T07:14:38.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 --2- 172.21.15.113:0/3437207627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c10103b60 0x7f3c1010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 --2- 172.21.15.113:0/3437207627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c101031a0 0x7f3c10103620 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 --2- 172.21.15.113:0/3437207627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10101fb0 0x7f3c101023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.426+0000 7f3c165ef640 1 -- 172.21.15.113:0/3437207627 >> 172.21.15.113:0/3437207627 conn(0x7f3c100fd520 msgr2=0x7f3c100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 -- 172.21.15.113:0/3437207627 shutdown_connections 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 -- 172.21.15.113:0/3437207627 wait complete. 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 Processor -- start 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 -- start start 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 0x7f3c1019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c101031a0 0x7f3c1019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10103b60 0x7f3c101a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1010dc10 con 0x7f3c10101fb0 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1010da90 con 0x7f3c10103b60 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c165ef640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1010dd90 con 0x7f3c101031a0 2024-04-22T07:14:38.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 0x7f3c1019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.427+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 0x7f3c1019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34530/0 (socket says 172.21.15.113:34530) 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 -- 172.21.15.113:0/1016921816 learned_addr learned my addr 172.21.15.113:0/1016921816 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c14b8c640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10103b60 0x7f3c101a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0f7fe640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c101031a0 0x7f3c1019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 -- 172.21.15.113:0/1016921816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c101031a0 msgr2=0x7f3c1019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c101031a0 0x7f3c1019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 -- 172.21.15.113:0/1016921816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10103b60 msgr2=0x7f3c101a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10103b60 0x7f3c101a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 -- 172.21.15.113:0/1016921816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c101a2090 con 0x7f3c10101fb0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0ffff640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 0x7f3c1019a510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d770 tx=0x7f3c0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00017070 con 0x7f3c10101fb0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3c00009c30 con 0x7f3c10101fb0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c101a2380 con 0x7f3c10101fb0 2024-04-22T07:14:38.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00003d70 con 0x7f3c10101fb0 2024-04-22T07:14:38.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.428+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a28a0 con 0x7f3c10101fb0 2024-04-22T07:14:38.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.429+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c101a2b00 con 0x7f3c10101fb0 2024-04-22T07:14:38.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.430+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3c00003f10 con 0x7f3c10101fb0 2024-04-22T07:14:38.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.431+0000 7f3c0d7fa640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3bf80a3bb0 0x7f3bf80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.431+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3c000cfcb0 con 0x7f3c10101fb0 2024-04-22T07:14:38.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.431+0000 7f3c0f7fe640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3bf80a3bb0 0x7f3bf80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.431+0000 7f3c0f7fe640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3bf80a3bb0 0x7f3bf80a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c04006010 tx=0x7f3c04005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:38.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.435+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c00009de0 con 0x7f3c10101fb0 2024-04-22T07:14:38.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:38 smithi133 ceph-mon[31198]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:38.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:38 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2806321363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:38.467 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:38 smithi154 ceph-mon[31862]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-22T07:14:38.467 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:38 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2806321363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:38.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.612+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c101a3330 con 0x7f3c10101fb0 2024-04-22T07:14:38.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.613+0000 7f3c0d7fa640 1 -- 172.21.15.113:0/1016921816 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f3c0009c050 con 0x7f3c10101fb0 2024-04-22T07:14:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:38.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.614+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3bf80a3bb0 msgr2=0x7f3bf80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3bf80a3bb0 0x7f3bf80a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c04006010 tx=0x7f3c04005b50 comp rx=0 tx=0).stop 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 msgr2=0x7f3c1019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 0x7f3c1019a510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000d770 tx=0x7f3c0000dc30 comp rx=0 tx=0).stop 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 shutdown_connections 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f3bf80a3bb0 0x7f3bf80a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10103b60 0x7f3c101a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c101031a0 0x7f3c1019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 --2- 172.21.15.113:0/1016921816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c10101fb0 0x7f3c1019a510 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 >> 172.21.15.113:0/1016921816 conn(0x7f3c100fd520 msgr2=0x7f3c10074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 shutdown_connections 2024-04-22T07:14:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.615+0000 7f3c165ef640 1 -- 172.21.15.113:0/1016921816 wait complete. 2024-04-22T07:14:38.624 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:38.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 -- 172.21.15.113:0/1559838778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 msgr2=0x7fcfac103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1559838778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac103610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009900 tx=0x7fcfa002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:38.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 -- 172.21.15.113:0/1559838778 shutdown_connections 2024-04-22T07:14:38.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1559838778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfac103b50 0x7fcfac10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1559838778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac103610 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1559838778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 0x7fcfac1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 -- 172.21.15.113:0/1559838778 >> 172.21.15.113:0/1559838778 conn(0x7fcfac0fd530 msgr2=0x7fcfac0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 -- 172.21.15.113:0/1559838778 shutdown_connections 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.726+0000 7fcfb233d640 1 -- 172.21.15.113:0/1559838778 wait complete. 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 Processor -- start 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 -- start start 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 0x7fcfac198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfac103b50 0x7fcfac19f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac10dd00 con 0x7fcfac103190 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac10db80 con 0x7fcfac101fa0 2024-04-22T07:14:38.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb233d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac10de80 con 0x7fcfac103b50 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfab7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfab7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34546/0 (socket says 172.21.15.113:34546) 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfab7fe640 1 -- 172.21.15.113:0/1813598046 learned_addr learned my addr 172.21.15.113:0/1813598046 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfb08da640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfac103b50 0x7fcfac19f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.727+0000 7fcfabfff640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 0x7fcfac198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfab7fe640 1 -- 172.21.15.113:0/1813598046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfac103b50 msgr2=0x7fcfac19f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfab7fe640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfac103b50 0x7fcfac19f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfab7fe640 1 -- 172.21.15.113:0/1813598046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 msgr2=0x7fcfac198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfab7fe640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 0x7fcfac198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfab7fe640 1 -- 172.21.15.113:0/1813598046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac1a0050 con 0x7fcfac103190 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfabfff640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 0x7fcfac198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfab7fe640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac198970 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0036040 tx=0x7fcfa0004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:38.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0041070 con 0x7fcfac103190 2024-04-22T07:14:38.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcfa0004aa0 con 0x7fcfac103190 2024-04-22T07:14:38.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac1a02e0 con 0x7fcfac103190 2024-04-22T07:14:38.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0034440 con 0x7fcfac103190 2024-04-22T07:14:38.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.728+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac1a08b0 con 0x7fcfac103190 2024-04-22T07:14:38.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.729+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac109390 con 0x7fcfac103190 2024-04-22T07:14:38.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.730+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fcfa0044070 con 0x7fcfac103190 2024-04-22T07:14:38.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.730+0000 7fcfa97fa640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcf940a3bb0 0x7fcf940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:38.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.731+0000 7fcfabfff640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcf940a3bb0 0x7fcf940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:38.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.731+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcfa00f2580 con 0x7fcfac103190 2024-04-22T07:14:38.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.731+0000 7fcfabfff640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcf940a3bb0 0x7fcf940a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c006fd0 tx=0x7fcf9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:38.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.734+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfa00bef70 con 0x7fcfac103190 2024-04-22T07:14:38.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.910+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcfac02d810 con 0x7fcfac103190 2024-04-22T07:14:38.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.911+0000 7fcfa97fa640 1 -- 172.21.15.113:0/1813598046 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fcfa00be870 con 0x7fcfac103190 2024-04-22T07:14:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcf940a3bb0 msgr2=0x7fcf940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcf940a3bb0 0x7fcf940a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c006fd0 tx=0x7fcf9c008040 comp rx=0 tx=0).stop 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 msgr2=0x7fcfac198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac198970 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0036040 tx=0x7fcfa0004370 comp rx=0 tx=0).stop 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 shutdown_connections 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fcf940a3bb0 0x7fcf940a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfac103b50 0x7fcfac19f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfac103190 0x7fcfac198970 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 --2- 172.21.15.113:0/1813598046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac101fa0 0x7fcfac198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:38.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 >> 172.21.15.113:0/1813598046 conn(0x7fcfac0fd530 msgr2=0x7fcfac106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:38.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 shutdown_connections 2024-04-22T07:14:38.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:38.913+0000 7fcfb233d640 1 -- 172.21.15.113:0/1813598046 wait complete. 2024-04-22T07:14:38.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:39.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 -- 172.21.15.113:0/1965470604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484100e70 msgr2=0x7f2484101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 --2- 172.21.15.113:0/1965470604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484100e70 0x7f2484101290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2474009900 tx=0x7f247402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 -- 172.21.15.113:0/1965470604 shutdown_connections 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 --2- 172.21.15.113:0/1965470604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2484102980 0x7f24841092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 --2- 172.21.15.113:0/1965470604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2484101fc0 0x7f2484102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 --2- 172.21.15.113:0/1965470604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484100e70 0x7f2484101290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 -- 172.21.15.113:0/1965470604 >> 172.21.15.113:0/1965470604 conn(0x7f24840fc3e0 msgr2=0x7f24840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 -- 172.21.15.113:0/1965470604 shutdown_connections 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 -- 172.21.15.113:0/1965470604 wait complete. 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 Processor -- start 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.022+0000 7f2489ace640 1 -- start start 2024-04-22T07:14:39.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2484100e70 0x7f24841982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2484101fc0 0x7f24841987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 0x7f248419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f248410dc20 con 0x7f2484102980 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f248410daa0 con 0x7f2484100e70 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f248410dda0 con 0x7f2484101fc0 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 0x7f248419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 0x7f248419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34560/0 (socket says 172.21.15.113:34560) 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 -- 172.21.15.113:0/2530888881 learned_addr learned my addr 172.21.15.113:0/2530888881 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f24837fe640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2484100e70 0x7f24841982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f247bfff640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2484101fc0 0x7f24841987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 -- 172.21.15.113:0/2530888881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2484101fc0 msgr2=0x7f24841987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2484101fc0 0x7f24841987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 -- 172.21.15.113:0/2530888881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2484100e70 msgr2=0x7f24841982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2484100e70 0x7f24841982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 -- 172.21.15.113:0/2530888881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248419fec0 con 0x7f2484102980 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2483fff640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 0x7f248419f850 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f247c00eaa0 tx=0x7f247c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00cdb0 con 0x7f2484102980 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24841a01b0 con 0x7f2484102980 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f247c005020 con 0x7f2484102980 2024-04-22T07:14:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0116b0 con 0x7f2484102980 2024-04-22T07:14:39.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.023+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24841a0700 con 0x7f2484102980 2024-04-22T07:14:39.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.024+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2484108180 con 0x7f2484102980 2024-04-22T07:14:39.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.025+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f247c0051d0 con 0x7f2484102980 2024-04-22T07:14:39.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.026+0000 7f24817fa640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24700a3bb0 0x7f24700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.026+0000 7f24837fe640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24700a3bb0 0x7f24700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.026+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f247c0d0690 con 0x7f2484102980 2024-04-22T07:14:39.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.027+0000 7f24837fe640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24700a3bb0 0x7f24700a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f247402c8b0 tx=0x7f2474000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.031+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f247c016030 con 0x7f2484102980 2024-04-22T07:14:39.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.252+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2484062b50 con 0x7f2484102980 2024-04-22T07:14:39.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.253+0000 7f24817fa640 1 -- 172.21.15.113:0/2530888881 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11875 (secure 0 0 0) 0x7f247c09d080 con 0x7f2484102980 2024-04-22T07:14:39.251 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:12:58.522890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24473,24506]},"id":1}]} 2024-04-22T07:14:39.251 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24700a3bb0 msgr2=0x7f24700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24700a3bb0 0x7f24700a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f247402c8b0 tx=0x7f2474000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 msgr2=0x7f248419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 0x7f248419f850 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f247c00eaa0 tx=0x7f247c00ef60 comp rx=0 tx=0).stop 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 shutdown_connections 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f24700a3bb0 0x7f24700a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2484102980 0x7f248419f850 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2484101fc0 0x7f24841987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 --2- 172.21.15.113:0/2530888881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2484100e70 0x7f24841982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 >> 172.21.15.113:0/2530888881 conn(0x7f24840fc3e0 msgr2=0x7f2484105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 shutdown_connections 2024-04-22T07:14:39.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.255+0000 7f2489ace640 1 -- 172.21.15.113:0/2530888881 wait complete. 2024-04-22T07:14:39.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 -- 172.21.15.113:0/2881682416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828105370 msgr2=0x7f9828107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 --2- 172.21.15.113:0/2881682416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828105370 0x7f9828107760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9810009900 tx=0x7f981002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 -- 172.21.15.113:0/2881682416 shutdown_connections 2024-04-22T07:14:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 --2- 172.21.15.113:0/2881682416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9828107ca0 0x7f982810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 --2- 172.21.15.113:0/2881682416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828105370 0x7f9828107760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 --2- 172.21.15.113:0/2881682416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9828102a40 0x7f9828104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 -- 172.21.15.113:0/2881682416 >> 172.21.15.113:0/2881682416 conn(0x7f98280fc470 msgr2=0x7f98280fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 -- 172.21.15.113:0/2881682416 shutdown_connections 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.365+0000 7f982cfbd640 1 -- 172.21.15.113:0/2881682416 wait complete. 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 Processor -- start 2024-04-22T07:14:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 -- start start 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 0x7f9828198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9828105370 0x7f9828198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9828107ca0 0x7f982819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f982810dc80 con 0x7f9828102a40 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f982810db00 con 0x7f9828105370 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982cfbd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f982810de00 con 0x7f9828107ca0 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 0x7f9828198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 0x7f9828198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34572/0 (socket says 172.21.15.113:34572) 2024-04-22T07:14:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 -- 172.21.15.113:0/774380882 learned_addr learned my addr 172.21.15.113:0/774380882 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f9826d9d640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9828107ca0 0x7f982819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f981dd9b640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9828105370 0x7f9828198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 -- 172.21.15.113:0/774380882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9828107ca0 msgr2=0x7f982819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9828107ca0 0x7f982819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 -- 172.21.15.113:0/774380882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9828105370 msgr2=0x7f9828198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9828105370 0x7f9828198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.366+0000 7f982659c640 1 -- 172.21.15.113:0/774380882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982819fef0 con 0x7f9828102a40 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.367+0000 7f982659c640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 0x7f9828198320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f981800c890 tx=0x7f981800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.367+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818004220 con 0x7f9828102a40 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.367+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98281a01e0 con 0x7f9828102a40 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.367+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98180043c0 con 0x7f9828102a40 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.367+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98180049e0 con 0x7f9828102a40 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.367+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98281a0780 con 0x7f9828102a40 2024-04-22T07:14:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.368+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98280fdf60 con 0x7f9828102a40 2024-04-22T07:14:39.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.369+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9818005140 con 0x7f9828102a40 2024-04-22T07:14:39.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.369+0000 7f981ffff640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f98140a3bb0 0x7f98140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.370+0000 7f981dd9b640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f98140a3bb0 0x7f98140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.370+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f98180d0150 con 0x7f9828102a40 2024-04-22T07:14:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.370+0000 7f981dd9b640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f98140a3bb0 0x7f98140a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9810006010 tx=0x7f9810000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.374+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9818014030 con 0x7f9828102a40 2024-04-22T07:14:39.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.553+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9828062b50 con 0x7f9828102a40 2024-04-22T07:14:39.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.555+0000 7f981ffff640 1 -- 172.21.15.113:0/774380882 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22700 (secure 0 0 0) 0x7f981809cb40 con 0x7f9828102a40 2024-04-22T07:14:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:13:56.180466+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f98140a3bb0 msgr2=0x7f98140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f98140a3bb0 0x7f98140a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9810006010 tx=0x7f9810000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 msgr2=0x7f9828198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 0x7f9828198320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f981800c890 tx=0x7f981800cd50 comp rx=0 tx=0).stop 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 shutdown_connections 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f98140a3bb0 0x7f98140a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9828107ca0 0x7f982819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9828105370 0x7f9828198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 --2- 172.21.15.113:0/774380882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9828102a40 0x7f9828198320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 >> 172.21.15.113:0/774380882 conn(0x7f98280fc470 msgr2=0x7f9828103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 shutdown_connections 2024-04-22T07:14:39.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.557+0000 7f982cfbd640 1 -- 172.21.15.113:0/774380882 wait complete. 2024-04-22T07:14:39.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:39 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/570821342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:39 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1016921816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:39 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1813598046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.564 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:39 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/570821342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.565 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:39 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1016921816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.565 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:39 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1813598046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:39.566 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:39.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:39 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/570821342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:39 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1016921816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:39 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1813598046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:39.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1029973067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8102a30 msgr2=0x7f5de8104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1029973067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8102a30 0x7f5de8104e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009900 tx=0x7f5ddc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:39.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1029973067 shutdown_connections 2024-04-22T07:14:39.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1029973067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de8107c90 0x7f5de810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1029973067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de8105360 0x7f5de8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1029973067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8102a30 0x7f5de8104e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1029973067 >> 172.21.15.113:0/1029973067 conn(0x7f5de80fc440 msgr2=0x7f5de80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1029973067 shutdown_connections 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.669+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1029973067 wait complete. 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 Processor -- start 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 -- start start 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de8102a30 0x7f5de8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 0x7f5de81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de8107c90 0x7f5de819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5de810dc10 con 0x7f5de8105360 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5de810da90 con 0x7f5de8102a30 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5dee7a4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5de810dd90 con 0x7f5de8107c90 2024-04-22T07:14:39.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 0x7f5de81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 0x7f5de81987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34598/0 (socket says 172.21.15.113:34598) 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 -- 172.21.15.113:0/1259903594 learned_addr learned my addr 172.21.15.113:0/1259903594 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de7fff640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de8102a30 0x7f5de8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5decd41640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de8107c90 0x7f5de819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 -- 172.21.15.113:0/1259903594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de8107c90 msgr2=0x7f5de819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de8107c90 0x7f5de819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 -- 172.21.15.113:0/1259903594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de8102a30 msgr2=0x7f5de8198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de8102a30 0x7f5de8198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.670+0000 7f5de77fe640 1 -- 172.21.15.113:0/1259903594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de819feb0 con 0x7f5de8105360 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.671+0000 7f5de77fe640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 0x7f5de81987d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c970 tx=0x7f5dd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.671+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8014070 con 0x7f5de8105360 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.671+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de81a01a0 con 0x7f5de8105360 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.671+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5dd8004440 con 0x7f5de8105360 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.671+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003e40 con 0x7f5de8105360 2024-04-22T07:14:39.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.671+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de81a06f0 con 0x7f5de8105360 2024-04-22T07:14:39.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.672+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de80fdf30 con 0x7f5de8105360 2024-04-22T07:14:39.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.673+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f5dd8017070 con 0x7f5de8105360 2024-04-22T07:14:39.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.673+0000 7f5de57fa640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dd00a3bb0 0x7f5dd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.673+0000 7f5de7fff640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dd00a3bb0 0x7f5dd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.673+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5dd80d0250 con 0x7f5de8105360 2024-04-22T07:14:39.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.674+0000 7f5de7fff640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dd00a3bb0 0x7f5dd00a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006010 tx=0x7f5ddc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.678+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dd801e020 con 0x7f5de8105360 2024-04-22T07:14:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.853+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5de8062b50 con 0x7f5de8105360 2024-04-22T07:14:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.854+0000 7f5de57fa640 1 -- 172.21.15.113:0/1259903594 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f5dd809cc40 con 0x7f5de8105360 2024-04-22T07:14:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dd00a3bb0 msgr2=0x7f5dd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dd00a3bb0 0x7f5dd00a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006010 tx=0x7f5ddc005a40 comp rx=0 tx=0).stop 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 msgr2=0x7f5de81987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 0x7f5de81987d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c970 tx=0x7f5dd800ce30 comp rx=0 tx=0).stop 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 shutdown_connections 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f5dd00a3bb0 0x7f5dd00a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5de8107c90 0x7f5de819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5de8105360 0x7f5de81987d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 --2- 172.21.15.113:0/1259903594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5de8102a30 0x7f5de8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 >> 172.21.15.113:0/1259903594 conn(0x7f5de80fc440 msgr2=0x7f5de81032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 shutdown_connections 2024-04-22T07:14:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.856+0000 7f5dee7a4640 1 -- 172.21.15.113:0/1259903594 wait complete. 2024-04-22T07:14:39.865 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 -- 172.21.15.113:0/4049170079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c101fa0 msgr2=0x7f610c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 --2- 172.21.15.113:0/4049170079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c101fa0 0x7f610c1023c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6108009900 tx=0x7f610802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 -- 172.21.15.113:0/4049170079 shutdown_connections 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 --2- 172.21.15.113:0/4049170079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 0x7f610c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 --2- 172.21.15.113:0/4049170079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f610c103190 0x7f610c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 --2- 172.21.15.113:0/4049170079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c101fa0 0x7f610c1023c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 -- 172.21.15.113:0/4049170079 >> 172.21.15.113:0/4049170079 conn(0x7f610c0fd4f0 msgr2=0x7f610c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 -- 172.21.15.113:0/4049170079 shutdown_connections 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 -- 172.21.15.113:0/4049170079 wait complete. 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 Processor -- start 2024-04-22T07:14:39.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.968+0000 7f61132eb640 1 -- start start 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f610c101fa0 0x7f610c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 0x7f610c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 0x7f610c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f610c10dc50 con 0x7f610c103190 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f610c10dad0 con 0x7f610c103b50 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f610c10ddd0 con 0x7f610c101fa0 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 0x7f610c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 0x7f610c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34606/0 (socket says 172.21.15.113:34606) 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 -- 172.21.15.113:0/4166988849 learned_addr learned my addr 172.21.15.113:0/4166988849 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6111888640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 0x7f610c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6111087640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f610c101fa0 0x7f610c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 -- 172.21.15.113:0/4166988849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f610c101fa0 msgr2=0x7f610c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f610c101fa0 0x7f610c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 -- 172.21.15.113:0/4166988849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 msgr2=0x7f610c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 0x7f610c19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 -- 172.21.15.113:0/4166988849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c19ff10 con 0x7f610c103190 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6111888640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 0x7f610c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f6110886640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 0x7f610c198830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00d770 tx=0x7f60fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc017070 con 0x7f610c103190 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f60fc0054f0 con 0x7f610c103190 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c1a0200 con 0x7f610c103190 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc004de0 con 0x7f610c103190 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.969+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1a0700 con 0x7f610c103190 2024-04-22T07:14:39.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.970+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c1093e0 con 0x7f610c103190 2024-04-22T07:14:39.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.971+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f60fc004f80 con 0x7f610c103190 2024-04-22T07:14:39.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.972+0000 7f61027fc640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f60f80a3bb0 0x7f60f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:39.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.972+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f60fc0d0390 con 0x7f610c103190 2024-04-22T07:14:39.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.972+0000 7f6111087640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f60f80a3bb0 0x7f60f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:39.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.972+0000 7f6111087640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f60f80a3bb0 0x7f60f80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f610802c240 tx=0x7f6108005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:39.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:39.976+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60fc09cd80 con 0x7f610c103190 2024-04-22T07:14:40.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.154+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f610c02d810 con 0x7f610c103190 2024-04-22T07:14:40.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.154+0000 7f61027fc640 1 -- 172.21.15.113:0/4166988849 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f60fc09c680 con 0x7f610c103190 2024-04-22T07:14:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:40.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f60f80a3bb0 msgr2=0x7f60f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f60f80a3bb0 0x7f60f80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f610802c240 tx=0x7f6108005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 msgr2=0x7f610c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 0x7f610c198830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00d770 tx=0x7f60fc00dc30 comp rx=0 tx=0).stop 2024-04-22T07:14:40.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 shutdown_connections 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f60f80a3bb0 0x7f60f80a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f610c103b50 0x7f610c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f610c103190 0x7f610c198830 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 --2- 172.21.15.113:0/4166988849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f610c101fa0 0x7f610c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 >> 172.21.15.113:0/4166988849 conn(0x7f610c0fd4f0 msgr2=0x7f610c1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 shutdown_connections 2024-04-22T07:14:40.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.156+0000 7f61132eb640 1 -- 172.21.15.113:0/4166988849 wait complete. 2024-04-22T07:14:40.165 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 -- 172.21.15.113:0/1799447940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20100620 msgr2=0x7f1c20100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1799447940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20100620 0x7f1c20100a40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009900 tx=0x7f1c1802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 -- 172.21.15.113:0/1799447940 shutdown_connections 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1799447940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c201021d0 0x7f1c20108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1799447940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 0x7f1c20101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1799447940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20100620 0x7f1c20100a40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 -- 172.21.15.113:0/1799447940 >> 172.21.15.113:0/1799447940 conn(0x7f1c200fbb90 msgr2=0x7f1c200fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 -- 172.21.15.113:0/1799447940 shutdown_connections 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 -- 172.21.15.113:0/1799447940 wait complete. 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 Processor -- start 2024-04-22T07:14:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.270+0000 7f1c249b2640 1 -- start start 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c20100620 0x7f1c20197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 0x7f1c201977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 0x7f1c2019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010cb80 con 0x7f1c201021d0 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010ca00 con 0x7f1c20101810 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010cd00 con 0x7f1c20100620 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 0x7f1c2019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 0x7f1c2019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34626/0 (socket says 172.21.15.113:34626) 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 -- 172.21.15.113:0/1977479401 learned_addr learned my addr 172.21.15.113:0/1977479401 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1e59c640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c20100620 0x7f1c20197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1dd9b640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 0x7f1c201977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 -- 172.21.15.113:0/1977479401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c20100620 msgr2=0x7f1c20197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c20100620 0x7f1c20197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 -- 172.21.15.113:0/1977479401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 msgr2=0x7f1c201977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 0x7f1c201977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 -- 172.21.15.113:0/1977479401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2019ee90 con 0x7f1c201021d0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1e59c640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c20100620 0x7f1c20197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1dd9b640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 0x7f1c201977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c1ed9d640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 0x7f1c2019e820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000d750 tx=0x7f1c1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10017070 con 0x7f1c201021d0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2019f180 con 0x7f1c201021d0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1c100054f0 con 0x7f1c201021d0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10004cd0 con 0x7f1c201021d0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.271+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2019f750 con 0x7f1c201021d0 2024-04-22T07:14:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.272+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c20107a10 con 0x7f1c201021d0 2024-04-22T07:14:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.273+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f1c10009bd0 con 0x7f1c201021d0 2024-04-22T07:14:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.274+0000 7f1c0f7fe640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1c080a3bb0 0x7f1c080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.274+0000 7f1c1e59c640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1c080a3bb0 0x7f1c080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.274+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f1c100d0310 con 0x7f1c201021d0 2024-04-22T07:14:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.274+0000 7f1c1e59c640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1c080a3bb0 0x7f1c080a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006010 tx=0x7f1c1802c950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:40.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.278+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c1009cd00 con 0x7f1c201021d0 2024-04-22T07:14:40.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.454+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c2002d5c0 con 0x7f1c201021d0 2024-04-22T07:14:40.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.454+0000 7f1c0f7fe640 1 -- 172.21.15.113:0/1977479401 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1c1009c600 con 0x7f1c201021d0 2024-04-22T07:14:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1c080a3bb0 msgr2=0x7f1c080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1c080a3bb0 0x7f1c080a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1c18006010 tx=0x7f1c1802c950 comp rx=0 tx=0).stop 2024-04-22T07:14:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 msgr2=0x7f1c2019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 0x7f1c2019e820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000d750 tx=0x7f1c1000dc10 comp rx=0 tx=0).stop 2024-04-22T07:14:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 shutdown_connections 2024-04-22T07:14:40.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f1c080a3bb0 0x7f1c080a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c201021d0 0x7f1c2019e820 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c20101810 0x7f1c201977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 --2- 172.21.15.113:0/1977479401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c20100620 0x7f1c20197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 >> 172.21.15.113:0/1977479401 conn(0x7f1c200fbb90 msgr2=0x7f1c20104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:40.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 shutdown_connections 2024-04-22T07:14:40.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.456+0000 7f1c249b2640 1 -- 172.21.15.113:0/1977479401 wait complete. 2024-04-22T07:14:40.464 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:40 smithi113 ceph-mon[27933]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:40.464 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:40 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2530888881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:40.464 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:40 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/774380882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:40.464 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:40 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1259903594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:40.464 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:40 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4166988849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:40.465 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:40.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 -- 172.21.15.113:0/2175855998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340105380 msgr2=0x7f9340107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 --2- 172.21.15.113:0/2175855998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340105380 0x7f9340107770 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9328009900 tx=0x7f932802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 -- 172.21.15.113:0/2175855998 shutdown_connections 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 --2- 172.21.15.113:0/2175855998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 0x7f934010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 --2- 172.21.15.113:0/2175855998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340105380 0x7f9340107770 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 --2- 172.21.15.113:0/2175855998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340102a50 0x7f9340104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 -- 172.21.15.113:0/2175855998 >> 172.21.15.113:0/2175855998 conn(0x7f93400fc3e0 msgr2=0x7f93400fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 -- 172.21.15.113:0/2175855998 shutdown_connections 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.565+0000 7f9346e81640 1 -- 172.21.15.113:0/2175855998 wait complete. 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 Processor -- start 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 -- start start 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 0x7f934019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340105380 0x7f934019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 0x7f93401a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f934010dcb0 con 0x7f9340102a50 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f934010db30 con 0x7f9340107cb0 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9346e81640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f934010de30 con 0x7f9340105380 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9344c1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 0x7f934019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9344c1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 0x7f934019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34642/0 (socket says 172.21.15.113:34642) 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9344c1d640 1 -- 172.21.15.113:0/4048336625 learned_addr learned my addr 172.21.15.113:0/4048336625 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f933ffff640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340105380 0x7f934019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f934541e640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 0x7f93401a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.566+0000 7f9344c1d640 1 -- 172.21.15.113:0/4048336625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340105380 msgr2=0x7f934019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9344c1d640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340105380 0x7f934019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9344c1d640 1 -- 172.21.15.113:0/4048336625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 msgr2=0x7f93401a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9344c1d640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 0x7f93401a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9344c1d640 1 -- 172.21.15.113:0/4048336625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93401a2180 con 0x7f9340102a50 2024-04-22T07:14:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f933ffff640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340105380 0x7f934019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f934541e640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 0x7f93401a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9344c1d640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 0x7f934019a560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f933000b4a0 tx=0x7f933000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330014070 con 0x7f9340102a50 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9330010ae0 con 0x7f9340102a50 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93401a2470 con 0x7f9340102a50 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330017ce0 con 0x7f9340102a50 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.567+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93401a2a10 con 0x7f9340102a50 2024-04-22T07:14:40.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.568+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93401097a0 con 0x7f9340102a50 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:40 smithi154 ceph-mon[31862]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:40 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2530888881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:40 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/774380882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:40 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1259903594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:40 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4166988849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:40 smithi133 ceph-mon[31198]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:40 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2530888881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:40.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:40 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/774380882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:40.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:40 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1259903594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:40.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:40 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4166988849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:40.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.569+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9330017070 con 0x7f9340102a50 2024-04-22T07:14:40.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.569+0000 7f933dffb640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f932c0a3bb0 0x7f932c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.569+0000 7f933ffff640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f932c0a3bb0 0x7f932c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.570+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f93300cfb80 con 0x7f9340102a50 2024-04-22T07:14:40.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.570+0000 7f933ffff640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f932c0a3bb0 0x7f932c0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f932802c240 tx=0x7f9328005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:40.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.574+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f933009c570 con 0x7f9340102a50 2024-04-22T07:14:40.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.788+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9340062b50 con 0x7f9340102a50 2024-04-22T07:14:40.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.789+0000 7f933dffb640 1 -- 172.21.15.113:0/4048336625 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11875 (secure 0 0 0) 0x7f933009be70 con 0x7f9340102a50 2024-04-22T07:14:40.787 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:12:58.522890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24469,24473,24506]},"id":1}]} 2024-04-22T07:14:40.787 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f932c0a3bb0 msgr2=0x7f932c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f932c0a3bb0 0x7f932c0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f932802c240 tx=0x7f9328005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 msgr2=0x7f934019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 0x7f934019a560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f933000b4a0 tx=0x7f933000b960 comp rx=0 tx=0).stop 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 shutdown_connections 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f932c0a3bb0 0x7f932c0a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9340107cb0 0x7f93401a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9340105380 0x7f934019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 --2- 172.21.15.113:0/4048336625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9340102a50 0x7f934019a560 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 >> 172.21.15.113:0/4048336625 conn(0x7f93400fc3e0 msgr2=0x7f9340102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 shutdown_connections 2024-04-22T07:14:40.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.791+0000 7f9346e81640 1 -- 172.21.15.113:0/4048336625 wait complete. 2024-04-22T07:14:40.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.900+0000 7fbda0132640 1 -- 172.21.15.113:0/1199930223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98101fc0 msgr2=0x7fbd98102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.900+0000 7fbda0132640 1 --2- 172.21.15.113:0/1199930223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98101fc0 0x7fbd98102440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009900 tx=0x7fbd8802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- 172.21.15.113:0/1199930223 shutdown_connections 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 --2- 172.21.15.113:0/1199930223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd98102980 0x7fbd981092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 --2- 172.21.15.113:0/1199930223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98101fc0 0x7fbd98102440 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 --2- 172.21.15.113:0/1199930223 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd98100e70 0x7fbd98101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- 172.21.15.113:0/1199930223 >> 172.21.15.113:0/1199930223 conn(0x7fbd980fc3e0 msgr2=0x7fbd980fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- 172.21.15.113:0/1199930223 shutdown_connections 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- 172.21.15.113:0/1199930223 wait complete. 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 Processor -- start 2024-04-22T07:14:40.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- start start 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 0x7fbd98069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd98101fc0 0x7fbd9819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd98102980 0x7fbd9819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810caf0 con 0x7fbd98100e70 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810c970 con 0x7fbd98102980 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbda0132640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9810cc70 con 0x7fbd98101fc0 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbd9dece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 0x7fbd98069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbd9dece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 0x7fbd98069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34660/0 (socket says 172.21.15.113:34660) 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbd9dece640 1 -- 172.21.15.113:0/3065303640 learned_addr learned my addr 172.21.15.113:0/3065303640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.901+0000 7fbd9e6cf640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd98102980 0x7fbd9819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd9dece640 1 -- 172.21.15.113:0/3065303640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd98101fc0 msgr2=0x7fbd9819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd9dece640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd98101fc0 0x7fbd9819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd9dece640 1 -- 172.21.15.113:0/3065303640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd98102980 msgr2=0x7fbd9819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd9dece640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd98102980 0x7fbd9819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd9dece640 1 -- 172.21.15.113:0/3065303640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9819fee0 con 0x7fbd98100e70 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd9dece640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 0x7fbd98069c50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c920 tx=0x7fbd9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94007c70 con 0x7fbd98100e70 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbd94007e10 con 0x7fbd98100e70 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981a0170 con 0x7fbd98100e70 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940188c0 con 0x7fbd98100e70 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.902+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a06c0 con 0x7fbd98100e70 2024-04-22T07:14:40.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.903+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd98108180 con 0x7fbd98100e70 2024-04-22T07:14:40.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.904+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fbd94018a60 con 0x7fbd98100e70 2024-04-22T07:14:40.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.904+0000 7fbd8effd640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbd840a3bb0 0x7fbd840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:40.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.904+0000 7fbd9d6cd640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbd840a3bb0 0x7fbd840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:40.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.905+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbd940d08c0 con 0x7fbd98100e70 2024-04-22T07:14:40.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.905+0000 7fbd9d6cd640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbd840a3bb0 0x7fbd840a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd88036040 tx=0x7fbd88000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:40.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:40.908+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd94014030 con 0x7fbd98100e70 2024-04-22T07:14:41.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.089+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd9802d790 con 0x7fbd98100e70 2024-04-22T07:14:41.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.091+0000 7fbd8effd640 1 -- 172.21.15.113:0/3065303640 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22700 (secure 0 0 0) 0x7fbd9409d340 con 0x7fbd98100e70 2024-04-22T07:14:41.089 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:13:56.180466+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:41.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.092+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbd840a3bb0 msgr2=0x7fbd840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.092+0000 7fbda0132640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbd840a3bb0 0x7fbd840a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd88036040 tx=0x7fbd88000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.092+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 msgr2=0x7fbd98069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.092+0000 7fbda0132640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 0x7fbd98069c50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c920 tx=0x7fbd9400cde0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 shutdown_connections 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbd840a3bb0 0x7fbd840a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd98102980 0x7fbd9819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd98101fc0 0x7fbd9819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 --2- 172.21.15.113:0/3065303640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd98100e70 0x7fbd98069c50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 >> 172.21.15.113:0/3065303640 conn(0x7fbd980fc3e0 msgr2=0x7fbd98105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 shutdown_connections 2024-04-22T07:14:41.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.093+0000 7fbda0132640 1 -- 172.21.15.113:0/3065303640 wait complete. 2024-04-22T07:14:41.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T07:12:54.514841+0000', 'modified': '2024-04-22T07:12:58.522890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24473, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/217490889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 217490889}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 217490889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/1615298521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1615298521}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1615298521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/3935683574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3935683574}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3935683574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.154:6839/3404626603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 3404626603}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 3404626603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6837/65907218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 65907218}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 65907218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6839/982201245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 982201245}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 982201245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24469, 24473, 24506]} 2024-04-22T07:14:41.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-22T07:14:41.103 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T07:14:41.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.204+0000 7f9eb99ca640 1 -- 172.21.15.113:0/946070024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4102ab0 msgr2=0x7f9eb4104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.204+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/946070024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4102ab0 0x7f9eb4104ea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009900 tx=0x7f9ea402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- 172.21.15.113:0/946070024 shutdown_connections 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/946070024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eb4107d10 0x7f9eb410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/946070024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb41053e0 0x7f9eb41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/946070024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4102ab0 0x7f9eb4104ea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- 172.21.15.113:0/946070024 >> 172.21.15.113:0/946070024 conn(0x7f9eb40fc400 msgr2=0x7f9eb40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- 172.21.15.113:0/946070024 shutdown_connections 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- 172.21.15.113:0/946070024 wait complete. 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 Processor -- start 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- start start 2024-04-22T07:14:41.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eb4102ab0 0x7f9eb419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb41053e0 0x7f9eb419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 0x7f9eb41a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb410dc10 con 0x7f9eb4107d10 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb410da90 con 0x7f9eb41053e0 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.205+0000 7f9eb99ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb410dd90 con 0x7f9eb4102ab0 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 0x7f9eb41a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 0x7f9eb41a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34684/0 (socket says 172.21.15.113:34684) 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 -- 172.21.15.113:0/2178290523 learned_addr learned my addr 172.21.15.113:0/2178290523 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eb4102ab0 0x7f9eb419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb41053e0 0x7f9eb419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 -- 172.21.15.113:0/2178290523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eb4102ab0 msgr2=0x7f9eb419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eb4102ab0 0x7f9eb419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 -- 172.21.15.113:0/2178290523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb41053e0 msgr2=0x7f9eb419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb41053e0 0x7f9eb419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 -- 172.21.15.113:0/2178290523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb41a20e0 con 0x7f9eb4107d10 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb37fe640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 0x7f9eb41a1a70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00eaa0 tx=0x7f9eac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb89c8640 1 -- 172.21.15.113:0/2178290523 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00cdb0 con 0x7f9eb4107d10 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb41a23d0 con 0x7f9eb4107d10 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb89c8640 1 -- 172.21.15.113:0/2178290523 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9eac005020 con 0x7f9eb4107d10 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb89c8640 1 -- 172.21.15.113:0/2178290523 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0116b0 con 0x7f9eb4107d10 2024-04-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.206+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb41a2920 con 0x7f9eb4107d10 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.208+0000 7f9eb89c8640 1 -- 172.21.15.113:0/2178290523 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f9eac0051d0 con 0x7f9eb4107d10 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.208+0000 7f9ebc5b6740 1 Processor -- start 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 -- start start 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a642830 0x562e7a642cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a6432d0 0x562e7a649eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb89c8640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9e9c0a3bb0 0x7f9e9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562e7a5a6b10 con 0x562e7a641e40 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e7a544240 con 0x562e7a6432d0 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562e7a5a2ad0 con 0x562e7a642830 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb37fe640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9e9c0a3bb0 0x7f9e9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34690/0 (socket says 172.21.15.113:34690) 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2183614318 learned_addr learned my addr 172.21.15.113:0/2183614318 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb89c8640 1 -- 172.21.15.113:0/2178290523 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9eac0d05e0 con 0x7f9eb4107d10 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a6432d0 0x562e7a649eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a642830 0x562e7a642cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2183614318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a642830 msgr2=0x562e7a642cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a642830 0x562e7a642cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2183614318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a6432d0 msgr2=0x562e7a649eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a6432d0 0x562e7a649eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2183614318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7a2b92d0 con 0x562e7a641e40 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a642830 0x562e7a642cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800afd0 tx=0x7f9ea800a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906a608bae29aa80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9eb37fe640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9e9c0a3bb0 0x7f9e9c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9eac09c5d0 tx=0x7f9eac06e360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.209+0000 7f9e99ffb640 1 -- 172.21.15.113:0/2183614318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8019030 con 0x562e7a641e40 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9e99ffb640 1 -- 172.21.15.113:0/2183614318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ea80054f0 con 0x562e7a641e40 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9e99ffb640 1 -- 172.21.15.113:0/2183614318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8004ce0 con 0x562e7a641e40 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2183614318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 msgr2=0x562e7a642260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800afd0 tx=0x7f9ea800a720 comp rx=0 tx=0).stop 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2183614318 shutdown_connections 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a6432d0 0x562e7a649eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a642830 0x562e7a642cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2183614318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642260 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2183614318 >> 172.21.15.113:0/2183614318 conn(0x562e7a63df40 msgr2=0x562e7a63c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2183614318 shutdown_connections 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2183614318 wait complete. 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.210+0000 7f9ebc5b6740 1 -- start start 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 Processor -- start 2024-04-22T07:14:41.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a641e40 0x562e7a64be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 0x562e7a64c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a6533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562e7a64a4b0 con 0x562e7a642830 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e7a52f190 con 0x562e7a641e40 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562e7a64a630 con 0x562e7a6432d0 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9ebc5b6740 10 client.0 _ll_register_callbacks cb 0x562e7a52a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 0x562e7a64c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 0x562e7a64c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34698/0 (socket says 172.21.15.113:34698) 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 -- 172.21.15.113:0/4110452005 learned_addr learned my addr 172.21.15.113:0/4110452005 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a641e40 0x562e7a64be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 -- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 msgr2=0x562e7a6533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a6533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 -- 172.21.15.113:0/4110452005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a641e40 msgr2=0x562e7a64be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a641e40 0x562e7a64be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 -- 172.21.15.113:0/4110452005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e7a3836b0 con 0x562e7a642830 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 0x562e7a64c3a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005ab0 tx=0x7f9ea4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574cfa5da05fd96 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.211+0000 7f9e92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9e92ffd640 1 -- 172.21.15.113:0/4110452005 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4005d50 con 0x562e7a642830 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9e92ffd640 1 -- 172.21.15.113:0/4110452005 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9ea4005230 con 0x562e7a642830 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea80054f0 con 0x562e7a642830 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9e92ffd640 1 -- 172.21.15.113:0/4110452005 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9ea404d050 con 0x562e7a642830 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 10 client.15213 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:41.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562e7a656730 con 0x562e7a642830 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9e92ffd640 1 -- 172.21.15.113:0/4110452005 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ea80054f0 con 0x562e7a642830 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9e92ffd640 1 -- 172.21.15.113:0/4110452005 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9ea4003920 con 0x562e7a642830 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 10 client.15213 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 20 client.15213 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a658db0 0x562e7a65a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 4 client.15213 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.212+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e7a5b4530 con 0x562e7a658db0 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.213+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a658db0 0x562e7a65a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.213+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a658db0 0x562e7a65a220 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b04e36a57c953e60 server_cookie=c7998092bf2d7f13 in_seq=0 out_seq=0 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.213+0000 7f9e92ffd640 10 client.15213 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9e92ffd640 1 -- 172.21.15.113:0/4110452005 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e7a5b4530 con 0x562e7a658db0 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9e92ffd640 10 client.15213 handle_command_reply: tid=0 2024-04-22T07:14:41.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 client.15213 shutdown 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 msgr2=0x562e7a64c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 0x562e7a64c3a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005ab0 tx=0x7f9ea4004270 comp rx=0 tx=0).stop 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 shutdown_connections 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a658db0 0x562e7a65a220 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a6533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a642830 0x562e7a64c3a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9e92ffd640 0 client.15213 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/4110452005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a641e40 0x562e7a64be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 >> 172.21.15.113:0/4110452005 conn(0x562e7a63df40 msgr2=0x562e7a63ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 shutdown_connections 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/4110452005 wait complete. 2024-04-22T07:14:41.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.214+0000 7f9ebc5b6740 20 client.15213 trim_cache size 0 max 16384 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.232+0000 7f9ebc5b6740 1 Processor -- start 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.232+0000 7f9ebc5b6740 1 -- start start 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a646840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562e7a658560 con 0x562e7a641e40 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e7a658c30 con 0x562e7a642830 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562e7a659300 con 0x562e7a6432d0 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34708/0 (socket says 172.21.15.113:34708) 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2729807131 learned_addr learned my addr 172.21.15.113:0/2729807131 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a646840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2729807131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 msgr2=0x562e7a646840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a646840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2729807131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 msgr2=0x562e7a6533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 -- 172.21.15.113:0/2729807131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7a63d630 con 0x562e7a641e40 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800afd0 tx=0x7f9ea800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103a98642ceb8b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9e997fa640 1 -- 172.21.15.113:0/2729807131 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8019040 con 0x562e7a641e40 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9e997fa640 1 -- 172.21.15.113:0/2729807131 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ea802a870 con 0x562e7a641e40 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.233+0000 7f9e997fa640 1 -- 172.21.15.113:0/2729807131 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011be0 con 0x562e7a641e40 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2729807131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 msgr2=0x562e7a642c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800afd0 tx=0x7f9ea800a2d0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2729807131 shutdown_connections 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a6432d0 0x562e7a646840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/2729807131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a641e40 0x562e7a642c10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2729807131 >> 172.21.15.113:0/2729807131 conn(0x562e7a63df40 msgr2=0x562e7a649e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2729807131 shutdown_connections 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/2729807131 wait complete. 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.234+0000 7f9ebc5b6740 1 -- start start 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 Processor -- start 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a641e40 0x562e7a64f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 0x562e7a651f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562e7a65a0a0 con 0x562e7a6432d0 2024-04-22T07:14:41.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e7a6599d0 con 0x562e7a642830 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562e7a65a770 con 0x562e7a641e40 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 10 client.0 _ll_register_callbacks cb 0x7f9ea4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 0x562e7a651f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 0x562e7a651f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34712/0 (socket says 172.21.15.113:34712) 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 -- 172.21.15.113:0/367391006 learned_addr learned my addr 172.21.15.113:0/367391006 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a641e40 0x562e7a64f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 -- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a641e40 msgr2=0x562e7a64f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a641e40 0x562e7a64f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 -- 172.21.15.113:0/367391006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 msgr2=0x562e7a6518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 -- 172.21.15.113:0/367391006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e7a6527a0 con 0x562e7a6432d0 2024-04-22T07:14:41.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9eb27fc640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 0x562e7a651f70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ac30 tx=0x7f9ea8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21481e5b0661fcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9e92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9e92ffd640 1 -- 172.21.15.113:0/367391006 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8038020 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9e92ffd640 1 -- 172.21.15.113:0/367391006 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9ea802a660 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.235+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea802a870 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9e92ffd640 1 -- 172.21.15.113:0/367391006 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9ea8019040 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 10 client.15219 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562e7a6bef20 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9e92ffd640 1 -- 172.21.15.113:0/367391006 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ea802a870 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9e92ffd640 1 -- 172.21.15.113:0/367391006 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f9ea801e9c0 con 0x562e7a6432d0 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 10 client.15219 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T07:14:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 20 client.15219 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a6c3400 0x562e7a6c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 4 client.15219 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "status"}) -- 0x562e7a5ae3a0 con 0x562e7a6c3400 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.236+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a6c3400 0x562e7a6c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9e92ffd640 10 client.15219 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9eb2ffd640 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a6c3400 0x562e7a6c2110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f50fbcede450ecb server_cookie=1c362caae084c905 in_seq=0 out_seq=0 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9e92ffd640 1 -- 172.21.15.113:0/367391006 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562e7a5ae3a0 con 0x562e7a6c3400 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9e92ffd640 10 client.15219 handle_command_reply: tid=0 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 client.15219 shutdown 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 msgr2=0x562e7a651f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 0x562e7a651f70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ac30 tx=0x7f9ea8020040 comp rx=0 tx=0).stop 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 shutdown_connections 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x562e7a6c3400 0x562e7a6c2110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562e7a6432d0 0x562e7a651f70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e7a642830 0x562e7a6518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9e92ffd640 0 client.15219 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 --2- 172.21.15.113:0/367391006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562e7a641e40 0x562e7a64f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 >> 172.21.15.113:0/367391006 conn(0x562e7a63df40 msgr2=0x562e7a649df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 shutdown_connections 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 1 -- 172.21.15.113:0/367391006 wait complete. 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.237+0000 7f9ebc5b6740 20 client.15219 trim_cache size 0 max 16384 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:41.242 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 2, 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24469, 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 103.063019787, 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 88, 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 86, 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 119.83155487099999 2024-04-22T07:14:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:41.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9e9c0a3bb0 msgr2=0x7f9e9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9e9c0a3bb0 0x7f9e9c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9eac09c5d0 tx=0x7f9eac06e360 comp rx=0 tx=0).stop 2024-04-22T07:14:41.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 msgr2=0x7f9eb41a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 0x7f9eb41a1a70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00eaa0 tx=0x7f9eac00ef60 comp rx=0 tx=0).stop 2024-04-22T07:14:41.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 shutdown_connections 2024-04-22T07:14:41.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f9e9c0a3bb0 0x7f9e9c0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb4107d10 0x7f9eb41a1a70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb41053e0 0x7f9eb419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 --2- 172.21.15.113:0/2178290523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9eb4102ab0 0x7f9eb419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 >> 172.21.15.113:0/2178290523 conn(0x7f9eb40fc400 msgr2=0x7f9eb4102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 shutdown_connections 2024-04-22T07:14:41.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.238+0000 7f9eb99ca640 1 -- 172.21.15.113:0/2178290523 wait complete. 2024-04-22T07:14:41.246 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-22T07:14:41.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 -- 172.21.15.113:0/913006537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae4107cd0 msgr2=0x7fbae410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 --2- 172.21.15.113:0/913006537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae4107cd0 0x7fbae410a0c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009ca0 tx=0x7fbadc02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 -- 172.21.15.113:0/913006537 shutdown_connections 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 --2- 172.21.15.113:0/913006537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae4107cd0 0x7fbae410a0c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 --2- 172.21.15.113:0/913006537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae41053a0 0x7fbae4107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 --2- 172.21.15.113:0/913006537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 0x7fbae4104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 -- 172.21.15.113:0/913006537 >> 172.21.15.113:0/913006537 conn(0x7fbae40fc400 msgr2=0x7fbae40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 -- 172.21.15.113:0/913006537 shutdown_connections 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.348+0000 7fbaea206640 1 -- 172.21.15.113:0/913006537 wait complete. 2024-04-22T07:14:41.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 Processor -- start 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 -- start start 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 0x7fbae41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 0x7fbae4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae4107cd0 0x7fbae419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410dc10 con 0x7fbae41053a0 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410da90 con 0x7fbae4102a70 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbaea206640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410dd90 con 0x7fbae4107cd0 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 0x7fbae4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 0x7fbae4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34726/0 (socket says 172.21.15.113:34726) 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.349+0000 7fbae2ffd640 1 -- 172.21.15.113:0/7107 learned_addr learned my addr 172.21.15.113:0/7107 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae3fff640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae4107cd0 0x7fbae419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae2ffd640 1 -- 172.21.15.113:0/7107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae4107cd0 msgr2=0x7fbae419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae2ffd640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae4107cd0 0x7fbae419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae2ffd640 1 -- 172.21.15.113:0/7107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 msgr2=0x7fbae41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae37fe640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 0x7fbae41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae2ffd640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 0x7fbae41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae2ffd640 1 -- 172.21.15.113:0/7107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae419fee0 con 0x7fbae41053a0 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae37fe640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 0x7fbae41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae2ffd640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 0x7fbae4198800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbad400d740 tx=0x7fbad400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae0ff9640 1 -- 172.21.15.113:0/7107 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4017070 con 0x7fbae41053a0 2024-04-22T07:14:41.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae41a01d0 con 0x7fbae41053a0 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae0ff9640 1 -- 172.21.15.113:0/7107 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbad400de50 con 0x7fbae41053a0 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbae0ff9640 1 -- 172.21.15.113:0/7107 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4004d90 con 0x7fbae41053a0 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.350+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae41a0720 con 0x7fbae41053a0 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.352+0000 7fbae0ff9640 1 -- 172.21.15.113:0/7107 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fbad4005020 con 0x7fbae41053a0 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.352+0000 7fbaecdf8740 1 Processor -- start 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.352+0000 7fbaecdf8740 1 -- start start 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b168856260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168856cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae0ff9640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbacc0a3bb0 0x7fbacc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b16885deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b1687bab10 con 0x55b1688572d0 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b168758240 con 0x55b168855e40 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b1687b6ad0 con 0x55b168856830 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b16885deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae2ffd640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbacc0a3bb0 0x7fbacc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b16885deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34736/0 (socket says 172.21.15.113:34736) 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 -- 172.21.15.113:0/1463587741 learned_addr learned my addr 172.21.15.113:0/1463587741 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae37fe640 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168856cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae0ff9640 1 -- 172.21.15.113:0/7107 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbad40cfe50 con 0x7fbae41053a0 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b168856260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 -- 172.21.15.113:0/1463587741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 msgr2=0x55b168856cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168856cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 -- 172.21.15.113:0/1463587741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 msgr2=0x55b168856260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b168856260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 -- 172.21.15.113:0/1463587741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1684cd2d0 con 0x55b1688572d0 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae2ffd640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbacc0a3bb0 0x7fbacc0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbad409b000 tx=0x7fbad4066790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbae3fff640 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b16885deb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbadc032460 tx=0x7fbadc032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57267a45e1b28a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbac9ffb640 1 -- 172.21.15.113:0/1463587741 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc048070 con 0x55b1688572d0 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbac9ffb640 1 -- 172.21.15.113:0/1463587741 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbadc003970 con 0x55b1688572d0 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.353+0000 7fbac9ffb640 1 -- 172.21.15.113:0/1463587741 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc039070 con 0x55b1688572d0 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 -- 172.21.15.113:0/1463587741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 msgr2=0x55b16885deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b16885deb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbadc032460 tx=0x7fbadc032340 comp rx=0 tx=0).stop 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 -- 172.21.15.113:0/1463587741 shutdown_connections 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b16885deb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168856cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/1463587741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b168856260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 -- 172.21.15.113:0/1463587741 >> 172.21.15.113:0/1463587741 conn(0x55b168851f40 msgr2=0x55b168850490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 -- 172.21.15.113:0/1463587741 shutdown_connections 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 -- 172.21.15.113:0/1463587741 wait complete. 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.354+0000 7fbaecdf8740 1 -- start start 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 Processor -- start 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168856830 0x55b168860420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168867460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b16885e450 con 0x55b1688572d0 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b168743190 con 0x55b168856830 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b16885e5d0 con 0x55b168855e40 2024-04-22T07:14:41.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbaecdf8740 10 client.0 _ll_register_callbacks cb 0x55b16873e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168867460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168867460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34752/0 (socket says 172.21.15.113:34752) 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 -- 172.21.15.113:0/2287873063 learned_addr learned my addr 172.21.15.113:0/2287873063 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168856830 0x55b168860420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 -- 172.21.15.113:0/2287873063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 msgr2=0x55b16885fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 -- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168856830 msgr2=0x55b168860420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168856830 0x55b168860420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 -- 172.21.15.113:0/2287873063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1685976b0 con 0x55b1688572d0 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168867460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009620 tx=0x7fbadc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b011598c45d989e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2287873063 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc052060 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.355+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2287873063 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbadc0047f0 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbadc003970 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2287873063 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbadc039050 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 10 client.15231 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b16886a7b0 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2287873063 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbadc003970 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2287873063 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbadc034b80 con 0x55b1688572d0 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 10 client.15231 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 10 client.15231 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 20 client.15231 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b16886c8f0 0x55b16886dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 4 client.15231 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1687c8530 con 0x55b16886c8f0 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.356+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b16886c8f0 0x55b16886dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.357+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b16886c8f0 0x55b16886dd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c921817ac8f47bb5 server_cookie=dd9a1989e9a073b0 in_seq=0 out_seq=0 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.357+0000 7fbac2ffd640 10 client.15231 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.357+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2287873063 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1687c8530 con 0x55b16886c8f0 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.357+0000 7fbac2ffd640 10 client.15231 handle_command_reply: tid=0 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.357+0000 7fbaecdf8740 1 client.15231 shutdown 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 msgr2=0x55b168867460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168867460 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009620 tx=0x7fbadc0021d0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 shutdown_connections 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b16886c8f0 0x55b16886dd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168867460 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbac2ffd640 0 client.15231 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168856830 0x55b168860420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2287873063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 >> 172.21.15.113:0/2287873063 conn(0x55b168851f40 msgr2=0x55b16884fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 shutdown_connections 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2287873063 wait complete. 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.358+0000 7fbaecdf8740 20 client.15231 trim_cache size 0 max 16384 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.375+0000 7fbaecdf8740 1 Processor -- start 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.375+0000 7fbaecdf8740 1 -- start start 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.375+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b16885faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168867420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168860030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b16886c0a0 con 0x55b1688572d0 2024-04-22T07:14:41.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b16886c770 con 0x55b168855e40 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b16886ce40 con 0x55b168856830 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168860030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168860030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34760/0 (socket says 172.21.15.113:34760) 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 -- 172.21.15.113:0/2861992749 learned_addr learned my addr 172.21.15.113:0/2861992749 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 -- 172.21.15.113:0/2861992749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 msgr2=0x55b168867420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168867420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168867420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 -- 172.21.15.113:0/2861992749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 msgr2=0x55b16885faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b16885faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 -- 172.21.15.113:0/2861992749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16884ad80 con 0x55b1688572d0 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168860030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbadc032b90 tx=0x7fbadc032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2626d25ce6a994 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbac97fa640 1 -- 172.21.15.113:0/2861992749 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc05c020 con 0x55b1688572d0 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbac97fa640 1 -- 172.21.15.113:0/2861992749 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbadc005c30 con 0x55b1688572d0 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.376+0000 7fbac97fa640 1 -- 172.21.15.113:0/2861992749 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc052070 con 0x55b1688572d0 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2861992749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 msgr2=0x55b168860030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168860030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbadc032b90 tx=0x7fbadc032340 comp rx=0 tx=0).stop 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2861992749 shutdown_connections 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b1688572d0 0x55b168860030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168856830 0x55b168867420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2861992749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b168855e40 0x55b16885faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2861992749 >> 172.21.15.113:0/2861992749 conn(0x55b168851f40 msgr2=0x55b16885a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2861992749 shutdown_connections 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2861992749 wait complete. 2024-04-22T07:14:41.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- start start 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 Processor -- start 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 0x55b168865410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1688572d0 0x55b168865a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b16886dbe0 con 0x55b168856830 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b16886d510 con 0x55b1688572d0 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b16886e2b0 con 0x55b168855e40 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbaecdf8740 10 client.0 _ll_register_callbacks cb 0x7fbadc046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbae37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 0x55b168865410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.377+0000 7fbae37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 0x55b168865410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34764/0 (socket says 172.21.15.113:34764) 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 -- 172.21.15.113:0/2064113811 learned_addr learned my addr 172.21.15.113:0/2064113811 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1688572d0 0x55b168865a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 -- 172.21.15.113:0/2064113811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 msgr2=0x55b16885a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 -- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1688572d0 msgr2=0x55b168865a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1688572d0 0x55b168865a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 -- 172.21.15.113:0/2064113811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b168866250 con 0x55b168856830 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1688572d0 0x55b168865a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbae37fe640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 0x55b168865410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009790 tx=0x7fbad80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17ad842ea3033bb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2064113811 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004220 con 0x55b168856830 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2064113811 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbad80043c0 con 0x55b168856830 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbadc005c30 con 0x55b168856830 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2064113811 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbad801e320 con 0x55b168856830 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 10 client.15237 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:41.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b168867650 con 0x55b168856830 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2064113811 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fbadc005c30 con 0x55b168856830 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2064113811 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fbad800f040 con 0x55b168856830 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 10 client.15237 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 10 client.15237 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 20 client.15237 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b1688d7100 0x55b1688d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 4 client.15237 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.378+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "status"}) -- 0x55b1687c23a0 con 0x55b1688d7100 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b1688d7100 0x55b1688d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbac2ffd640 10 client.15237 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbae3fff640 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b1688d7100 0x55b1688d5e10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5ebb2ef32b6161f server_cookie=ed764eebae991849 in_seq=0 out_seq=0 2024-04-22T07:14:41.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbac2ffd640 1 -- 172.21.15.113:0/2064113811 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b1687c23a0 con 0x55b1688d7100 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbac2ffd640 10 client.15237 handle_command_reply: tid=0 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 client.15237 shutdown 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 msgr2=0x55b168865410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 0x55b168865410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009790 tx=0x7fbad80128e0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 shutdown_connections 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55b1688d7100 0x55b1688d5e10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1688572d0 0x55b168865a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b168856830 0x55b168865410 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbac2ffd640 0 client.15237 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.379+0000 7fbaecdf8740 1 --2- 172.21.15.113:0/2064113811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b168855e40 0x55b16885a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 >> 172.21.15.113:0/2064113811 conn(0x55b168851f40 msgr2=0x55b16884f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 shutdown_connections 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaecdf8740 1 -- 172.21.15.113:0/2064113811 wait complete. 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaecdf8740 20 client.15237 trim_cache size 0 max 16384 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 1, 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24473, 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:14:41.444 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 104.212261079, 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 88, 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 86, 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 122.697809294 2024-04-22T07:14:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:41.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbacc0a3bb0 msgr2=0x7fbacc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbacc0a3bb0 0x7fbacc0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbad409b000 tx=0x7fbad4066790 comp rx=0 tx=0).stop 2024-04-22T07:14:41.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 msgr2=0x7fbae4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 0x7fbae4198800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbad400d740 tx=0x7fbad400dc00 comp rx=0 tx=0).stop 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 shutdown_connections 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fbacc0a3bb0 0x7fbacc0a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbae4107cd0 0x7fbae419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbae41053a0 0x7fbae4198800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 --2- 172.21.15.113:0/7107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbae4102a70 0x7fbae41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 >> 172.21.15.113:0/7107 conn(0x7fbae40fc400 msgr2=0x7fbae41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 shutdown_connections 2024-04-22T07:14:41.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.380+0000 7fbaea206640 1 -- 172.21.15.113:0/7107 wait complete. 2024-04-22T07:14:41.448 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:41 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1977479401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:41.448 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:41 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4048336625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:41.448 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:41 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3065303640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:41.449 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 -- 172.21.15.113:0/880460463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79641010a0 msgr2=0x7f7964101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 --2- 172.21.15.113:0/880460463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79641010a0 0x7f7964101500 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 -- 172.21.15.113:0/880460463 shutdown_connections 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 --2- 172.21.15.113:0/880460463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964101a40 0x7f7964109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 --2- 172.21.15.113:0/880460463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79641010a0 0x7f7964101500 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 --2- 172.21.15.113:0/880460463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7964102450 0x7f7964102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 -- 172.21.15.113:0/880460463 >> 172.21.15.113:0/880460463 conn(0x7f79640fbb90 msgr2=0x7f79640fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 -- 172.21.15.113:0/880460463 shutdown_connections 2024-04-22T07:14:41.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.553+0000 7f796ad40640 1 -- 172.21.15.113:0/880460463 wait complete. 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 Processor -- start 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 -- start start 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79641010a0 0x7f7964100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 0x7f796419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964102450 0x7f796419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f796410cb50 con 0x7f7964101a40 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f796410c9d0 con 0x7f7964102450 2024-04-22T07:14:41.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f796ad40640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f796410ccd0 con 0x7f79641010a0 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 0x7f796419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 0x7f796419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34786/0 (socket says 172.21.15.113:34786) 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 -- 172.21.15.113:0/577780295 learned_addr learned my addr 172.21.15.113:0/577780295 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f79692dd640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964102450 0x7f796419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 -- 172.21.15.113:0/577780295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79641010a0 msgr2=0x7f7964100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7968adc640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79641010a0 0x7f7964100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79641010a0 0x7f7964100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 -- 172.21.15.113:0/577780295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964102450 msgr2=0x7f796419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964102450 0x7f796419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f7963fff640 1 -- 172.21.15.113:0/577780295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796419ff70 con 0x7f7964101a40 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.554+0000 7f79692dd640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964102450 0x7f796419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:41.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.555+0000 7f7963fff640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 0x7f796419d040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7950006010 tx=0x7f79500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.555+0000 7f7961ffb640 1 -- 172.21.15.113:0/577780295 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950041070 con 0x7f7964101a40 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.555+0000 7f7961ffb640 1 -- 172.21.15.113:0/577780295 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f79500044c0 con 0x7f7964101a40 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.555+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641a01a0 con 0x7f7964101a40 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.555+0000 7f7961ffb640 1 -- 172.21.15.113:0/577780295 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950034440 con 0x7f7964101a40 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.555+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641a0770 con 0x7f7964101a40 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.557+0000 7f7961ffb640 1 -- 172.21.15.113:0/577780295 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7950044070 con 0x7f7964101a40 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.557+0000 7f796da0d740 1 Processor -- start 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.557+0000 7f796da0d740 1 -- start start 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f796da0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46297260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f796da0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b46297cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f796da0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b4629eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7961ffb640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f79540a3bb0 0x7f79540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f796da0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b461fbb10 con 0x560b462982d0 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f796da0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b46199240 con 0x560b46296e40 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f796da0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b461f7ad0 con 0x560b46297830 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b4629eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f79692dd640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f79540a3bb0 0x7f79540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b4629eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34800/0 (socket says 172.21.15.113:34800) 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 -- 172.21.15.113:0/304821329 learned_addr learned my addr 172.21.15.113:0/304821329 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7968adc640 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b46297cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7961ffb640 1 -- 172.21.15.113:0/577780295 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f79500f2140 con 0x7f7964101a40 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f79692dd640 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46297260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 -- 172.21.15.113:0/304821329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 msgr2=0x560b46297cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b46297cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 -- 172.21.15.113:0/304821329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 msgr2=0x560b46297260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46297260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 -- 172.21.15.113:0/304821329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b45f0e2d0 con 0x560b462982d0 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f79692dd640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f79540a3bb0 0x7f79540a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f795c005c40 tx=0x7f795c0127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f7963fff640 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b4629eeb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f79500efc50 tx=0x7f795008fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb1010492b04bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f794effd640 1 -- 172.21.15.113:0/304821329 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795004b020 con 0x560b462982d0 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f794effd640 1 -- 172.21.15.113:0/304821329 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f79500beb30 con 0x560b462982d0 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.558+0000 7f794effd640 1 -- 172.21.15.113:0/304821329 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500bbc60 con 0x560b462982d0 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 -- 172.21.15.113:0/304821329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 msgr2=0x560b4629eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b4629eeb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f79500efc50 tx=0x7f795008fec0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 -- 172.21.15.113:0/304821329 shutdown_connections 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b4629eeb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b46297cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 --2- 172.21.15.113:0/304821329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46297260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 -- 172.21.15.113:0/304821329 >> 172.21.15.113:0/304821329 conn(0x560b46292f40 msgr2=0x560b46291490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 -- 172.21.15.113:0/304821329 shutdown_connections 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 -- 172.21.15.113:0/304821329 wait complete. 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 -- start start 2024-04-22T07:14:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.559+0000 7f796da0d740 1 Processor -- start 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46296e40 0x560b462a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 0x560b462a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b46298ad0 con 0x560b46297830 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b46184190 con 0x560b462982d0 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b4629f3f0 con 0x560b46296e40 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 10 client.0 _ll_register_callbacks cb 0x560b4617f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 0x560b462a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 0x560b462a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34806/0 (socket says 172.21.15.113:34806) 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 -- 172.21.15.113:0/1622118752 learned_addr learned my addr 172.21.15.113:0/1622118752 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f79692dd640 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 -- 172.21.15.113:0/1622118752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46296e40 msgr2=0x560b462a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46296e40 0x560b462a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 -- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 msgr2=0x560b462a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 -- 172.21.15.113:0/1622118752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b45fd86b0 con 0x560b46297830 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7963fff640 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 0x560b462a1110 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f79500be3c0 tx=0x7f79500bde10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ebc7ee4b7a9fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7943fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7943fff640 1 -- 172.21.15.113:0/1622118752 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795004b070 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f7943fff640 1 -- 172.21.15.113:0/1622118752 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f79500bc730 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.560+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79500beb30 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f7943fff640 1 -- 172.21.15.113:0/1622118752 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79500bba80 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 10 client.15249 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b462ab4c0 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f7943fff640 1 -- 172.21.15.113:0/1622118752 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f79500beb30 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f7943fff640 1 -- 172.21.15.113:0/1622118752 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f79500bcf60 con 0x560b46297830 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 10 client.15249 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 10 client.15249 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 20 client.15249 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b462ad950 0x560b462aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 4 client.15249 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b46209530 con 0x560b462ad950 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.561+0000 7f7968adc640 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b462ad950 0x560b462aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f7943fff640 10 client.15249 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f7968adc640 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b462ad950 0x560b462aedc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7e64e0f62a0185 server_cookie=d84706aa5cf49b1f in_seq=0 out_seq=0 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f7943fff640 1 -- 172.21.15.113:0/1622118752 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b46209530 con 0x560b462ad950 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f7943fff640 10 client.15249 handle_command_reply: tid=0 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 client.15249 shutdown 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 msgr2=0x560b462a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 0x560b462a1110 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f79500be3c0 tx=0x7f79500bde10 comp rx=0 tx=0).stop 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 shutdown_connections 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b462ad950 0x560b462aedc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46297830 0x560b462a1110 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 --2- 172.21.15.113:0/1622118752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46296e40 0x560b462a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f7943fff640 0 client.15249 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.562+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 >> 172.21.15.113:0/1622118752 conn(0x560b46292f40 msgr2=0x560b46290a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.563+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 shutdown_connections 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.563+0000 7f796da0d740 1 -- 172.21.15.113:0/1622118752 wait complete. 2024-04-22T07:14:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.563+0000 7f796da0d740 20 client.15249 trim_cache size 0 max 16384 2024-04-22T07:14:41.578 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:41 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1977479401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:41.578 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:41 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4048336625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:41.578 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:41 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3065303640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:41.578 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:41 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1977479401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:41.579 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:41 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4048336625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:41.579 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:41 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3065303640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 Processor -- start 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 -- start start 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46291350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b46291890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b462ad100 con 0x560b462982d0 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b462ad7d0 con 0x560b46296e40 2024-04-22T07:14:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f796da0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b462adea0 con 0x560b46297830 2024-04-22T07:14:41.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b46291890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b46291890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34820/0 (socket says 172.21.15.113:34820) 2024-04-22T07:14:41.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 -- 172.21.15.113:0/2049403667 learned_addr learned my addr 172.21.15.113:0/2049403667 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f7963fff640 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f7968adc640 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46291350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 -- 172.21.15.113:0/2049403667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 msgr2=0x560b462a8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 -- 172.21.15.113:0/2049403667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 msgr2=0x560b46291350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46291350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 -- 172.21.15.113:0/2049403667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4628ce70 con 0x560b462982d0 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f7968adc640 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46291350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f79692dd640 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b46291890 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f795c010900 tx=0x7f795c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ce865e0677b9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f794e7fc640 1 -- 172.21.15.113:0/2049403667 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c01d900 con 0x560b462982d0 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f794e7fc640 1 -- 172.21.15.113:0/2049403667 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f795c029480 con 0x560b462982d0 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.581+0000 7f794e7fc640 1 -- 172.21.15.113:0/2049403667 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c01e630 con 0x560b462982d0 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 -- 172.21.15.113:0/2049403667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 msgr2=0x560b46291890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b46291890 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f795c010900 tx=0x7f795c00c000 comp rx=0 tx=0).stop 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 -- 172.21.15.113:0/2049403667 shutdown_connections 2024-04-22T07:14:41.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b462982d0 0x560b46291890 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 --2- 172.21.15.113:0/2049403667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b46296e40 0x560b46291350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 -- 172.21.15.113:0/2049403667 >> 172.21.15.113:0/2049403667 conn(0x560b46292f40 msgr2=0x560b4629b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 -- 172.21.15.113:0/2049403667 shutdown_connections 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 -- 172.21.15.113:0/2049403667 wait complete. 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 -- start start 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.582+0000 7f796da0d740 1 Processor -- start 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 0x560b462a3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b462aec40 con 0x560b46296e40 2024-04-22T07:14:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b462ae570 con 0x560b462982d0 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b462af310 con 0x560b46297830 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 10 client.0 _ll_register_callbacks cb 0x7f795800a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 0x560b462a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 0x560b462a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34828/0 (socket says 172.21.15.113:34828) 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 -- 172.21.15.113:0/3291540701 learned_addr learned my addr 172.21.15.113:0/3291540701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7963fff640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f79692dd640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 -- 172.21.15.113:0/3291540701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 msgr2=0x560b462a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 -- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 msgr2=0x560b462a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:41.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 -- 172.21.15.113:0/3291540701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b462a7180 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7968adc640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 0x560b462a3630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x560b462af500 tx=0x7f795800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4ee6bae4e31ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7943fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7943fff640 1 -- 172.21.15.113:0/3291540701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958018e60 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f7943fff640 1 -- 172.21.15.113:0/3291540701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f795801a0b0 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.583+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795c029480 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f7943fff640 1 -- 172.21.15.113:0/3291540701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f795801a2b0 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 10 client.15255 fetch_fsmap learned FSMap version 21 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b462a84c0 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f7943fff640 1 -- 172.21.15.113:0/3291540701 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f795c029480 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f7943fff640 1 -- 172.21.15.113:0/3291540701 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7958023c30 con 0x560b46296e40 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 10 client.15255 fetch_fsmap finished waiting for FSMap version 21 2024-04-22T07:14:41.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 20 client.15255 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b46318310 0x560b46317020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 4 client.15255 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "status"}) -- 0x560b462033a0 con 0x560b46318310 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f7963fff640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b46318310 0x560b46317020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f7943fff640 10 client.15255 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.584+0000 7f7963fff640 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b46318310 0x560b46317020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79fd168fc5ee25 server_cookie=8792b35dc2330822 in_seq=0 out_seq=0 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f7943fff640 1 -- 172.21.15.113:0/3291540701 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560b462033a0 con 0x560b46318310 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f7943fff640 10 client.15255 handle_command_reply: tid=0 2024-04-22T07:14:41.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 client.15255 shutdown 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 msgr2=0x560b462a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 0x560b462a3630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x560b462af500 tx=0x7f795800af40 comp rx=0 tx=0).stop 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 shutdown_connections 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b462982d0 0x560b462a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b46297830 0x560b462a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560b46318310 0x560b46317020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 --2- 172.21.15.113:0/3291540701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b46296e40 0x560b462a3630 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f7943fff640 0 client.15255 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 >> 172.21.15.113:0/3291540701 conn(0x560b46292f40 msgr2=0x560b46291350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 shutdown_connections 2024-04-22T07:14:41.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 1 -- 172.21.15.113:0/3291540701 wait complete. 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796da0d740 20 client.15255 trim_cache size 0 max 16384 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24506, 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:14:41.594 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 106.428366041, 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 21, 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 88, 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 86, 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 112.073804169 2024-04-22T07:14:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f79540a3bb0 msgr2=0x7f79540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796ad40640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f79540a3bb0 0x7f79540a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f795c005c40 tx=0x7f795c0127c0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 msgr2=0x7f796419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.585+0000 7f796ad40640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 0x7f796419d040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7950006010 tx=0x7f79500040c0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 shutdown_connections 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f79540a3bb0 0x7f79540a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7964102450 0x7f796419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7964101a40 0x7f796419d040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 --2- 172.21.15.113:0/577780295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79641010a0 0x7f7964100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 >> 172.21.15.113:0/577780295 conn(0x7f79640fbb90 msgr2=0x7f79641053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 shutdown_connections 2024-04-22T07:14:41.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.586+0000 7f796ad40640 1 -- 172.21.15.113:0/577780295 wait complete. 2024-04-22T07:14:41.598 INFO:teuthology.run_tasks:Running task sequential... 2024-04-22T07:14:41.643 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-22T07:14:41.643 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-22T07:14:41.644 INFO:teuthology.task.print:Enabling mgr modules 2024-04-22T07:14:41.644 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-22T07:14:41.645 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T07:14:41.645 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:14:41.645 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-22T07:14:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 -- 172.21.15.113:0/3083833293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4102040 msgr2=0x7f58c41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3083833293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4102040 0x7f58c41024c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009900 tx=0x7f58b802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 -- 172.21.15.113:0/3083833293 shutdown_connections 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3083833293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c4102a00 0x7f58c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3083833293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4102040 0x7f58c41024c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3083833293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58c4100e60 0x7f58c4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 -- 172.21.15.113:0/3083833293 >> 172.21.15.113:0/3083833293 conn(0x7f58c40fc420 msgr2=0x7f58c40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 -- 172.21.15.113:0/3083833293 shutdown_connections 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.745+0000 7f58cc54b640 1 -- 172.21.15.113:0/3083833293 wait complete. 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 Processor -- start 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 -- start start 2024-04-22T07:14:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 0x7f58c41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58c4102040 0x7f58c4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c4102a00 0x7f58c419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410cb80 con 0x7f58c4100e60 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410ca00 con 0x7f58c4102a00 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58cc54b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58c410cd00 con 0x7f58c4102040 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58ca2e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 0x7f58c41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58ca2e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 0x7f58c41981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34852/0 (socket says 172.21.15.113:34852) 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58ca2e7640 1 -- 172.21.15.113:0/3476680410 learned_addr learned my addr 172.21.15.113:0/3476680410 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58c9ae6640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58c4102040 0x7f58c4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58caae8640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c4102a00 0x7f58c419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.746+0000 7f58ca2e7640 1 -- 172.21.15.113:0/3476680410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58c4102040 msgr2=0x7f58c4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58ca2e7640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58c4102040 0x7f58c4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58ca2e7640 1 -- 172.21.15.113:0/3476680410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c4102a00 msgr2=0x7f58c419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58ca2e7640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c4102a00 0x7f58c419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58ca2e7640 1 -- 172.21.15.113:0/3476680410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c419fda0 con 0x7f58c4100e60 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58ca2e7640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 0x7f58c41981d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c970 tx=0x7f58c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0007c10 con 0x7f58c4100e60 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f58c0007db0 con 0x7f58c4100e60 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c41a0090 con 0x7f58c4100e60 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c000f910 con 0x7f58c4100e60 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.747+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c41a0600 con 0x7f58c4100e60 2024-04-22T07:14:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.748+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c41a08c0 con 0x7f58c4100e60 2024-04-22T07:14:41.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.749+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f58c000fab0 con 0x7f58c4100e60 2024-04-22T07:14:41.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.749+0000 7f58b77fe640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58b00a3cd0 0x7f58b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:41.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.750+0000 7f58c9ae6640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58b00a3cd0 0x7f58b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:41.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.750+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f58c00d0810 con 0x7f58c4100e60 2024-04-22T07:14:41.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.750+0000 7f58c9ae6640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58b00a3cd0 0x7f58b00a6190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006010 tx=0x7f58b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:41.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.754+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58c009d290 con 0x7f58c4100e60 2024-04-22T07:14:41.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:41.982+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f58c41a0a90 con 0x7f58c4100e60 2024-04-22T07:14:42.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.220+0000 7f58b77fe640 1 -- 172.21.15.113:0/3476680410 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f58c009cb90 con 0x7f58c4100e60 2024-04-22T07:14:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58b00a3cd0 msgr2=0x7f58b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58b00a3cd0 0x7f58b00a6190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006010 tx=0x7f58b8000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 msgr2=0x7f58c41981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 0x7f58c41981d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f58c000c970 tx=0x7f58c000ce30 comp rx=0 tx=0).stop 2024-04-22T07:14:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 shutdown_connections 2024-04-22T07:14:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f58b00a3cd0 0x7f58b00a6190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c4102a00 0x7f58c419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58c4102040 0x7f58c4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 --2- 172.21.15.113:0/3476680410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c4100e60 0x7f58c41981d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 >> 172.21.15.113:0/3476680410 conn(0x7f58c40fc420 msgr2=0x7f58c40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 shutdown_connections 2024-04-22T07:14:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.222+0000 7f58cc54b640 1 -- 172.21.15.113:0/3476680410 wait complete. 2024-04-22T07:14:42.238 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.345+0000 7f193bcc7640 1 -- 172.21.15.113:0/229374110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934107c90 msgr2=0x7f193410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.345+0000 7f193bcc7640 1 --2- 172.21.15.113:0/229374110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934107c90 0x7f193410a0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f192c009ca0 tx=0x7f192c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- 172.21.15.113:0/229374110 shutdown_connections 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 --2- 172.21.15.113:0/229374110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934107c90 0x7f193410a0e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 --2- 172.21.15.113:0/229374110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1934105360 0x7f1934107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 --2- 172.21.15.113:0/229374110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934102a30 0x7f1934104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- 172.21.15.113:0/229374110 >> 172.21.15.113:0/229374110 conn(0x7f19340fc420 msgr2=0x7f19340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- 172.21.15.113:0/229374110 shutdown_connections 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- 172.21.15.113:0/229374110 wait complete. 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 Processor -- start 2024-04-22T07:14:42.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- start start 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1934102a30 0x7f1934198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 0x7f1934198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934107c90 0x7f193419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f193410dc30 con 0x7f1934105360 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f193410dab0 con 0x7f1934107c90 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.346+0000 7f193bcc7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f193410ddb0 con 0x7f1934102a30 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 0x7f1934198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 0x7f1934198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34886/0 (socket says 172.21.15.113:34886) 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 -- 172.21.15.113:0/1147064801 learned_addr learned my addr 172.21.15.113:0/1147064801 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939a63640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1934102a30 0x7f1934198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f193a264640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934107c90 0x7f193419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 -- 172.21.15.113:0/1147064801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1934102a30 msgr2=0x7f1934198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1934102a30 0x7f1934198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 -- 172.21.15.113:0/1147064801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934107c90 msgr2=0x7f193419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934107c90 0x7f193419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 -- 172.21.15.113:0/1147064801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193419fe50 con 0x7f1934105360 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f193a264640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934107c90 0x7f193419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f1939262640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 0x7f1934198770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f192400e9f0 tx=0x7f192400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192400ccc0 con 0x7f1934105360 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19240054f0 con 0x7f1934105360 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19341a0140 con 0x7f1934105360 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924010690 con 0x7f1934105360 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.347+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a0640 con 0x7f1934105360 2024-04-22T07:14:42.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.348+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19340fdf10 con 0x7f1934105360 2024-04-22T07:14:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.349+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f19240046c0 con 0x7f1934105360 2024-04-22T07:14:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.350+0000 7f192affd640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f19200a3ca0 0x7f19200a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.350+0000 7f1939a63640 1 -- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f19200a3ca0 msgr2=0x7f19200a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.350+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f19240d0ad0 con 0x7f1934105360 2024-04-22T07:14:42.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.350+0000 7f1939a63640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f19200a3ca0 0x7f19200a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:14:42.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.354+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1924053ff0 con 0x7f1934105360 2024-04-22T07:14:42.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.537+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f193402d5c0 con 0x7f1934105360 2024-04-22T07:14:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.541+0000 7f192affd640 1 -- 172.21.15.113:0/1147064801 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f192409d4c0 con 0x7f1934105360 2024-04-22T07:14:42.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f19200a3ca0 msgr2=0x7f19200a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:14:42.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f19200a3ca0 0x7f19200a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 msgr2=0x7f1934198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 0x7f1934198770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f192400e9f0 tx=0x7f192400eeb0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 shutdown_connections 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f19200a3ca0 0x7f19200a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1934107c90 0x7f193419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934105360 0x7f1934198770 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 --2- 172.21.15.113:0/1147064801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1934102a30 0x7f1934198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 >> 172.21.15.113:0/1147064801 conn(0x7f19340fc420 msgr2=0x7f1934103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 shutdown_connections 2024-04-22T07:14:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.543+0000 7f193bcc7640 1 -- 172.21.15.113:0/1147064801 wait complete. 2024-04-22T07:14:42.551 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-mon[27933]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:14:42.551 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3476680410' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T07:14:42.552 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: ignoring --setuser ceph since I am not root 2024-04-22T07:14:42.552 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: ignoring --setgroup ceph since I am not root 2024-04-22T07:14:42.552 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:42.406+0000 7f1451def200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:14:42.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-22T07:14:42.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-mon[31198]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:14:42.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3476680410' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T07:14:42.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: ignoring --setuser ceph since I am not root 2024-04-22T07:14:42.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: ignoring --setgroup ceph since I am not root 2024-04-22T07:14:42.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-mgr[32586]: -- 172.21.15.133:0/223626900 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55736fc785a0 con 0x55736fc4cc00 2024-04-22T07:14:42.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:42.398+0000 7f7d550d9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:14:42.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-mon[31862]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:14:42.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3476680410' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T07:14:42.566 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: ignoring --setuser ceph since I am not root 2024-04-22T07:14:42.566 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: ignoring --setgroup ceph since I am not root 2024-04-22T07:14:42.566 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-mgr[33720]: -- 172.21.15.154:0/3502938149 <== mon.0 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560887b785a0 con 0x560887b4d000 2024-04-22T07:14:42.566 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:42.401+0000 7fe75ade3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 -- 172.21.15.113:0/1541937454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 msgr2=0x7f5324109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 --2- 172.21.15.113:0/1541937454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f5324109320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f531c009ca0 tx=0x7f531c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 -- 172.21.15.113:0/1541937454 shutdown_connections 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 --2- 172.21.15.113:0/1541937454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f5324109320 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 --2- 172.21.15.113:0/1541937454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5324102040 0x7f53241024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 --2- 172.21.15.113:0/1541937454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5324100e60 0x7f5324101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 -- 172.21.15.113:0/1541937454 >> 172.21.15.113:0/1541937454 conn(0x7f53240fc420 msgr2=0x7f53240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 -- 172.21.15.113:0/1541937454 shutdown_connections 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 -- 172.21.15.113:0/1541937454 wait complete. 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 Processor -- start 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.656+0000 7f532bfa4640 1 -- start start 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5324100e60 0x7f532406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5324102040 0x7f532419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f53241a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f532410cb30 con 0x7f5324102a00 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f532410c9b0 con 0x7f5324100e60 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f532410ccb0 con 0x7f5324102040 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f53241a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f53241a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34896/0 (socket says 172.21.15.113:34896) 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 -- 172.21.15.113:0/382694484 learned_addr learned my addr 172.21.15.113:0/382694484 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532953f640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5324102040 0x7f532419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f5329d40640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5324100e60 0x7f532406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 -- 172.21.15.113:0/382694484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5324102040 msgr2=0x7f532419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5324102040 0x7f532419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 -- 172.21.15.113:0/382694484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5324100e60 msgr2=0x7f532406a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5324100e60 0x7f532406a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 -- 172.21.15.113:0/382694484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53241a2080 con 0x7f5324102a00 2024-04-22T07:14:42.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532a541640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f53241a1890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f531c005cc0 tx=0x7f531c032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:42.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c041020 con 0x7f5324102a00 2024-04-22T07:14:42.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53241a22b0 con 0x7f5324102a00 2024-04-22T07:14:42.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f531c003e40 con 0x7f5324102a00 2024-04-22T07:14:42.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c039040 con 0x7f5324102a00 2024-04-22T07:14:42.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.657+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241a27c0 con 0x7f5324102a00 2024-04-22T07:14:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.659+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532404fce0 con 0x7f5324102a00 2024-04-22T07:14:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.659+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f531c044020 con 0x7f5324102a00 2024-04-22T07:14:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.660+0000 7f531affd640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f53100a3cd0 0x7f53100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.660+0000 7f5329d40640 1 -- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f53100a3cd0 msgr2=0x7f53100a6190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.660+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f531c0f28b0 con 0x7f5324102a00 2024-04-22T07:14:42.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.660+0000 7f5329d40640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f53100a3cd0 0x7f53100a6190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:14:42.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.664+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f531c04b020 con 0x7f5324102a00 2024-04-22T07:14:42.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:42.596+0000 7f1451def200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:14:42.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:42.704+0000 7f1451def200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:14:42.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.849+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f532402d5c0 con 0x7f5324102a00 2024-04-22T07:14:42.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.853+0000 7f531affd640 1 -- 172.21.15.113:0/382694484 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f532402d5c0 con 0x7f5324102a00 2024-04-22T07:14:42.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f53100a3cd0 msgr2=0x7f53100a6190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:14:42.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f53100a3cd0 0x7f53100a6190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 msgr2=0x7f53241a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f53241a1890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f531c005cc0 tx=0x7f531c032950 comp rx=0 tx=0).stop 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 shutdown_connections 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f53100a3cd0 0x7f53100a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5324102a00 0x7f53241a1890 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5324102040 0x7f532419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 --2- 172.21.15.113:0/382694484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5324100e60 0x7f532406a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 >> 172.21.15.113:0/382694484 conn(0x7f53240fc420 msgr2=0x7f5324105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 shutdown_connections 2024-04-22T07:14:42.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:42.856+0000 7f532bfa4640 1 -- 172.21.15.113:0/382694484 wait complete. 2024-04-22T07:14:42.864 INFO:teuthology.run_tasks:Running task kclient... 2024-04-22T07:14:42.890 INFO:tasks.kclient:Mounting kernel clients... 2024-04-22T07:14:42.890 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-22T07:14:42.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:42.890 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-22T07:14:42.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:42.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-22T07:14:42.973 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-04-22T07:14:42.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:14:42.974 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'rbytes': False, 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-22T07:14:42.975 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T07:14:42.975 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T07:14:42.975 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T07:14:42.988 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:42.581+0000 7f7d550d9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:14:42.989 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:42.683+0000 7f7d550d9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:14:42.989 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:42.990+0000 7f7d550d9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:14:42.991 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:42.586+0000 7fe75ade3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:14:42.991 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:42.687+0000 7fe75ade3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:14:42.991 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:42.993+0000 7fe75ade3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:14:43.231 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:43.024+0000 7f1451def200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:14:43.231 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:43.112+0000 7f1451def200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:14:43.263 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:14:43.314 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:43 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:43.075+0000 7f7d550d9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:14:43.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:43 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3476680410' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T07:14:43.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:43 smithi133 ceph-mon[31198]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T07:14:43.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:43 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1147064801' entity='client.admin' 2024-04-22T07:14:43.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:43 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/382694484' entity='client.admin' 2024-04-22T07:14:43.315 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:43 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:43.077+0000 7fe75ade3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:14:43.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:43 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3476680410' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T07:14:43.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:43 smithi154 ceph-mon[31862]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T07:14:43.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:43 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1147064801' entity='client.admin' 2024-04-22T07:14:43.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:43 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/382694484' entity='client.admin' 2024-04-22T07:14:43.513 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:43 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3476680410' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T07:14:43.513 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:43 smithi113 ceph-mon[27933]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T07:14:43.513 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:43 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1147064801' entity='client.admin' 2024-04-22T07:14:43.513 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:43 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/382694484' entity='client.admin' 2024-04-22T07:14:44.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:44.253+0000 7f7d550d9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:14:44.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:44.423+0000 7f7d550d9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:14:44.565 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:44.248+0000 7fe75ade3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:14:44.565 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:44.417+0000 7fe75ade3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:14:44.688 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:44.339+0000 7f1451def200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:14:44.688 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:44.512+0000 7f1451def200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:14:44.688 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:44.689+0000 7f1451def200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:14:44.868 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:44.587+0000 7fe75ade3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:14:44.877 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:44.594+0000 7f7d550d9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:14:44.877 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:44.879+0000 7f7d550d9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:14:45.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.038+0000 7fb960389700 1 -- 172.21.15.113:0/1037707188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 msgr2=0x7fb958101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:45.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.038+0000 7fb960389700 1 --2- 172.21.15.113:0/1037707188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb958101dc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb948009990 tx=0x7fb948009ca0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- 172.21.15.113:0/1037707188 shutdown_connections 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 --2- 172.21.15.113:0/1037707188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb958102300 0x7fb95810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 --2- 172.21.15.113:0/1037707188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb958107990 0x7fb958107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 --2- 172.21.15.113:0/1037707188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb958101dc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- 172.21.15.113:0/1037707188 >> 172.21.15.113:0/1037707188 conn(0x7fb9580788f0 msgr2=0x7fb958078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- 172.21.15.113:0/1037707188 shutdown_connections 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- 172.21.15.113:0/1037707188 wait complete. 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 Processor -- start 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- start start 2024-04-22T07:14:45.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb95819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb958102300 0x7fb95819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb958107990 0x7fb9581a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb958111c60 con 0x7fb958101940 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb958111ae0 con 0x7fb958107990 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb960389700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb958111de0 con 0x7fb958102300 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.039+0000 7fb95e125700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb95819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb95819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34910/0 (socket says 172.21.15.113:34910) 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 -- 172.21.15.113:0/3434277293 learned_addr learned my addr 172.21.15.113:0/3434277293 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:45.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e926700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb958107990 0x7fb9581a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95d924700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb958102300 0x7fb95819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 -- 172.21.15.113:0/3434277293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb958102300 msgr2=0x7fb95819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb958102300 0x7fb95819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 -- 172.21.15.113:0/3434277293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb958107990 msgr2=0x7fb9581a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb958107990 0x7fb9581a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 -- 172.21.15.113:0/3434277293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9581a5680 con 0x7fb958101940 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb95e125700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb95819a0c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d950 tx=0x7fb95400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954016070 con 0x7fb958101940 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9581a5850 con 0x7fb958101940 2024-04-22T07:14:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb95400fd60 con 0x7fb958101940 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95400e610 con 0x7fb958101940 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.040+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581a5db0 con 0x7fb958101940 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.041+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb958069730 con 0x7fb958101940 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.042+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fb954019070 con 0x7fb958101940 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.043+0000 7fb94f7fe700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 0x7fb9440a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.043+0000 7fb95d924700 1 -- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 msgr2=0x7fb9440a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.043+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb9540cfe20 con 0x7fb958101940 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.043+0000 7fb95d924700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 0x7fb9440a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:14:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.047+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb95409c810 con 0x7fb958101940 2024-04-22T07:14:45.160 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:44.870+0000 7fe75ade3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:14:45.161 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:44.961+0000 7fe75ade3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:14:45.173 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:44.971+0000 7f7d550d9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:14:45.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.244+0000 7fb95d924700 1 -- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 msgr2=0x7fb9440a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:45.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.244+0000 7fb95d924700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 0x7fb9440a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:14:45.242 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:44.991+0000 7f1451def200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:14:45.242 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:45.090+0000 7f1451def200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:14:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.301+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb95802df80 con 0x7fb958101940 2024-04-22T07:14:45.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.302+0000 7fb94f7fe700 1 -- 172.21.15.113:0/3434277293 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7fb95409c110 con 0x7fb958101940 2024-04-22T07:14:45.300 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 msgr2=0x7fb9440a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 0x7fb9440a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 msgr2=0x7fb95819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb95819a0c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d950 tx=0x7fb95400dc60 comp rx=0 tx=0).stop 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 shutdown_connections 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7fb9440a3c90 0x7fb9440a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb958107990 0x7fb9581a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb958101940 0x7fb95819a0c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 --2- 172.21.15.113:0/3434277293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb958102300 0x7fb95819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:45.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 >> 172.21.15.113:0/3434277293 conn(0x7fb9580788f0 msgr2=0x7fb9580ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:45.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 shutdown_connections 2024-04-22T07:14:45.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:45.304+0000 7fb960389700 1 -- 172.21.15.113:0/3434277293 wait complete. 2024-04-22T07:14:45.497 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3434277293' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T07:14:45.498 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:45.300+0000 7f1451def200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:14:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:45 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3434277293' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T07:14:45.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:45.174+0000 7f7d550d9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:14:45.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:45.369+0000 7f7d550d9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:14:45.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:45 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3434277293' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T07:14:45.565 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:45.163+0000 7fe75ade3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:14:45.566 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:45.357+0000 7fe75ade3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:14:45.834 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:45.499+0000 7f1451def200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:14:46.022 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T07:14:46.022 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T07:14:46.022 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-04-22T07:14:46.022 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T07:14:46.022 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T07:14:46.023 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T07:14:46.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T07:14:46.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T07:14:46.023 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T07:14:46.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:46.023 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-04-22T07:14:46.039 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T07:14:46.039 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T07:14:46.039 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T07:14:46.039 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:14:46.039 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42111sec preferred_lft 42111sec 2024-04-22T07:14:46.040 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::6e08:c44e:94c5:8ba4/64 scope link noprefixroute 2024-04-22T07:14:46.041 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:14:46.041 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T07:14:46.041 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:14:46.041 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-22T07:14:46.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2024-04-22T07:14:46.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2024-04-22T07:14:46.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2024-04-22T07:14:46.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T07:14:46.042 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-22T07:14:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:14:46.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:46.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:46.223 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T07:14:46.242 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:46.114+0000 7f1451def200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:14:46.247 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-04-22T07:14:46.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:46.249 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-04-22T07:14:46.313 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-04-22T07:14:46.313 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T07:14:46.313 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-22T07:14:46.314 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-22T07:14:46.314 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T07:14:46.314 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:14:46.314 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-22T07:14:46.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-22T07:14:46.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-22T07:14:46.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-22T07:14:46.314 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-22T07:14:46.315 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:45.967+0000 7f7d550d9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:14:46.315 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:45.965+0000 7fe75ade3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:14:46.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:14:46.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:46.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:46.587 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-22T07:14:46.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:46.602 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-04-22T07:14:46.658 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:14:46.658 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-22T07:14:46.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:14:46.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T07:14:46.658 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-22T07:14:46.678 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:46.387+0000 7f7d550d9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:14:46.679 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:46.481+0000 7f7d550d9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:14:46.682 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:46.390+0000 7fe75ade3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:14:46.682 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:46.485+0000 7fe75ade3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:14:46.682 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:46.684+0000 7fe75ade3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:14:46.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:14:46.765 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:46.573+0000 7f1451def200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:14:46.765 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:46.675+0000 7f1451def200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:14:46.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:46.776 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T07:14:46.776 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:14:46.776 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-22T07:14:46.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T07:14:46.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T07:14:46.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T07:14:46.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T07:14:46.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T07:14:46.777 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-22T07:14:46.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:14:46.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:46.917 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:14:46.917 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-04-22T07:14:46.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-04-22T07:14:46.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T07:14:46.918 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-04-22T07:14:46.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:14:46.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:46.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T07:14:46.997 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T07:14:47.013 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T07:14:47.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T07:14:47.013 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:14:47.067 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:46.680+0000 7f7d550d9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:14:47.073 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-22T07:14:47.073 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-22T07:14:47.073 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/9a5b95cc-e4a5-4f02-a5dd-c51dfbfd1ae4 2024-04-22T07:14:47.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:47.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/9a5b95cc-e4a5-4f02-a5dd-c51dfbfd1ae4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-22T07:14:47.087 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:46.897+0000 7f1451def200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:14:47.147 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-22T07:14:47.147 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-22T07:14:47.147 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-22T07:14:47.258 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-22T07:14:47.258 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.133:3300,172.21.15.154:3300" 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.113:3300,172.21.15.133:3300,172.21.15.154:3300:/volumes/qa/sv_1/9a5b95cc-e4a5-4f02-a5dd-c51dfbfd1ae4 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=81035712-0076-11ef-bc93-c7b262605968" will pass to kernel 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout: You just mounted a file system that supports labels which does not 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-22T07:14:47.259 INFO:teuthology.orchestra.run.smithi113.stdout: applications will generate AVC messages and not be allowed access to 2024-04-22T07:14:47.260 INFO:teuthology.orchestra.run.smithi113.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-22T07:14:47.261 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-22T07:14:47.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:47.262 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:14:47.290 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T07:14:47.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T07:14:47.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:47.385 INFO:teuthology.orchestra.run.smithi113.stdout:client24917 2024-04-22T07:14:47.386 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:14:47.386 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:14:47.405 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T07:14:47.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T07:14:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:47.503 INFO:teuthology.orchestra.run.smithi113.stdout:client24917 2024-04-22T07:14:47.504 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T07:14:47.539 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:47.206+0000 7f7d550d9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:14:47.539 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:47.300+0000 7f7d550d9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:14:47.540 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:47.208+0000 7fe75ade3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:14:47.540 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:47.301+0000 7fe75ade3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:14:47.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T07:14:47.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:47.560 INFO:teuthology.orchestra.run.smithi113.stdout:client24917 2024-04-22T07:14:47.560 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/81035712-0076-11ef-bc93-c7b262605968.client24917/status') 2024-04-22T07:14:47.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:14:47.618 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.24917 (3)172.21.15.113:0/3249358072 2024-04-22T07:14:47.618 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2024-04-22T07:14:47.618 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2024-04-22T07:14:47.618 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2024-04-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi113.stderr:72 bytes copied, 2.3045e-05 s, 3.1 MB/s 2024-04-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:14:47.626 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T07:14:47.637 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T07:14:47.637 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:14:47.637 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-22T07:14:47.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 -- 172.21.15.113:0/1109366985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac1031d0 msgr2=0x7f8fac103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:47.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/1109366985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac1031d0 0x7f8fac103650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009900 tx=0x7f8f9c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:47.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 -- 172.21.15.113:0/1109366985 shutdown_connections 2024-04-22T07:14:47.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/1109366985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fac103b90 0x7f8fac10a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:47.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/1109366985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac1031d0 0x7f8fac103650 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/1109366985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fac101fe0 0x7f8fac102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 -- 172.21.15.113:0/1109366985 >> 172.21.15.113:0/1109366985 conn(0x7f8fac0fd530 msgr2=0x7f8fac0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 -- 172.21.15.113:0/1109366985 shutdown_connections 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.740+0000 7f8fb23f4640 1 -- 172.21.15.113:0/1109366985 wait complete. 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 Processor -- start 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 -- start start 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 0x7f8fac1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fac1031d0 0x7f8fac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fac103b90 0x7f8fac19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac10dc40 con 0x7f8fac101fe0 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac10dac0 con 0x7f8fac1031d0 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb23f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac10ddc0 con 0x7f8fac103b90 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 0x7f8fac1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 0x7f8fac1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52632/0 (socket says 172.21.15.113:52632) 2024-04-22T07:14:47.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 -- 172.21.15.113:0/2862616526 learned_addr learned my addr 172.21.15.113:0/2862616526 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fab7fe640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fac1031d0 0x7f8fac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fb0991640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fac103b90 0x7f8fac19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 -- 172.21.15.113:0/2862616526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fac103b90 msgr2=0x7f8fac19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fac103b90 0x7f8fac19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 -- 172.21.15.113:0/2862616526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fac1031d0 msgr2=0x7f8fac198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fac1031d0 0x7f8fac198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.741+0000 7f8fabfff640 1 -- 172.21.15.113:0/2862616526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac19fe10 con 0x7f8fac101fe0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.742+0000 7f8fabfff640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 0x7f8fac1981f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9c0 tx=0x7f8fa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.742+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000cd80 con 0x7f8fac101fe0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.742+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac1a0100 con 0x7f8fac101fe0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.742+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8fa00035f0 con 0x7f8fac101fe0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.742+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0010690 con 0x7f8fac101fe0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.742+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a0670 con 0x7f8fac101fe0 2024-04-22T07:14:47.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.743+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac1092a0 con 0x7f8fac101fe0 2024-04-22T07:14:47.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.744+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f8fa00037a0 con 0x7f8fac101fe0 2024-04-22T07:14:47.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.745+0000 7f8fa97fa640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 0x7f8f940a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:47.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.745+0000 7f8fab7fe640 1 -- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 msgr2=0x7f8f940a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:47.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.745+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8fa00d0920 con 0x7f8fac101fe0 2024-04-22T07:14:47.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.745+0000 7f8fab7fe640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 0x7f8f940a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:14:47.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.749+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fa009d380 con 0x7f8fac101fe0 2024-04-22T07:14:47.811 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:47.452+0000 7f1451def200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:14:47.812 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:47.551+0000 7f1451def200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:14:47.813 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:47.541+0000 7f7d550d9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:14:47.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:47.672+0000 7f7d550d9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:14:47.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:47.767+0000 7f7d550d9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:14:47.815 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:47.542+0000 7fe75ade3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:14:47.815 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:47.673+0000 7fe75ade3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:14:47.815 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:47.767+0000 7fe75ade3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:14:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.946+0000 7f8fab7fe640 1 -- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 msgr2=0x7f8f940a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:47.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.946+0000 7f8fab7fe640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 0x7f8f940a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:14:47.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:47.973+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f8fac02d5c0 con 0x7f8fac101fe0 2024-04-22T07:14:48.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.025+0000 7f8fa97fa640 1 -- 172.21.15.113:0/2862616526 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f8fa009cc80 con 0x7f8fac101fe0 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 msgr2=0x7f8f940a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 0x7f8f940a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 msgr2=0x7f8fac1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 0x7f8fac1981f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9c0 tx=0x7f8fa000ee80 comp rx=0 tx=0).stop 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 shutdown_connections 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f8f940a3ca0 0x7f8f940a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fac103b90 0x7f8fac19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fac1031d0 0x7f8fac198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 --2- 172.21.15.113:0/2862616526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fac101fe0 0x7f8fac1981f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 >> 172.21.15.113:0/2862616526 conn(0x7f8fac0fd530 msgr2=0x7f8fac106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 shutdown_connections 2024-04-22T07:14:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.027+0000 7f8fb23f4640 1 -- 172.21.15.113:0/2862616526 wait complete. 2024-04-22T07:14:48.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-22T07:14:48.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:48 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2862616526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T07:14:48.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:47.813+0000 7f1451def200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:14:48.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:47.954+0000 7f1451def200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:14:48.084 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:48.056+0000 7f1451def200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- 172.21.15.113:0/3216494958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0102af0 msgr2=0x7f96e0104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- 172.21.15.113:0/3216494958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0102af0 0x7f96e0104ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009900 tx=0x7f96d402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- 172.21.15.113:0/3216494958 shutdown_connections 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- 172.21.15.113:0/3216494958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e0107d50 0x7f96e010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- 172.21.15.113:0/3216494958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0105420 0x7f96e0107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- 172.21.15.113:0/3216494958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0102af0 0x7f96e0104ee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- 172.21.15.113:0/3216494958 >> 172.21.15.113:0/3216494958 conn(0x7f96e00fc420 msgr2=0x7f96e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- 172.21.15.113:0/3216494958 shutdown_connections 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- 172.21.15.113:0/3216494958 wait complete. 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 Processor -- start 2024-04-22T07:14:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- start start 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e0102af0 0x7f96e0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0105420 0x7f96e01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0107d50 0x7f96e019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010dc20 con 0x7f96e0105420 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010daa0 con 0x7f96e0107d50 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.143+0000 7f96e6ad2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010dda0 con 0x7f96e0102af0 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0105420 0x7f96e01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0105420 0x7f96e01987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52660/0 (socket says 172.21.15.113:52660) 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 -- 172.21.15.113:0/4149631421 learned_addr learned my addr 172.21.15.113:0/4149631421 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96e486e640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e0102af0 0x7f96e0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96e506f640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0107d50 0x7f96e019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e0102af0 msgr2=0x7f96e0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e0102af0 0x7f96e0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0107d50 msgr2=0x7f96e019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0107d50 0x7f96e019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 -- 172.21.15.113:0/4149631421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e019fe90 con 0x7f96e0105420 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96dffff640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0105420 0x7f96e01987b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00e9a0 tx=0x7f96cc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00cd80 con 0x7f96e0105420 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f96cc0035f0 con 0x7f96e0105420 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc010690 con 0x7f96e0105420 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.144+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e01a0180 con 0x7f96e0105420 2024-04-22T07:14:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.145+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e01a0720 con 0x7f96e0105420 2024-04-22T07:14:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.146+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e00fdf10 con 0x7f96e0105420 2024-04-22T07:14:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.146+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f96cc0037a0 con 0x7f96e0105420 2024-04-22T07:14:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.147+0000 7f96ddffb640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 0x7f96d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.147+0000 7f96e486e640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 msgr2=0x7f96d00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.147+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f96cc0d0620 con 0x7f96e0105420 2024-04-22T07:14:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.147+0000 7f96e486e640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 0x7f96d00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:14:48.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.151+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96cc09d010 con 0x7f96e0105420 2024-04-22T07:14:48.182 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:48 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2862616526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T07:14:48.182 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:47.881+0000 7fe75ade3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:14:48.182 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:48.184+0000 7fe75ade3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:14:48.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2862616526' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T07:14:48.186 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:47.881+0000 7f7d550d9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:14:48.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.348+0000 7f96e486e640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 msgr2=0x7f96d00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:48.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.348+0000 7f96e486e640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 0x7f96d00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:14:48.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.380+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 --> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f96e0062430 con 0x7f96d00a3d20 2024-04-22T07:14:48.506 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:48.198+0000 7f1451def200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:14:48.506 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:48.508+0000 7f1451def200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:14:48.564 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:48.187+0000 7f7d550d9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:14:48.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.749+0000 7f96e486e640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 msgr2=0x7f96d00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.113:6800/610623186 2024-04-22T07:14:48.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:48.749+0000 7f96e486e640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 0x7f96d00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:14:49.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:49.044+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 20) ==== 138664+0+0 (secure 0 0 0) 0x7f96cc098f80 con 0x7f96e0105420 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:48.789+0000 7f7d550d9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y[32582]: 2024-04-22T07:14:49.002+0000 7f7d550d9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2862616526' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: Standby manager daemon z restarted 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: Standby manager daemon z started 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T07:14:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: Standby manager daemon y restarted 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: Standby manager daemon y started 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:49 smithi133 ceph-mon[31198]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:14:49.065 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:48.784+0000 7fe75ade3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:14:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z[33716]: 2024-04-22T07:14:48.997+0000 7fe75ade3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2862616526' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: Standby manager daemon z restarted 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: Standby manager daemon z started 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:14:49.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: Standby manager daemon y restarted 2024-04-22T07:14:49.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: Standby manager daemon y started 2024-04-22T07:14:49.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T07:14:49.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:14:49.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T07:14:49.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:49 smithi154 ceph-mon[31862]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:14:49.334 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.121+0000 7f1451def200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:14:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2862616526' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: Standby manager daemon z restarted 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: Standby manager daemon z started 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.154:0/795295832' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: Standby manager daemon y restarted 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: Standby manager daemon y started 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T07:14:49.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-mon[27933]: from='mgr.? 172.21.15.133:0/2775288775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T07:14:49.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:49.359+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f96cc0430a0 con 0x7f96e0105420 2024-04-22T07:14:49.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:49.359+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 msgr2=0x7f96d00a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T07:14:49.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:49.359+0000 7f96ddffb640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/610623186,v1:172.21.15.113:6801/610623186] conn(0x7f96d00a3d20 0x7f96d00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.344+0000 7f1451def200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b0f85700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b0f85700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b0f85700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b0f85700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b0f85700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.597 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b378a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.598 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b378a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.598 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b378a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.598 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b378a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:49.598 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:49.587+0000 7f13b378a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:50.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: Active manager daemon x restarted 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: Activating manager daemon x 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: mgrmap e21: x(active, starting, since 0.013937s), standbys: z, y 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T07:14:50.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:14:50.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: Manager daemon x is now available 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T07:14:50.317 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:50 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: Active manager daemon x restarted 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: Activating manager daemon x 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: mgrmap e21: x(active, starting, since 0.013937s), standbys: z, y 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:14:50.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:14:50.319 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:14:50.320 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T07:14:50.321 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T07:14:50.321 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T07:14:50.321 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: Manager daemon x is now available 2024-04-22T07:14:50.321 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:14:50.321 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T07:14:50.321 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:50 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T07:14:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-22T07:14:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: Active manager daemon x restarted 2024-04-22T07:14:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: Activating manager daemon x 2024-04-22T07:14:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T07:14:50.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: mgrmap e21: x(active, starting, since 0.013937s), standbys: z, y 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:14:50.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T07:14:50.337 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: Manager daemon x is now available 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T07:14:50.338 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T07:14:50.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.363+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mon.0 v2:172.21.15.113:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f96cc0781f0 con 0x7f96e0105420 2024-04-22T07:14:50.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.363+0000 7f96ddffb640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96d0035660 0x7f96d0035a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.364+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f96cc01be80 con 0x7f96d0035660 2024-04-22T07:14:50.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.365+0000 7f96e486e640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96d0035660 0x7f96d0035a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.366+0000 7f96e486e640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96d0035660 0x7f96d0035a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d4005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:50.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.387+0000 7f96ddffb640 1 -- 172.21.15.113:0/4149631421 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f96cc01be80 con 0x7f96d0035660 2024-04-22T07:14:50.385 INFO:teuthology.orchestra.run.smithi113.stdout:{} 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96d0035660 msgr2=0x7f96d0035a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96d0035660 0x7f96d0035a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d4005850 comp rx=0 tx=0).stop 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0105420 msgr2=0x7f96e01987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 --2- 172.21.15.113:0/4149631421 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96e0105420 0x7f96e01987b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00e9a0 tx=0x7f96cc00ee60 comp rx=0 tx=0).stop 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e506f640 1 -- 172.21.15.113:0/4149631421 reap_dead start 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 shutdown_connections 2024-04-22T07:14:50.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 >> 172.21.15.113:0/4149631421 conn(0x7f96e00fc420 msgr2=0x7f96e0103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:50.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 shutdown_connections 2024-04-22T07:14:50.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.389+0000 7f96e6ad2640 1 -- 172.21.15.113:0/4149631421 wait complete. 2024-04-22T07:14:50.398 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-22T07:14:50.410 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-22T07:14:50.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3105771400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd901029f0 msgr2=0x7fdd90104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3105771400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd901029f0 0x7fdd90104de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009900 tx=0x7fdd8002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3105771400 shutdown_connections 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3105771400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd90107c50 0x7fdd9010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3105771400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd90105320 0x7fdd90107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3105771400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd901029f0 0x7fdd90104de0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.585+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3105771400 >> 172.21.15.113:0/3105771400 conn(0x7fdd900fc3e0 msgr2=0x7fdd900fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3105771400 shutdown_connections 2024-04-22T07:14:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3105771400 wait complete. 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 Processor -- start 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 -- start start 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd901029f0 0x7fdd90198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 0x7fdd901987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd90107c50 0x7fdd9019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9010dbf0 con 0x7fdd90105320 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9010da70 con 0x7fdd901029f0 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.586+0000 7fdd94dc7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9010dd70 con 0x7fdd90107c50 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 0x7fdd901987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 0x7fdd901987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52806/0 (socket says 172.21.15.113:52806) 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 -- 172.21.15.113:0/3828200793 learned_addr learned my addr 172.21.15.113:0/3828200793 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8e59c640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd901029f0 0x7fdd90198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8ed9d640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd90107c50 0x7fdd9019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 -- 172.21.15.113:0/3828200793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd90107c50 msgr2=0x7fdd9019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd90107c50 0x7fdd9019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 -- 172.21.15.113:0/3828200793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd901029f0 msgr2=0x7fdd90198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd901029f0 0x7fdd90198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 -- 172.21.15.113:0/3828200793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9019feb0 con 0x7fdd90105320 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd8dd9b640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 0x7fdd901987d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c970 tx=0x7fdd8400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84007bb0 con 0x7fdd90105320 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd84007d50 con 0x7fdd90105320 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd901a01a0 con 0x7fdd90105320 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84018780 con 0x7fdd90105320 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.587+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a06a0 con 0x7fdd90105320 2024-04-22T07:14:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.588+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd900fded0 con 0x7fdd90105320 2024-04-22T07:14:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.589+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fdd8400fa40 con 0x7fdd90105320 2024-04-22T07:14:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.590+0000 7fdd7f7fe640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd780a39b0 0x7fdd780a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.590+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdd840d0120 con 0x7fdd90105320 2024-04-22T07:14:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.590+0000 7fdd8e59c640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd780a39b0 0x7fdd780a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.590+0000 7fdd8e59c640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd780a39b0 0x7fdd780a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd80005b50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:50.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.594+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd84014030 con 0x7fdd90105320 2024-04-22T07:14:50.629 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:50.375+0000 7f13ae780700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:50.629 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:50.375+0000 7f13ae780700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:50.629 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:50.375+0000 7f13ae780700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:50.629 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:50.375+0000 7f13ae780700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:50.629 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:14:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:14:50.375+0000 7f13ae780700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:14:50.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.780+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd90062b50 con 0x7fdd90105320 2024-04-22T07:14:50.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.781+0000 7fdd7f7fe640 1 -- 172.21.15.113:0/3828200793 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7fdd8409cd00 con 0x7fdd90105320 2024-04-22T07:14:50.779 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:50.780 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:50.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd780a39b0 msgr2=0x7fdd780a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd780a39b0 0x7fdd780a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd80005b50 comp rx=0 tx=0).stop 2024-04-22T07:14:50.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 msgr2=0x7fdd901987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 0x7fdd901987d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c970 tx=0x7fdd8400ce30 comp rx=0 tx=0).stop 2024-04-22T07:14:50.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 shutdown_connections 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd780a39b0 0x7fdd780a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd90107c50 0x7fdd9019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd90105320 0x7fdd901987d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 --2- 172.21.15.113:0/3828200793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd901029f0 0x7fdd90198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 >> 172.21.15.113:0/3828200793 conn(0x7fdd900fc3e0 msgr2=0x7fdd901034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 shutdown_connections 2024-04-22T07:14:50.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.783+0000 7fdd94dc7640 1 -- 172.21.15.113:0/3828200793 wait complete. 2024-04-22T07:14:50.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:50.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:50.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 -- 172.21.15.113:0/2045959819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 msgr2=0x7f91f0100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/2045959819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0100470 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009900 tx=0x7f91e002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 -- 172.21.15.113:0/2045959819 shutdown_connections 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/2045959819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91f01009b0 0x7f91f010a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/2045959819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0100470 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/2045959819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91f0103580 0x7f91f01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 -- 172.21.15.113:0/2045959819 >> 172.21.15.113:0/2045959819 conn(0x7f91f00fbb90 msgr2=0x7f91f00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 -- 172.21.15.113:0/2045959819 shutdown_connections 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 -- 172.21.15.113:0/2045959819 wait complete. 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 Processor -- start 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.899+0000 7f91f4cd2640 1 -- start start 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91f01009b0 0x7f91f01012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91f0103580 0x7f91f0101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010dc30 con 0x7f91f00ffff0 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010dab0 con 0x7f91f0103580 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010ddb0 con 0x7f91f01009b0 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52816/0 (socket says 172.21.15.113:52816) 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 -- 172.21.15.113:0/3698774781 learned_addr learned my addr 172.21.15.113:0/3698774781 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91edd9b640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91f01009b0 0x7f91f01012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91eed9d640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91f0103580 0x7f91f0101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 -- 172.21.15.113:0/3698774781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91f01009b0 msgr2=0x7f91f01012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91f01009b0 0x7f91f01012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 -- 172.21.15.113:0/3698774781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91f0103580 msgr2=0x7f91f0101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91f0103580 0x7f91f0101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 -- 172.21.15.113:0/3698774781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f0102090 con 0x7f91f00ffff0 2024-04-22T07:14:50.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91ee59c640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0102c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f91e400c970 tx=0x7f91e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4004220 con 0x7f91f00ffff0 2024-04-22T07:14:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f91e40043c0 con 0x7f91f00ffff0 2024-04-22T07:14:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e400f450 con 0x7f91f00ffff0 2024-04-22T07:14:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01a23b0 con 0x7f91f00ffff0 2024-04-22T07:14:50.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.900+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01a2880 con 0x7f91f00ffff0 2024-04-22T07:14:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.901+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f00691c0 con 0x7f91f00ffff0 2024-04-22T07:14:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.903+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f91e4003730 con 0x7f91f00ffff0 2024-04-22T07:14:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.903+0000 7f91df7fe640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91d80a39b0 0x7f91d80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.903+0000 7f91edd9b640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91d80a39b0 0x7f91d80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.903+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f91e40cfdc0 con 0x7f91f00ffff0 2024-04-22T07:14:50.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.904+0000 7f91edd9b640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91d80a39b0 0x7f91d80a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009230 tx=0x7f91e0005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:50.908+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91e4014030 con 0x7f91f00ffff0 2024-04-22T07:14:51.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.089+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91f0062b50 con 0x7f91f00ffff0 2024-04-22T07:14:51.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.090+0000 7f91df7fe640 1 -- 172.21.15.113:0/3698774781 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f91e409c350 con 0x7f91f00ffff0 2024-04-22T07:14:51.088 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91d80a39b0 msgr2=0x7f91d80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91d80a39b0 0x7f91d80a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009230 tx=0x7f91e0005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 msgr2=0x7f91f0102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0102c80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f91e400c970 tx=0x7f91e400ce30 comp rx=0 tx=0).stop 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 shutdown_connections 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91d80a39b0 0x7f91d80a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91f0103580 0x7f91f0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91f01009b0 0x7f91f01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 --2- 172.21.15.113:0/3698774781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91f00ffff0 0x7f91f0102c80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 >> 172.21.15.113:0/3698774781 conn(0x7f91f00fbb90 msgr2=0x7f91f0106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:51.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 shutdown_connections 2024-04-22T07:14:51.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.092+0000 7f91f4cd2640 1 -- 172.21.15.113:0/3698774781 wait complete. 2024-04-22T07:14:51.101 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:51.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 -- 172.21.15.113:0/3113647837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 msgr2=0x7f47c0107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/3113647837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c0107730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009900 tx=0x7f47bc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 -- 172.21.15.113:0/3113647837 shutdown_connections 2024-04-22T07:14:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/3113647837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 0x7f47c010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/3113647837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c0107730 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/3113647837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c0102a10 0x7f47c0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 -- 172.21.15.113:0/3113647837 >> 172.21.15.113:0/3113647837 conn(0x7f47c00fc400 msgr2=0x7f47c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:51.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 -- 172.21.15.113:0/3113647837 shutdown_connections 2024-04-22T07:14:51.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.216+0000 7f47c7fdb640 1 -- 172.21.15.113:0/3113647837 wait complete. 2024-04-22T07:14:51.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 Processor -- start 2024-04-22T07:14:51.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 -- start start 2024-04-22T07:14:51.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c0102a10 0x7f47c0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 0x7f47c019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f47c010dbe0 con 0x7f47c0105340 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f47c010da60 con 0x7f47c0102a10 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c7fdb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f47c010dd60 con 0x7f47c0107c70 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52836/0 (socket says 172.21.15.113:52836) 2024-04-22T07:14:51.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 -- 172.21.15.113:0/2171083290 learned_addr learned my addr 172.21.15.113:0/2171083290 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5d77640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c0102a10 0x7f47c0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c6578640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 0x7f47c019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 -- 172.21.15.113:0/2171083290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 msgr2=0x7f47c019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 0x7f47c019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 -- 172.21.15.113:0/2171083290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c0102a10 msgr2=0x7f47c0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c0102a10 0x7f47c0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c5576640 1 -- 172.21.15.113:0/2171083290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c019fea0 con 0x7f47c0105340 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.217+0000 7f47c6578640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 0x7f47c019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.218+0000 7f47c5576640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c01987c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47bc036040 tx=0x7f47bc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.218+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc041070 con 0x7f47c0105340 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.218+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c01a0130 con 0x7f47c0105340 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.218+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f47bc0052f0 con 0x7f47c0105340 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.218+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc003d80 con 0x7f47c0105340 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.218+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c01a06d0 con 0x7f47c0105340 2024-04-22T07:14:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.219+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c00fdef0 con 0x7f47c0105340 2024-04-22T07:14:51.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.220+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f47bc044070 con 0x7f47c0105340 2024-04-22T07:14:51.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.221+0000 7f47b6ffd640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f47ac0a39b0 0x7f47ac0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.221+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f47bc0f2060 con 0x7f47c0105340 2024-04-22T07:14:51.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.221+0000 7f47c5d77640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f47ac0a39b0 0x7f47ac0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.221+0000 7f47c5d77640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f47ac0a39b0 0x7f47ac0a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009bd0 tx=0x7f47b0005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:51.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.225+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47bc0bec40 con 0x7f47c0105340 2024-04-22T07:14:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:51 smithi133 ceph-mon[31198]: [22/Apr/2024:07:14:49] ENGINE Bus STARTING 2024-04-22T07:14:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:51 smithi133 ceph-mon[31198]: [22/Apr/2024:07:14:49] ENGINE Serving on https://172.21.15.113:7150 2024-04-22T07:14:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:51 smithi133 ceph-mon[31198]: [22/Apr/2024:07:14:49] ENGINE Serving on http://172.21.15.113:8765 2024-04-22T07:14:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:51 smithi133 ceph-mon[31198]: [22/Apr/2024:07:14:49] ENGINE Bus STARTED 2024-04-22T07:14:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:51 smithi133 ceph-mon[31198]: mgrmap e22: x(active, since 1.02021s), standbys: z, y 2024-04-22T07:14:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:51 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3828200793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:51 smithi154 ceph-mon[31862]: [22/Apr/2024:07:14:49] ENGINE Bus STARTING 2024-04-22T07:14:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:51 smithi154 ceph-mon[31862]: [22/Apr/2024:07:14:49] ENGINE Serving on https://172.21.15.113:7150 2024-04-22T07:14:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:51 smithi154 ceph-mon[31862]: [22/Apr/2024:07:14:49] ENGINE Serving on http://172.21.15.113:8765 2024-04-22T07:14:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:51 smithi154 ceph-mon[31862]: [22/Apr/2024:07:14:49] ENGINE Bus STARTED 2024-04-22T07:14:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:51 smithi154 ceph-mon[31862]: mgrmap e22: x(active, since 1.02021s), standbys: z, y 2024-04-22T07:14:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:51 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3828200793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:51 smithi113 ceph-mon[27933]: [22/Apr/2024:07:14:49] ENGINE Bus STARTING 2024-04-22T07:14:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:51 smithi113 ceph-mon[27933]: [22/Apr/2024:07:14:49] ENGINE Serving on https://172.21.15.113:7150 2024-04-22T07:14:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:51 smithi113 ceph-mon[27933]: [22/Apr/2024:07:14:49] ENGINE Serving on http://172.21.15.113:8765 2024-04-22T07:14:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:51 smithi113 ceph-mon[27933]: [22/Apr/2024:07:14:49] ENGINE Bus STARTED 2024-04-22T07:14:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:51 smithi113 ceph-mon[27933]: mgrmap e22: x(active, since 1.02021s), standbys: z, y 2024-04-22T07:14:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:51 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3828200793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:51.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.429+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47c002d7c0 con 0x7f47c0105340 2024-04-22T07:14:51.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.429+0000 7f47b6ffd640 1 -- 172.21.15.113:0/2171083290 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f47bc0be540 con 0x7f47c0105340 2024-04-22T07:14:51.427 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f47ac0a39b0 msgr2=0x7f47ac0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f47ac0a39b0 0x7f47ac0a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009bd0 tx=0x7f47b0005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 msgr2=0x7f47c01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c01987c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f47bc036040 tx=0x7f47bc005dd0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 shutdown_connections 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f47ac0a39b0 0x7f47ac0a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47c0107c70 0x7f47c019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c0105340 0x7f47c01987c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 --2- 172.21.15.113:0/2171083290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c0102a10 0x7f47c0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 >> 172.21.15.113:0/2171083290 conn(0x7f47c00fc400 msgr2=0x7f47c0103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:51.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 shutdown_connections 2024-04-22T07:14:51.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.432+0000 7f47c7fdb640 1 -- 172.21.15.113:0/2171083290 wait complete. 2024-04-22T07:14:51.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:51.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 -- 172.21.15.113:0/1223084621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1107d0 msgr2=0x7f6efc112bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1223084621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1107d0 0x7f6efc112bc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009cd0 tx=0x7f6ef402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 -- 172.21.15.113:0/1223084621 shutdown_connections 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1223084621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1107d0 0x7f6efc112bc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1223084621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1098b0 0x7f6efc110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1223084621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 0x7f6efc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.556+0000 7f6f041b2640 1 -- 172.21.15.113:0/1223084621 >> 172.21.15.113:0/1223084621 conn(0x7f6efc06ede0 msgr2=0x7f6efc071220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 -- 172.21.15.113:0/1223084621 shutdown_connections 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 -- 172.21.15.113:0/1223084621 wait complete. 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 Processor -- start 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 -- start start 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 0x7f6efc10bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 0x7f6efc10a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1107d0 0x7f6efc10a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc116410 con 0x7f6efc1098b0 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc116290 con 0x7f6efc1107d0 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f041b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc116590 con 0x7f6efc108f50 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f0174d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 0x7f6efc10a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f0174d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 0x7f6efc10a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52880/0 (socket says 172.21.15.113:52880) 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f0174d640 1 -- 172.21.15.113:0/1216424686 learned_addr learned my addr 172.21.15.113:0/1216424686 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f01f4e640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 0x7f6efc10bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.557+0000 7f6f0274f640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1107d0 0x7f6efc10a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0174d640 1 -- 172.21.15.113:0/1216424686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 msgr2=0x7f6efc10bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0174d640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 0x7f6efc10bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0174d640 1 -- 172.21.15.113:0/1216424686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1107d0 msgr2=0x7f6efc10a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0174d640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1107d0 0x7f6efc10a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0174d640 1 -- 172.21.15.113:0/1216424686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc10aff0 con 0x7f6efc1098b0 2024-04-22T07:14:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0274f640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1107d0 0x7f6efc10a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f01f4e640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 0x7f6efc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f0174d640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 0x7f6efc10a220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00e9c0 tx=0x7f6eec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec00cd80 con 0x7f6efc1098b0 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6eec0035f0 con 0x7f6efc1098b0 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc1aac10 con 0x7f6efc1098b0 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec010690 con 0x7f6efc1098b0 2024-04-22T07:14:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.558+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1ab160 con 0x7f6efc1098b0 2024-04-22T07:14:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.560+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6efc10f990 con 0x7f6efc1098b0 2024-04-22T07:14:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.561+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f6eec0037a0 con 0x7f6efc1098b0 2024-04-22T07:14:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.561+0000 7f6ef2ffd640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6ee80a39b0 0x7f6ee80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.561+0000 7f6f01f4e640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6ee80a39b0 0x7f6ee80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.561+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f6eec0d0550 con 0x7f6efc1098b0 2024-04-22T07:14:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.562+0000 7f6f01f4e640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6ee80a39b0 0x7f6ee80a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009fc0 tx=0x7f6ef8005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:51.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.568+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6eec09d130 con 0x7f6efc1098b0 2024-04-22T07:14:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.755+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6efc062bf0 con 0x7f6efc1098b0 2024-04-22T07:14:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.756+0000 7f6ef2ffd640 1 -- 172.21.15.113:0/1216424686 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f6eec09ca30 con 0x7f6efc1098b0 2024-04-22T07:14:51.754 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6ee80a39b0 msgr2=0x7f6ee80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6ee80a39b0 0x7f6ee80a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009fc0 tx=0x7f6ef8005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 msgr2=0x7f6efc10a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 0x7f6efc10a220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00e9c0 tx=0x7f6eec00ee80 comp rx=0 tx=0).stop 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 shutdown_connections 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6ee80a39b0 0x7f6ee80a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6efc1107d0 0x7f6efc10a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6efc1098b0 0x7f6efc10a220 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 --2- 172.21.15.113:0/1216424686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6efc108f50 0x7f6efc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 >> 172.21.15.113:0/1216424686 conn(0x7f6efc06ede0 msgr2=0x7f6efc070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 shutdown_connections 2024-04-22T07:14:51.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.758+0000 7f6f041b2640 1 -- 172.21.15.113:0/1216424686 wait complete. 2024-04-22T07:14:51.768 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 -- 172.21.15.113:0/2830889670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84102980 msgr2=0x7fea841092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 --2- 172.21.15.113:0/2830889670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84102980 0x7fea841092a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009ca0 tx=0x7fea7c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 -- 172.21.15.113:0/2830889670 shutdown_connections 2024-04-22T07:14:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 --2- 172.21.15.113:0/2830889670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84102980 0x7fea841092a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 --2- 172.21.15.113:0/2830889670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea84101fc0 0x7fea84102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 --2- 172.21.15.113:0/2830889670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84100e70 0x7fea84101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 -- 172.21.15.113:0/2830889670 >> 172.21.15.113:0/2830889670 conn(0x7fea840fc3e0 msgr2=0x7fea840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 -- 172.21.15.113:0/2830889670 shutdown_connections 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.869+0000 7fea888a7640 1 -- 172.21.15.113:0/2830889670 wait complete. 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 Processor -- start 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 -- start start 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 0x7fea8419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea84101fc0 0x7fea8419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84102980 0x7fea841a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea8410cb10 con 0x7fea84100e70 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea8410c990 con 0x7fea84102980 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea888a7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea8410cc90 con 0x7fea84101fc0 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea8259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 0x7fea8419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea8259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 0x7fea8419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52894/0 (socket says 172.21.15.113:52894) 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea82d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84102980 0x7fea841a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea8259c640 1 -- 172.21.15.113:0/2478408112 learned_addr learned my addr 172.21.15.113:0/2478408112 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea81d9b640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea84101fc0 0x7fea8419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea82d9d640 1 -- 172.21.15.113:0/2478408112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea84101fc0 msgr2=0x7fea8419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea82d9d640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea84101fc0 0x7fea8419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea82d9d640 1 -- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 msgr2=0x7fea8419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea82d9d640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 0x7fea8419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea82d9d640 1 -- 172.21.15.113:0/2478408112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea841a20f0 con 0x7fea84102980 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.870+0000 7fea8259c640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 0x7fea8419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.871+0000 7fea82d9d640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84102980 0x7fea841a1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009bf0 tx=0x7fea7c0323b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.871+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c041020 con 0x7fea84102980 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.871+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea841a2380 con 0x7fea84102980 2024-04-22T07:14:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.871+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea7c0329a0 con 0x7fea84102980 2024-04-22T07:14:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.871+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c039040 con 0x7fea84102980 2024-04-22T07:14:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.871+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea841a2890 con 0x7fea84102980 2024-04-22T07:14:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.872+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea841a2af0 con 0x7fea84102980 2024-04-22T07:14:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.873+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fea7c044020 con 0x7fea84102980 2024-04-22T07:14:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.874+0000 7fea737fe640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea6c0a39b0 0x7fea6c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.874+0000 7fea8259c640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea6c0a39b0 0x7fea6c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:51.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.874+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fea7c0f2240 con 0x7fea84102980 2024-04-22T07:14:51.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.874+0000 7fea8259c640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea6c0a39b0 0x7fea6c0a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea74011550 tx=0x7fea74005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:51.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:51.877+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea7c04b020 con 0x7fea84102980 2024-04-22T07:14:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.056+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7fea7c0bad50 con 0x7fea84102980 2024-04-22T07:14:52.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.086+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea841a2de0 con 0x7fea84102980 2024-04-22T07:14:52.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.088+0000 7fea737fe640 1 -- 172.21.15.113:0/2478408112 <== mon.2 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7fea7c0b6130 con 0x7fea84102980 2024-04-22T07:14:52.087 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:52.087 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea6c0a39b0 msgr2=0x7fea6c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea6c0a39b0 0x7fea6c0a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea74011550 tx=0x7fea74005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84102980 msgr2=0x7fea841a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84102980 0x7fea841a1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009bf0 tx=0x7fea7c0323b0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 shutdown_connections 2024-04-22T07:14:52.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea6c0a39b0 0x7fea6c0a5e70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea84102980 0x7fea841a1a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea84101fc0 0x7fea8419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 --2- 172.21.15.113:0/2478408112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea84100e70 0x7fea8419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 >> 172.21.15.113:0/2478408112 conn(0x7fea840fc3e0 msgr2=0x7fea840fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:52.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 shutdown_connections 2024-04-22T07:14:52.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.090+0000 7fea888a7640 1 -- 172.21.15.113:0/2478408112 wait complete. 2024-04-22T07:14:52.100 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:52.101 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:52.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 -- 172.21.15.113:0/103113387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 msgr2=0x7f3b34103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 --2- 172.21.15.113:0/103113387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b34103610 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009900 tx=0x7f3b2802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:52.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 -- 172.21.15.113:0/103113387 shutdown_connections 2024-04-22T07:14:52.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 --2- 172.21.15.113:0/103113387 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34103b50 0x7f3b3410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 --2- 172.21.15.113:0/103113387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b34103610 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 --2- 172.21.15.113:0/103113387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 0x7f3b341023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 -- 172.21.15.113:0/103113387 >> 172.21.15.113:0/103113387 conn(0x7f3b340fd530 msgr2=0x7f3b340ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 -- 172.21.15.113:0/103113387 shutdown_connections 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.207+0000 7f3b395f4640 1 -- 172.21.15.113:0/103113387 wait complete. 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 Processor -- start 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 -- start start 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 0x7f3b34198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b341987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34103b50 0x7f3b3419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410dc00 con 0x7f3b34103190 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410da80 con 0x7f3b34101fa0 2024-04-22T07:14:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.208+0000 7f3b395f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3410dd80 con 0x7f3b34103b50 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b341987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b341987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52912/0 (socket says 172.21.15.113:52912) 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 -- 172.21.15.113:0/491192955 learned_addr learned my addr 172.21.15.113:0/491192955 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b32ffd640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 0x7f3b34198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b337fe640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34103b50 0x7f3b3419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 -- 172.21.15.113:0/491192955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34103b50 msgr2=0x7f3b3419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34103b50 0x7f3b3419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 -- 172.21.15.113:0/491192955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 msgr2=0x7f3b34198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 0x7f3b34198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 -- 172.21.15.113:0/491192955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3419fe90 con 0x7f3b34103190 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b32ffd640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 0x7f3b34198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b327fc640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b341987b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005ab0 tx=0x7f3b28005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28041070 con 0x7f3b34103190 2024-04-22T07:14:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341a0120 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3b280052f0 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28003d20 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.209+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a06c0 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.211+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b34109340 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.211+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f3b28044070 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.212+0000 7f3b1bfff640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.212+0000 7f3b32ffd640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.212+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f3b280f22c0 con 0x7f3b34103190 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.212+0000 7f3b32ffd640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b34102ff0 tx=0x7f3b24005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:52.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.216+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b280be2e0 con 0x7f3b34103190 2024-04-22T07:14:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3698774781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2171083290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1216424686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3698774781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2171083290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1216424686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3698774781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2171083290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:52.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1216424686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:52.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.402+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b3402d770 con 0x7f3b34103190 2024-04-22T07:14:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.402+0000 7f3b1bfff640 1 -- 172.21.15.113:0/491192955 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f3b280c25e0 con 0x7f3b34103190 2024-04-22T07:14:52.402 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3b1c0a3db0 msgr2=0x7f3b1c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b34102ff0 tx=0x7f3b24005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 msgr2=0x7f3b341987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b341987b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005ab0 tx=0x7f3b28005dd0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 shutdown_connections 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3b1c0a3db0 0x7f3b1c0a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b34103b50 0x7f3b3419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b34103190 0x7f3b341987b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 --2- 172.21.15.113:0/491192955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b34101fa0 0x7f3b34198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 >> 172.21.15.113:0/491192955 conn(0x7f3b340fd530 msgr2=0x7f3b34106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 shutdown_connections 2024-04-22T07:14:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.405+0000 7f3b395f4640 1 -- 172.21.15.113:0/491192955 wait complete. 2024-04-22T07:14:52.418 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 -- 172.21.15.113:0/2505918430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103190 msgr2=0x7f4bec103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/2505918430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103190 0x7f4bec103610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 -- 172.21.15.113:0/2505918430 shutdown_connections 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/2505918430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bec103b50 0x7f4bec10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/2505918430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103190 0x7f4bec103610 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/2505918430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec101fa0 0x7f4bec1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 -- 172.21.15.113:0/2505918430 >> 172.21.15.113:0/2505918430 conn(0x7f4bec0fd530 msgr2=0x7f4bec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 -- 172.21.15.113:0/2505918430 shutdown_connections 2024-04-22T07:14:52.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.525+0000 7f4bf10eb640 1 -- 172.21.15.113:0/2505918430 wait complete. 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 Processor -- start 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 -- start start 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bec101fa0 0x7f4bec075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec103190 0x7f4bec074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 0x7f4bec074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10dc00 con 0x7f4bec103b50 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10da80 con 0x7f4bec103190 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bf10eb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec10dd80 con 0x7f4bec101fa0 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 0x7f4bec074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bea59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec103190 0x7f4bec074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 0x7f4bec074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52938/0 (socket says 172.21.15.113:52938) 2024-04-22T07:14:52.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 -- 172.21.15.113:0/1443539652 learned_addr learned my addr 172.21.15.113:0/1443539652 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bead9d640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bec101fa0 0x7f4bec075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 -- 172.21.15.113:0/1443539652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bec101fa0 msgr2=0x7f4bec075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bec101fa0 0x7f4bec075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 -- 172.21.15.113:0/1443539652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec103190 msgr2=0x7f4bec074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec103190 0x7f4bec074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4beb59e640 1 -- 172.21.15.113:0/1443539652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec075110 con 0x7f4bec103b50 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.526+0000 7f4bea59c640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec103190 0x7f4bec074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.527+0000 7f4beb59e640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 0x7f4bec074880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4be400e970 tx=0x7f4be400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.528+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be400ccc0 con 0x7f4bec103b50 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.528+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec1a2390 con 0x7f4bec103b50 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.528+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4be400ce60 con 0x7f4bec103b50 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.528+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4010430 con 0x7f4bec103b50 2024-04-22T07:14:52.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.528+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec1a28b0 con 0x7f4bec103b50 2024-04-22T07:14:52.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.529+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bec0684e0 con 0x7f4bec103b50 2024-04-22T07:14:52.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.530+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f4be40106d0 con 0x7f4bec103b50 2024-04-22T07:14:52.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.531+0000 7f4bd3fff640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4bd40a3db0 0x7f4bd40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.531+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f4be40d0e00 con 0x7f4bec103b50 2024-04-22T07:14:52.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.531+0000 7f4bead9d640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4bd40a3db0 0x7f4bd40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.531+0000 7f4bead9d640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4bd40a3db0 0x7f4bd40a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4bec102ff0 tx=0x7f4bdc009500 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:52.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.535+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4be409d360 con 0x7f4bec103b50 2024-04-22T07:14:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.727+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4bec02d5c0 con 0x7f4bec103b50 2024-04-22T07:14:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.727+0000 7f4bd3fff640 1 -- 172.21.15.113:0/1443539652 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f4be40bbe20 con 0x7f4bec103b50 2024-04-22T07:14:52.726 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4bd40a3db0 msgr2=0x7f4bd40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4bd40a3db0 0x7f4bd40a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4bec102ff0 tx=0x7f4bdc009500 comp rx=0 tx=0).stop 2024-04-22T07:14:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 msgr2=0x7f4bec074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 0x7f4bec074880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4be400e970 tx=0x7f4be400ed70 comp rx=0 tx=0).stop 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 shutdown_connections 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4bd40a3db0 0x7f4bd40a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bec103b50 0x7f4bec074880 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec103190 0x7f4bec074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 --2- 172.21.15.113:0/1443539652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4bec101fa0 0x7f4bec075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.729+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 >> 172.21.15.113:0/1443539652 conn(0x7f4bec0fd530 msgr2=0x7f4bec106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.730+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 shutdown_connections 2024-04-22T07:14:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.730+0000 7f4bf10eb640 1 -- 172.21.15.113:0/1443539652 wait complete. 2024-04-22T07:14:52.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:52.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 -- 172.21.15.113:0/3817460265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c107c70 msgr2=0x7f8d9c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3817460265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c107c70 0x7f8d9c10a0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009be0 tx=0x7f8d9402bef0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 -- 172.21.15.113:0/3817460265 shutdown_connections 2024-04-22T07:14:52.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3817460265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c107c70 0x7f8d9c10a0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3817460265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c105340 0x7f8d9c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3817460265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 -- 172.21.15.113:0/3817460265 >> 172.21.15.113:0/3817460265 conn(0x7f8d9c0fc400 msgr2=0x7f8d9c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 -- 172.21.15.113:0/3817460265 shutdown_connections 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.862+0000 7f8da2c48640 1 -- 172.21.15.113:0/3817460265 wait complete. 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 Processor -- start 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 -- start start 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c105340 0x7f8d9c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c107c70 0x7f8d9c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c10dc00 con 0x7f8d9c102a10 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c10da80 con 0x7f8d9c107c70 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da2c48640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c10dd80 con 0x7f8d9c105340 2024-04-22T07:14:52.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da09e4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da09e4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52956/0 (socket says 172.21.15.113:52956) 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da09e4640 1 -- 172.21.15.113:0/3123166683 learned_addr learned my addr 172.21.15.113:0/3123166683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8d9bfff640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c105340 0x7f8d9c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.863+0000 7f8da11e5640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c107c70 0x7f8d9c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da09e4640 1 -- 172.21.15.113:0/3123166683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c105340 msgr2=0x7f8d9c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da09e4640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c105340 0x7f8d9c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da09e4640 1 -- 172.21.15.113:0/3123166683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c107c70 msgr2=0x7f8d9c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da09e4640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c107c70 0x7f8d9c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da09e4640 1 -- 172.21.15.113:0/3123166683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c19fec0 con 0x7f8d9c102a10 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da09e4640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c1982a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000e9c0 tx=0x7f8d9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9000cde0 con 0x7f8d9c102a10 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c1a01b0 con 0x7f8d9c102a10 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8d900035f0 con 0x7f8d9c102a10 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90010670 con 0x7f8d9c102a10 2024-04-22T07:14:52.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.864+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c1a0750 con 0x7f8d9c102a10 2024-04-22T07:14:52.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.865+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c0fdef0 con 0x7f8d9c102a10 2024-04-22T07:14:52.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.866+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f8d900037a0 con 0x7f8d9c102a10 2024-04-22T07:14:52.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.867+0000 7f8d99ffb640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8d8c0a3db0 0x7f8d8c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:52.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.867+0000 7f8d9bfff640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8d8c0a3db0 0x7f8d8c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:52.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.867+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8d900d0ce0 con 0x7f8d9c102a10 2024-04-22T07:14:52.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.868+0000 7f8d9bfff640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8d8c0a3db0 0x7f8d8c0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d880095f0 tx=0x7f8d88005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:52.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:52.872+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d9009d1e0 con 0x7f8d9c102a10 2024-04-22T07:14:53.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.063+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d9c062b50 con 0x7f8d9c102a10 2024-04-22T07:14:53.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.064+0000 7f8d99ffb640 1 -- 172.21.15.113:0/3123166683 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f8d9009cae0 con 0x7f8d9c102a10 2024-04-22T07:14:53.063 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8d8c0a3db0 msgr2=0x7f8d8c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8d8c0a3db0 0x7f8d8c0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d880095f0 tx=0x7f8d88005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 msgr2=0x7f8d9c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c1982a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000e9c0 tx=0x7f8d9000ee80 comp rx=0 tx=0).stop 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 shutdown_connections 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8d8c0a3db0 0x7f8d8c0a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c107c70 0x7f8d9c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d9c105340 0x7f8d9c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 --2- 172.21.15.113:0/3123166683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d9c102a10 0x7f8d9c1982a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 >> 172.21.15.113:0/3123166683 conn(0x7f8d9c0fc400 msgr2=0x7f8d9c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:53.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 shutdown_connections 2024-04-22T07:14:53.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.066+0000 7f8da2c48640 1 -- 172.21.15.113:0/3123166683 wait complete. 2024-04-22T07:14:53.079 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-22T07:14:53.079 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.181+0000 7f898121f640 1 -- 172.21.15.113:0/3415210808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 msgr2=0x7f897c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.181+0000 7f898121f640 1 --2- 172.21.15.113:0/3415210808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c1092d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8974009ca0 tx=0x7f897402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 -- 172.21.15.113:0/3415210808 shutdown_connections 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 --2- 172.21.15.113:0/3415210808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c1092d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 --2- 172.21.15.113:0/3415210808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c101ff0 0x7f897c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 --2- 172.21.15.113:0/3415210808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c100ea0 0x7f897c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 -- 172.21.15.113:0/3415210808 >> 172.21.15.113:0/3415210808 conn(0x7f897c0fc410 msgr2=0x7f897c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 -- 172.21.15.113:0/3415210808 shutdown_connections 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 -- 172.21.15.113:0/3415210808 wait complete. 2024-04-22T07:14:53.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 Processor -- start 2024-04-22T07:14:53.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.182+0000 7f898121f640 1 -- start start 2024-04-22T07:14:53.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f898121f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c100ea0 0x7f897c06b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f898121f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c101ff0 0x7f897c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f898121f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f898121f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10cb20 con 0x7f897c1029b0 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f898121f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10c9a0 con 0x7f897c100ea0 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f898121f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10cca0 con 0x7f897c101ff0 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c101ff0 0x7f897c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c100ea0 0x7f897c06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c19f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52968/0 (socket says 172.21.15.113:52968) 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 -- 172.21.15.113:0/65863107 learned_addr learned my addr 172.21.15.113:0/65863107 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 -- 172.21.15.113:0/65863107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c101ff0 msgr2=0x7f897c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c101ff0 0x7f897c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 -- 172.21.15.113:0/65863107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c100ea0 msgr2=0x7f897c06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c100ea0 0x7f897c06b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 -- 172.21.15.113:0/65863107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c19fe10 con 0x7f897c1029b0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897a59c640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c101ff0 0x7f897c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.183+0000 7f897b59e640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c19f550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8974005cc0 tx=0x7f8974032950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.184+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974041020 con 0x7f897c1029b0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.184+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c1a00a0 con 0x7f897c1029b0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.184+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8974003e40 con 0x7f897c1029b0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.184+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974039040 con 0x7f897c1029b0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.184+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c1a0560 con 0x7f897c1029b0 2024-04-22T07:14:53.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.185+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c1081b0 con 0x7f897c1029b0 2024-04-22T07:14:53.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.186+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f8974044020 con 0x7f897c1029b0 2024-04-22T07:14:53.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.186+0000 7f8963fff640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f89640a3db0 0x7f89640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.187+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f89740f2a90 con 0x7f897c1029b0 2024-04-22T07:14:53.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.187+0000 7f897ad9d640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f89640a3db0 0x7f89640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.187+0000 7f897ad9d640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f89640a3db0 0x7f89640a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f896c009bd0 tx=0x7f896c005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:53.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.191+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f897404b020 con 0x7f897c1029b0 2024-04-22T07:14:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T07:14:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2478408112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/491192955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1443539652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2478408112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/491192955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:53.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1443539652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:53.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T07:14:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2478408112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/491192955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1443539652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:53.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:14:53.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:53.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:53.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.421+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f897c02d810 con 0x7f897c1029b0 2024-04-22T07:14:53.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.421+0000 7f8963fff640 1 -- 172.21.15.113:0/65863107 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f89740bf100 con 0x7f897c1029b0 2024-04-22T07:14:53.420 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:53.420 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:14:53.420 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:14:53.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f89640a3db0 msgr2=0x7f89640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f89640a3db0 0x7f89640a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f896c009bd0 tx=0x7f896c005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:53.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 msgr2=0x7f897c19f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c19f550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8974005cc0 tx=0x7f8974032950 comp rx=0 tx=0).stop 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 shutdown_connections 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f89640a3db0 0x7f89640a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f897c1029b0 0x7f897c19f550 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f897c101ff0 0x7f897c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 --2- 172.21.15.113:0/65863107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c100ea0 0x7f897c06b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.423+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 >> 172.21.15.113:0/65863107 conn(0x7f897c0fc410 msgr2=0x7f897c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.424+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 shutdown_connections 2024-04-22T07:14:53.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.424+0000 7f898121f640 1 -- 172.21.15.113:0/65863107 wait complete. 2024-04-22T07:14:53.433 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-22T07:14:53.433 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-22T07:14:53.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:14:53.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.535+0000 7f9d4dd15640 1 -- 172.21.15.113:0/657978835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 msgr2=0x7f9d48102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.535+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/657978835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d48102440 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009900 tx=0x7f9d3c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.535+0000 7f9d4dd15640 1 -- 172.21.15.113:0/657978835 shutdown_connections 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.535+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/657978835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48102980 0x7f9d481092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.535+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/657978835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d48102440 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.536+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/657978835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d48100e70 0x7f9d48101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.536+0000 7f9d4dd15640 1 -- 172.21.15.113:0/657978835 >> 172.21.15.113:0/657978835 conn(0x7f9d480fc3e0 msgr2=0x7f9d480fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.536+0000 7f9d4dd15640 1 -- 172.21.15.113:0/657978835 shutdown_connections 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.536+0000 7f9d4dd15640 1 -- 172.21.15.113:0/657978835 wait complete. 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.536+0000 7f9d4dd15640 1 Processor -- start 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.536+0000 7f9d4dd15640 1 -- start start 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d4dd15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48100e70 0x7f9d481971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d4dd15640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d481976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d4dd15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d48102980 0x7f9d4819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d4dd15640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810cb30 con 0x7f9d48101fc0 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d4dd15640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810c9b0 con 0x7f9d48100e70 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d4dd15640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810ccb0 con 0x7f9d48102980 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d481976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d481976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52990/0 (socket says 172.21.15.113:52990) 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d46ffd640 1 -- 172.21.15.113:0/132358840 learned_addr learned my addr 172.21.15.113:0/132358840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48100e70 0x7f9d481971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d47fff640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d48102980 0x7f9d4819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 -- 172.21.15.113:0/132358840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d48102980 msgr2=0x7f9d4819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d48102980 0x7f9d4819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 -- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 msgr2=0x7f9d481976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d481976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 -- 172.21.15.113:0/132358840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4819edc0 con 0x7f9d48100e70 2024-04-22T07:14:53.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d46ffd640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d481976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:53.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d477fe640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48100e70 0x7f9d481971a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800eb00 tx=0x7f9d3800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:53.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.537+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38010430 con 0x7f9d48100e70 2024-04-22T07:14:53.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.538+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4819f0b0 con 0x7f9d48100e70 2024-04-22T07:14:53.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.538+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9d38010aa0 con 0x7f9d48100e70 2024-04-22T07:14:53.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.538+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38017b30 con 0x7f9d48100e70 2024-04-22T07:14:53.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.538+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4819f600 con 0x7f9d48100e70 2024-04-22T07:14:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.538+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d48108180 con 0x7f9d48100e70 2024-04-22T07:14:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.540+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f9d38017d10 con 0x7f9d48100e70 2024-04-22T07:14:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.540+0000 7f9d44ff9640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d300a3db0 0x7f9d300a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.540+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9d380d1430 con 0x7f9d48100e70 2024-04-22T07:14:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.540+0000 7f9d46ffd640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d300a3db0 0x7f9d300a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.541+0000 7f9d46ffd640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d300a3db0 0x7f9d300a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d481986b0 tx=0x7f9d3c02c850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:53.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.544+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d3809d990 con 0x7f9d48100e70 2024-04-22T07:14:53.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.729+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d48062390 con 0x7f9d300a3db0 2024-04-22T07:14:53.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.737+0000 7f9d44ff9640 1 -- 172.21.15.113:0/132358840 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257738 (secure 0 0 0) 0x7f9d48062390 con 0x7f9d300a3db0 2024-04-22T07:14:53.739 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:53.740 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d300a3db0 msgr2=0x7f9d300a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d300a3db0 0x7f9d300a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d481986b0 tx=0x7f9d3c02c850 comp rx=0 tx=0).stop 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48100e70 msgr2=0x7f9d481971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48100e70 0x7f9d481971a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800eb00 tx=0x7f9d3800c2d0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 shutdown_connections 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d300a3db0 0x7f9d300a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d48102980 0x7f9d4819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d48101fc0 0x7f9d481976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 --2- 172.21.15.113:0/132358840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d48100e70 0x7f9d481971a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 >> 172.21.15.113:0/132358840 conn(0x7f9d480fc3e0 msgr2=0x7f9d48105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:53.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 shutdown_connections 2024-04-22T07:14:53.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.740+0000 7f9d4dd15640 1 -- 172.21.15.113:0/132358840 wait complete. 2024-04-22T07:14:53.750 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T07:14:53.366437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187876,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":929,"num_read_kb":618,"num_write":467,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368420,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573404,"statfs":{"total":1151940427776,"available":1151563165696,"internally_reserved":0,"allocated":15433728,"data_stored":9202524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990817"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679807+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:14:49.679807+0000","last_peered":"2024-04-22T07:14:49.679807+0000","last_clean":"2024-04-22T07:14:49.679807+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:14:49.679807+0000","last_undegraded":"2024-04-22T07:14:49.679807+0000","last_fullsized":"2024-04-22T07:14:49.679807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566911+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:14:49.566911+0000","last_peered":"2024-04-22T07:14:49.566911+0000","last_clean":"2024-04-22T07:14:49.566911+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:14:49.566911+0000","last_undegraded":"2024-04-22T07:14:49.566911+0000","last_fullsized":"2024-04-22T07:14:49.566911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566794+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:14:49.566794+0000","last_peered":"2024-04-22T07:14:49.566794+0000","last_clean":"2024-04-22T07:14:49.566794+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:14:49.566794+0000","last_undegraded":"2024-04-22T07:14:49.566794+0000","last_fullsized":"2024-04-22T07:14:49.566794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522113+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:14:49.522113+0000","last_peered":"2024-04-22T07:14:49.522113+0000","last_clean":"2024-04-22T07:14:49.522113+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:14:49.522113+0000","last_undegraded":"2024-04-22T07:14:49.522113+0000","last_fullsized":"2024-04-22T07:14:49.522113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446077+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:14:49.446077+0000","last_peered":"2024-04-22T07:14:49.446077+0000","last_clean":"2024-04-22T07:14:49.446077+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:14:49.446077+0000","last_undegraded":"2024-04-22T07:14:49.446077+0000","last_fullsized":"2024-04-22T07:14:49.446077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454433+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:14:49.454433+0000","last_peered":"2024-04-22T07:14:49.454433+0000","last_clean":"2024-04-22T07:14:49.454433+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:14:49.454433+0000","last_undegraded":"2024-04-22T07:14:49.454433+0000","last_fullsized":"2024-04-22T07:14:49.454433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353645+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:14:49.353645+0000","last_peered":"2024-04-22T07:14:49.353645+0000","last_clean":"2024-04-22T07:14:49.353645+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:14:49.353645+0000","last_undegraded":"2024-04-22T07:14:49.353645+0000","last_fullsized":"2024-04-22T07:14:49.353645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446353+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:14:49.446353+0000","last_peered":"2024-04-22T07:14:49.446353+0000","last_clean":"2024-04-22T07:14:49.446353+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:14:49.446353+0000","last_undegraded":"2024-04-22T07:14:49.446353+0000","last_fullsized":"2024-04-22T07:14:49.446353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518246+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:14:49.518246+0000","last_peered":"2024-04-22T07:14:49.518246+0000","last_clean":"2024-04-22T07:14:49.518246+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:14:49.518246+0000","last_undegraded":"2024-04-22T07:14:49.518246+0000","last_fullsized":"2024-04-22T07:14:49.518246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454790+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:14:49.454790+0000","last_peered":"2024-04-22T07:14:49.454790+0000","last_clean":"2024-04-22T07:14:49.454790+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:14:49.454790+0000","last_undegraded":"2024-04-22T07:14:49.454790+0000","last_fullsized":"2024-04-22T07:14:49.454790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353879+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:14:49.353879+0000","last_peered":"2024-04-22T07:14:49.353879+0000","last_clean":"2024-04-22T07:14:49.353879+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:14:49.353879+0000","last_undegraded":"2024-04-22T07:14:49.353879+0000","last_fullsized":"2024-04-22T07:14:49.353879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521981+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:14:49.521981+0000","last_peered":"2024-04-22T07:14:49.521981+0000","last_clean":"2024-04-22T07:14:49.521981+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:14:49.521981+0000","last_undegraded":"2024-04-22T07:14:49.521981+0000","last_fullsized":"2024-04-22T07:14:49.521981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520446+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:14:49.520446+0000","last_peered":"2024-04-22T07:14:49.520446+0000","last_clean":"2024-04-22T07:14:49.520446+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:14:49.520446+0000","last_undegraded":"2024-04-22T07:14:49.520446+0000","last_fullsized":"2024-04-22T07:14:49.520446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567028+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:14:49.567028+0000","last_peered":"2024-04-22T07:14:49.567028+0000","last_clean":"2024-04-22T07:14:49.567028+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:14:49.567028+0000","last_undegraded":"2024-04-22T07:14:49.567028+0000","last_fullsized":"2024-04-22T07:14:49.567028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566423+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:14:49.566423+0000","last_peered":"2024-04-22T07:14:49.566423+0000","last_clean":"2024-04-22T07:14:49.566423+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:14:49.566423+0000","last_undegraded":"2024-04-22T07:14:49.566423+0000","last_fullsized":"2024-04-22T07:14:49.566423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.517925+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:14:49.517925+0000","last_peered":"2024-04-22T07:14:49.517925+0000","last_clean":"2024-04-22T07:14:49.517925+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:14:49.517925+0000","last_undegraded":"2024-04-22T07:14:49.517925+0000","last_fullsized":"2024-04-22T07:14:49.517925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679946+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:14:49.679946+0000","last_peered":"2024-04-22T07:14:49.679946+0000","last_clean":"2024-04-22T07:14:49.679946+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:14:49.679946+0000","last_undegraded":"2024-04-22T07:14:49.679946+0000","last_fullsized":"2024-04-22T07:14:49.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354137+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:14:49.354137+0000","last_peered":"2024-04-22T07:14:49.354137+0000","last_clean":"2024-04-22T07:14:49.354137+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:14:49.354137+0000","last_undegraded":"2024-04-22T07:14:49.354137+0000","last_fullsized":"2024-04-22T07:14:49.354137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356040+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:14:49.356040+0000","last_peered":"2024-04-22T07:14:49.356040+0000","last_clean":"2024-04-22T07:14:49.356040+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:14:49.356040+0000","last_undegraded":"2024-04-22T07:14:49.356040+0000","last_fullsized":"2024-04-22T07:14:49.356040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354290+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:14:49.354290+0000","last_peered":"2024-04-22T07:14:49.354290+0000","last_clean":"2024-04-22T07:14:49.354290+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:14:49.354290+0000","last_undegraded":"2024-04-22T07:14:49.354290+0000","last_fullsized":"2024-04-22T07:14:49.354290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447115+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:14:49.447115+0000","last_peered":"2024-04-22T07:14:49.447115+0000","last_clean":"2024-04-22T07:14:49.447115+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:14:49.447115+0000","last_undegraded":"2024-04-22T07:14:49.447115+0000","last_fullsized":"2024-04-22T07:14:49.447115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447288+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:14:49.447288+0000","last_peered":"2024-04-22T07:14:49.447288+0000","last_clean":"2024-04-22T07:14:49.447288+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:14:49.447288+0000","last_undegraded":"2024-04-22T07:14:49.447288+0000","last_fullsized":"2024-04-22T07:14:49.447288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567072+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:14:49.567072+0000","last_peered":"2024-04-22T07:14:49.567072+0000","last_clean":"2024-04-22T07:14:49.567072+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:14:49.567072+0000","last_undegraded":"2024-04-22T07:14:49.567072+0000","last_fullsized":"2024-04-22T07:14:49.567072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446441+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:14:49.446441+0000","last_peered":"2024-04-22T07:14:49.446441+0000","last_clean":"2024-04-22T07:14:49.446441+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:14:49.446441+0000","last_undegraded":"2024-04-22T07:14:49.446441+0000","last_fullsized":"2024-04-22T07:14:49.446441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518065+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:14:49.518065+0000","last_peered":"2024-04-22T07:14:49.518065+0000","last_clean":"2024-04-22T07:14:49.518065+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:14:49.518065+0000","last_undegraded":"2024-04-22T07:14:49.518065+0000","last_fullsized":"2024-04-22T07:14:49.518065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679550+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:14:49.679550+0000","last_peered":"2024-04-22T07:14:49.679550+0000","last_clean":"2024-04-22T07:14:49.679550+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:14:49.679550+0000","last_undegraded":"2024-04-22T07:14:49.679550+0000","last_fullsized":"2024-04-22T07:14:49.679550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451783+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:14:49.451783+0000","last_peered":"2024-04-22T07:14:49.451783+0000","last_clean":"2024-04-22T07:14:49.451783+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:14:49.451783+0000","last_undegraded":"2024-04-22T07:14:49.451783+0000","last_fullsized":"2024-04-22T07:14:49.451783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.500064+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:49.500064+0000","last_peered":"2024-04-22T07:14:49.500064+0000","last_clean":"2024-04-22T07:14:49.500064+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:49.500064+0000","last_undegraded":"2024-04-22T07:14:49.500064+0000","last_fullsized":"2024-04-22T07:14:49.500064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446021+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:14:49.446021+0000","last_peered":"2024-04-22T07:14:49.446021+0000","last_clean":"2024-04-22T07:14:49.446021+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:14:49.446021+0000","last_undegraded":"2024-04-22T07:14:49.446021+0000","last_fullsized":"2024-04-22T07:14:49.446021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454363+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:14:49.454363+0000","last_peered":"2024-04-22T07:14:49.454363+0000","last_clean":"2024-04-22T07:14:49.454363+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:14:49.454363+0000","last_undegraded":"2024-04-22T07:14:49.454363+0000","last_fullsized":"2024-04-22T07:14:49.454363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451662+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:14:49.451662+0000","last_peered":"2024-04-22T07:14:49.451662+0000","last_clean":"2024-04-22T07:14:49.451662+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:14:49.451662+0000","last_undegraded":"2024-04-22T07:14:49.451662+0000","last_fullsized":"2024-04-22T07:14:49.451662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520117+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:14:49.520117+0000","last_peered":"2024-04-22T07:14:49.520117+0000","last_clean":"2024-04-22T07:14:49.520117+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:14:49.520117+0000","last_undegraded":"2024-04-22T07:14:49.520117+0000","last_fullsized":"2024-04-22T07:14:49.520117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446512+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:14:49.446512+0000","last_peered":"2024-04-22T07:14:49.446512+0000","last_clean":"2024-04-22T07:14:49.446512+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:14:49.446512+0000","last_undegraded":"2024-04-22T07:14:49.446512+0000","last_fullsized":"2024-04-22T07:14:49.446512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451463+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:14:49.451463+0000","last_peered":"2024-04-22T07:14:49.451463+0000","last_clean":"2024-04-22T07:14:49.451463+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:14:49.451463+0000","last_undegraded":"2024-04-22T07:14:49.451463+0000","last_fullsized":"2024-04-22T07:14:49.451463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354021+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:14:49.354021+0000","last_peered":"2024-04-22T07:14:49.354021+0000","last_clean":"2024-04-22T07:14:49.354021+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:14:49.354021+0000","last_undegraded":"2024-04-22T07:14:49.354021+0000","last_fullsized":"2024-04-22T07:14:49.354021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454848+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:49.454848+0000","last_peered":"2024-04-22T07:14:49.454848+0000","last_clean":"2024-04-22T07:14:49.454848+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:49.454848+0000","last_undegraded":"2024-04-22T07:14:49.454848+0000","last_fullsized":"2024-04-22T07:14:49.454848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518517+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:14:49.518517+0000","last_peered":"2024-04-22T07:14:49.518517+0000","last_clean":"2024-04-22T07:14:49.518517+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:14:49.518517+0000","last_undegraded":"2024-04-22T07:14:49.518517+0000","last_fullsized":"2024-04-22T07:14:49.518517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353921+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:14:49.353921+0000","last_peered":"2024-04-22T07:14:49.353921+0000","last_clean":"2024-04-22T07:14:49.353921+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:14:49.353921+0000","last_undegraded":"2024-04-22T07:14:49.353921+0000","last_fullsized":"2024-04-22T07:14:49.353921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567045+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:14:49.567045+0000","last_peered":"2024-04-22T07:14:49.567045+0000","last_clean":"2024-04-22T07:14:49.567045+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:14:49.567045+0000","last_undegraded":"2024-04-22T07:14:49.567045+0000","last_fullsized":"2024-04-22T07:14:49.567045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451908+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:14:49.451908+0000","last_peered":"2024-04-22T07:14:49.451908+0000","last_clean":"2024-04-22T07:14:49.451908+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:14:49.451908+0000","last_undegraded":"2024-04-22T07:14:49.451908+0000","last_fullsized":"2024-04-22T07:14:49.451908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518495+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:14:49.518495+0000","last_peered":"2024-04-22T07:14:49.518495+0000","last_clean":"2024-04-22T07:14:49.518495+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:14:49.518495+0000","last_undegraded":"2024-04-22T07:14:49.518495+0000","last_fullsized":"2024-04-22T07:14:49.518495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.596414+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:49.596414+0000","last_peered":"2024-04-22T07:14:49.596414+0000","last_clean":"2024-04-22T07:14:49.596414+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:49.596414+0000","last_undegraded":"2024-04-22T07:14:49.596414+0000","last_fullsized":"2024-04-22T07:14:49.596414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":177,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446273+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:14:49.446273+0000","last_peered":"2024-04-22T07:14:49.446273+0000","last_clean":"2024-04-22T07:14:49.446273+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:14:49.446273+0000","last_undegraded":"2024-04-22T07:14:49.446273+0000","last_fullsized":"2024-04-22T07:14:49.446273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679544+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:14:49.679544+0000","last_peered":"2024-04-22T07:14:49.679544+0000","last_clean":"2024-04-22T07:14:49.679544+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:14:49.679544+0000","last_undegraded":"2024-04-22T07:14:49.679544+0000","last_fullsized":"2024-04-22T07:14:49.679544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520133+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:14:49.520133+0000","last_peered":"2024-04-22T07:14:49.520133+0000","last_clean":"2024-04-22T07:14:49.520133+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:14:49.520133+0000","last_undegraded":"2024-04-22T07:14:49.520133+0000","last_fullsized":"2024-04-22T07:14:49.520133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451635+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:14:49.451635+0000","last_peered":"2024-04-22T07:14:49.451635+0000","last_clean":"2024-04-22T07:14:49.451635+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:14:49.451635+0000","last_undegraded":"2024-04-22T07:14:49.451635+0000","last_fullsized":"2024-04-22T07:14:49.451635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353878+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:14:49.353878+0000","last_peered":"2024-04-22T07:14:49.353878+0000","last_clean":"2024-04-22T07:14:49.353878+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:14:49.353878+0000","last_undegraded":"2024-04-22T07:14:49.353878+0000","last_fullsized":"2024-04-22T07:14:49.353878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522205+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:49.522205+0000","last_peered":"2024-04-22T07:14:49.522205+0000","last_clean":"2024-04-22T07:14:49.522205+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:49.522205+0000","last_undegraded":"2024-04-22T07:14:49.522205+0000","last_fullsized":"2024-04-22T07:14:49.522205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353610+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:14:49.353610+0000","last_peered":"2024-04-22T07:14:49.353610+0000","last_clean":"2024-04-22T07:14:49.353610+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:14:49.353610+0000","last_undegraded":"2024-04-22T07:14:49.353610+0000","last_fullsized":"2024-04-22T07:14:49.353610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446009+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:14:49.446009+0000","last_peered":"2024-04-22T07:14:49.446009+0000","last_clean":"2024-04-22T07:14:49.446009+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:14:49.446009+0000","last_undegraded":"2024-04-22T07:14:49.446009+0000","last_fullsized":"2024-04-22T07:14:49.446009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451558+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:14:49.451558+0000","last_peered":"2024-04-22T07:14:49.451558+0000","last_clean":"2024-04-22T07:14:49.451558+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:14:49.451558+0000","last_undegraded":"2024-04-22T07:14:49.451558+0000","last_fullsized":"2024-04-22T07:14:49.451558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446380+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:14:49.446380+0000","last_peered":"2024-04-22T07:14:49.446380+0000","last_clean":"2024-04-22T07:14:49.446380+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:14:49.446380+0000","last_undegraded":"2024-04-22T07:14:49.446380+0000","last_fullsized":"2024-04-22T07:14:49.446380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679730+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:14:49.679730+0000","last_peered":"2024-04-22T07:14:49.679730+0000","last_clean":"2024-04-22T07:14:49.679730+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:14:49.679730+0000","last_undegraded":"2024-04-22T07:14:49.679730+0000","last_fullsized":"2024-04-22T07:14:49.679730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354097+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:14:49.354097+0000","last_peered":"2024-04-22T07:14:49.354097+0000","last_clean":"2024-04-22T07:14:49.354097+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:14:49.354097+0000","last_undegraded":"2024-04-22T07:14:49.354097+0000","last_fullsized":"2024-04-22T07:14:49.354097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454763+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:14:49.454763+0000","last_peered":"2024-04-22T07:14:49.454763+0000","last_clean":"2024-04-22T07:14:49.454763+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:14:49.454763+0000","last_undegraded":"2024-04-22T07:14:49.454763+0000","last_fullsized":"2024-04-22T07:14:49.454763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353758+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:14:49.353758+0000","last_peered":"2024-04-22T07:14:49.353758+0000","last_clean":"2024-04-22T07:14:49.353758+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:14:49.353758+0000","last_undegraded":"2024-04-22T07:14:49.353758+0000","last_fullsized":"2024-04-22T07:14:49.353758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454649+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:14:49.454649+0000","last_peered":"2024-04-22T07:14:49.454649+0000","last_clean":"2024-04-22T07:14:49.454649+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:14:49.454649+0000","last_undegraded":"2024-04-22T07:14:49.454649+0000","last_fullsized":"2024-04-22T07:14:49.454649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451592+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:14:49.451592+0000","last_peered":"2024-04-22T07:14:49.451592+0000","last_clean":"2024-04-22T07:14:49.451592+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:14:49.451592+0000","last_undegraded":"2024-04-22T07:14:49.451592+0000","last_fullsized":"2024-04-22T07:14:49.451592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":138,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520301+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:49.520301+0000","last_peered":"2024-04-22T07:14:49.520301+0000","last_clean":"2024-04-22T07:14:49.520301+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:49.520301+0000","last_undegraded":"2024-04-22T07:14:49.520301+0000","last_fullsized":"2024-04-22T07:14:49.520301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446669+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:14:49.446669+0000","last_peered":"2024-04-22T07:14:49.446669+0000","last_clean":"2024-04-22T07:14:49.446669+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:14:49.446669+0000","last_undegraded":"2024-04-22T07:14:49.446669+0000","last_fullsized":"2024-04-22T07:14:49.446669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521956+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:14:49.521956+0000","last_peered":"2024-04-22T07:14:49.521956+0000","last_clean":"2024-04-22T07:14:49.521956+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:14:49.521956+0000","last_undegraded":"2024-04-22T07:14:49.521956+0000","last_fullsized":"2024-04-22T07:14:49.521956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353712+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:14:49.353712+0000","last_peered":"2024-04-22T07:14:49.353712+0000","last_clean":"2024-04-22T07:14:49.353712+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:14:49.353712+0000","last_undegraded":"2024-04-22T07:14:49.353712+0000","last_fullsized":"2024-04-22T07:14:49.353712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.519969+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:14:49.519969+0000","last_peered":"2024-04-22T07:14:49.519969+0000","last_clean":"2024-04-22T07:14:49.519969+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:14:49.519969+0000","last_undegraded":"2024-04-22T07:14:49.519969+0000","last_fullsized":"2024-04-22T07:14:49.519969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353812+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:14:49.353812+0000","last_peered":"2024-04-22T07:14:49.353812+0000","last_clean":"2024-04-22T07:14:49.353812+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:14:49.353812+0000","last_undegraded":"2024-04-22T07:14:49.353812+0000","last_fullsized":"2024-04-22T07:14:49.353812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446310+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:14:49.446310+0000","last_peered":"2024-04-22T07:14:49.446310+0000","last_clean":"2024-04-22T07:14:49.446310+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:14:49.446310+0000","last_undegraded":"2024-04-22T07:14:49.446310+0000","last_fullsized":"2024-04-22T07:14:49.446310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520318+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:14:49.520318+0000","last_peered":"2024-04-22T07:14:49.520318+0000","last_clean":"2024-04-22T07:14:49.520318+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:14:49.520318+0000","last_undegraded":"2024-04-22T07:14:49.520318+0000","last_fullsized":"2024-04-22T07:14:49.520318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.559202+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:14:49.559202+0000","last_peered":"2024-04-22T07:14:49.559202+0000","last_clean":"2024-04-22T07:14:49.559202+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:14:49.559202+0000","last_undegraded":"2024-04-22T07:14:49.559202+0000","last_fullsized":"2024-04-22T07:14:49.559202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446988+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:14:49.446988+0000","last_peered":"2024-04-22T07:14:49.446988+0000","last_clean":"2024-04-22T07:14:49.446988+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:14:49.446988+0000","last_undegraded":"2024-04-22T07:14:49.446988+0000","last_fullsized":"2024-04-22T07:14:49.446988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522148+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:14:49.522148+0000","last_peered":"2024-04-22T07:14:49.522148+0000","last_clean":"2024-04-22T07:14:49.522148+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:14:49.522148+0000","last_undegraded":"2024-04-22T07:14:49.522148+0000","last_fullsized":"2024-04-22T07:14:49.522148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566877+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:14:49.566877+0000","last_peered":"2024-04-22T07:14:49.566877+0000","last_clean":"2024-04-22T07:14:49.566877+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:14:49.566877+0000","last_undegraded":"2024-04-22T07:14:49.566877+0000","last_fullsized":"2024-04-22T07:14:49.566877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518008+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:14:49.518008+0000","last_peered":"2024-04-22T07:14:49.518008+0000","last_clean":"2024-04-22T07:14:49.518008+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:14:49.518008+0000","last_undegraded":"2024-04-22T07:14:49.518008+0000","last_fullsized":"2024-04-22T07:14:49.518008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446005+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:14:49.446005+0000","last_peered":"2024-04-22T07:14:49.446005+0000","last_clean":"2024-04-22T07:14:49.446005+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:14:49.446005+0000","last_undegraded":"2024-04-22T07:14:49.446005+0000","last_fullsized":"2024-04-22T07:14:49.446005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446529+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:14:49.446529+0000","last_peered":"2024-04-22T07:14:49.446529+0000","last_clean":"2024-04-22T07:14:49.446529+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:14:49.446529+0000","last_undegraded":"2024-04-22T07:14:49.446529+0000","last_fullsized":"2024-04-22T07:14:49.446529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679532+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:14:49.679532+0000","last_peered":"2024-04-22T07:14:49.679532+0000","last_clean":"2024-04-22T07:14:49.679532+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:14:49.679532+0000","last_undegraded":"2024-04-22T07:14:49.679532+0000","last_fullsized":"2024-04-22T07:14:49.679532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355979+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:14:49.355979+0000","last_peered":"2024-04-22T07:14:49.355979+0000","last_clean":"2024-04-22T07:14:49.355979+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:14:49.355979+0000","last_undegraded":"2024-04-22T07:14:49.355979+0000","last_fullsized":"2024-04-22T07:14:49.355979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446724+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:14:49.446724+0000","last_peered":"2024-04-22T07:14:49.446724+0000","last_clean":"2024-04-22T07:14:49.446724+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:14:49.446724+0000","last_undegraded":"2024-04-22T07:14:49.446724+0000","last_fullsized":"2024-04-22T07:14:49.446724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446129+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:14:49.446129+0000","last_peered":"2024-04-22T07:14:49.446129+0000","last_clean":"2024-04-22T07:14:49.446129+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:14:49.446129+0000","last_undegraded":"2024-04-22T07:14:49.446129+0000","last_fullsized":"2024-04-22T07:14:49.446129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353868+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:14:49.353868+0000","last_peered":"2024-04-22T07:14:49.353868+0000","last_clean":"2024-04-22T07:14:49.353868+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:14:49.353868+0000","last_undegraded":"2024-04-22T07:14:49.353868+0000","last_fullsized":"2024-04-22T07:14:49.353868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356164+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:14:49.356164+0000","last_peered":"2024-04-22T07:14:49.356164+0000","last_clean":"2024-04-22T07:14:49.356164+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:14:49.356164+0000","last_undegraded":"2024-04-22T07:14:49.356164+0000","last_fullsized":"2024-04-22T07:14:49.356164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354047+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:14:49.354047+0000","last_peered":"2024-04-22T07:14:49.354047+0000","last_clean":"2024-04-22T07:14:49.354047+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:14:49.354047+0000","last_undegraded":"2024-04-22T07:14:49.354047+0000","last_fullsized":"2024-04-22T07:14:49.354047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.553901+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:14:49.553901+0000","last_peered":"2024-04-22T07:14:49.553901+0000","last_clean":"2024-04-22T07:14:49.553901+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:14:49.553901+0000","last_undegraded":"2024-04-22T07:14:49.553901+0000","last_fullsized":"2024-04-22T07:14:49.553901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353880+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:14:49.353880+0000","last_peered":"2024-04-22T07:14:49.353880+0000","last_clean":"2024-04-22T07:14:49.353880+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:14:49.353880+0000","last_undegraded":"2024-04-22T07:14:49.353880+0000","last_fullsized":"2024-04-22T07:14:49.353880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520280+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:14:49.520280+0000","last_peered":"2024-04-22T07:14:49.520280+0000","last_clean":"2024-04-22T07:14:49.520280+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:14:49.520280+0000","last_undegraded":"2024-04-22T07:14:49.520280+0000","last_fullsized":"2024-04-22T07:14:49.520280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354174+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:14:49.354174+0000","last_peered":"2024-04-22T07:14:49.354174+0000","last_clean":"2024-04-22T07:14:49.354174+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:14:49.354174+0000","last_undegraded":"2024-04-22T07:14:49.354174+0000","last_fullsized":"2024-04-22T07:14:49.354174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679633+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:14:49.679633+0000","last_peered":"2024-04-22T07:14:49.679633+0000","last_clean":"2024-04-22T07:14:49.679633+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:14:49.679633+0000","last_undegraded":"2024-04-22T07:14:49.679633+0000","last_fullsized":"2024-04-22T07:14:49.679633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355972+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:14:49.355972+0000","last_peered":"2024-04-22T07:14:49.355972+0000","last_clean":"2024-04-22T07:14:49.355972+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:14:49.355972+0000","last_undegraded":"2024-04-22T07:14:49.355972+0000","last_fullsized":"2024-04-22T07:14:49.355972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355819+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:14:49.355819+0000","last_peered":"2024-04-22T07:14:49.355819+0000","last_clean":"2024-04-22T07:14:49.355819+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:14:49.355819+0000","last_undegraded":"2024-04-22T07:14:49.355819+0000","last_fullsized":"2024-04-22T07:14:49.355819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566665+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:14:49.566665+0000","last_peered":"2024-04-22T07:14:49.566665+0000","last_clean":"2024-04-22T07:14:49.566665+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:14:49.566665+0000","last_undegraded":"2024-04-22T07:14:49.566665+0000","last_fullsized":"2024-04-22T07:14:49.566665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520295+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:14:49.520295+0000","last_peered":"2024-04-22T07:14:49.520295+0000","last_clean":"2024-04-22T07:14:49.520295+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:14:49.520295+0000","last_undegraded":"2024-04-22T07:14:49.520295+0000","last_fullsized":"2024-04-22T07:14:49.520295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679687+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:14:49.679687+0000","last_peered":"2024-04-22T07:14:49.679687+0000","last_clean":"2024-04-22T07:14:49.679687+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:14:49.679687+0000","last_undegraded":"2024-04-22T07:14:49.679687+0000","last_fullsized":"2024-04-22T07:14:49.679687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356248+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:14:49.356248+0000","last_peered":"2024-04-22T07:14:49.356248+0000","last_clean":"2024-04-22T07:14:49.356248+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:14:49.356248+0000","last_undegraded":"2024-04-22T07:14:49.356248+0000","last_fullsized":"2024-04-22T07:14:49.356248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354138+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:14:49.354138+0000","last_peered":"2024-04-22T07:14:49.354138+0000","last_clean":"2024-04-22T07:14:49.354138+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:14:49.354138+0000","last_undegraded":"2024-04-22T07:14:49.354138+0000","last_fullsized":"2024-04-22T07:14:49.354138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451527+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:14:49.451527+0000","last_peered":"2024-04-22T07:14:49.451527+0000","last_clean":"2024-04-22T07:14:49.451527+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:14:49.451527+0000","last_undegraded":"2024-04-22T07:14:49.451527+0000","last_fullsized":"2024-04-22T07:14:49.451527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446375+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:14:49.446375+0000","last_peered":"2024-04-22T07:14:49.446375+0000","last_clean":"2024-04-22T07:14:49.446375+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:14:49.446375+0000","last_undegraded":"2024-04-22T07:14:49.446375+0000","last_fullsized":"2024-04-22T07:14:49.446375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518310+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:14:49.518310+0000","last_peered":"2024-04-22T07:14:49.518310+0000","last_clean":"2024-04-22T07:14:49.518310+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:14:49.518310+0000","last_undegraded":"2024-04-22T07:14:49.518310+0000","last_fullsized":"2024-04-22T07:14:49.518310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520315+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:14:49.520315+0000","last_peered":"2024-04-22T07:14:49.520315+0000","last_clean":"2024-04-22T07:14:49.520315+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:14:49.520315+0000","last_undegraded":"2024-04-22T07:14:49.520315+0000","last_fullsized":"2024-04-22T07:14:49.520315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679771+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:14:49.679771+0000","last_peered":"2024-04-22T07:14:49.679771+0000","last_clean":"2024-04-22T07:14:49.679771+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:14:49.679771+0000","last_undegraded":"2024-04-22T07:14:49.679771+0000","last_fullsized":"2024-04-22T07:14:49.679771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597274,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":793,"num_read_kb":499,"num_write":284,"num_write_kb":1434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1261720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36688,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708464,"statfs":{"total":95995035648,"available":95957467136,"internally_reserved":0,"allocated":1380352,"data_stored":863942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1376256,"data_stored":863280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1581056,"data_stored":1063275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":538248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":471386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":539681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":995242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":470943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1728512,"data_stored":1194168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:14:53.755 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 -- 172.21.15.113:0/3326167090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc06a340 msgr2=0x7f9adc06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3326167090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc06a340 0x7f9adc06a720 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009900 tx=0x7f9ad802bcd0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 -- 172.21.15.113:0/3326167090 shutdown_connections 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3326167090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc10e960 0x7f9adc110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3326167090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06ac60 0x7f9adc10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3326167090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc06a340 0x7f9adc06a720 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 -- 172.21.15.113:0/3326167090 >> 172.21.15.113:0/3326167090 conn(0x7f9adc077f80 msgr2=0x7f9adc078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 -- 172.21.15.113:0/3326167090 shutdown_connections 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.893+0000 7f9ae3975640 1 -- 172.21.15.113:0/3326167090 wait complete. 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 Processor -- start 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 -- start start 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc06a340 0x7f9adc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06ac60 0x7f9adc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 0x7f9adc19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc114570 con 0x7f9adc10e960 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1143f0 con 0x7f9adc06ac60 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae3975640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1146f0 con 0x7f9adc06a340 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae1f12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 0x7f9adc19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae1f12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 0x7f9adc19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53012/0 (socket says 172.21.15.113:53012) 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae1f12640 1 -- 172.21.15.113:0/3011432511 learned_addr learned my addr 172.21.15.113:0/3011432511 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:53.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae0f10640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06ac60 0x7f9adc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae1711640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc06a340 0x7f9adc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.894+0000 7f9ae1f12640 1 -- 172.21.15.113:0/3011432511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc06a340 msgr2=0x7f9adc1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae1f12640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc06a340 0x7f9adc1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae1f12640 1 -- 172.21.15.113:0/3011432511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06ac60 msgr2=0x7f9adc1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae1f12640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06ac60 0x7f9adc1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae1f12640 1 -- 172.21.15.113:0/3011432511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc19aff0 con 0x7f9adc10e960 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae1711640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc06a340 0x7f9adc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae1f12640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 0x7f9adc19a870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400b9e0 tx=0x7f9ad400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4019070 con 0x7f9adc10e960 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ad4014440 con 0x7f9adc10e960 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9adc19b2e0 con 0x7f9adc10e960 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40125a0 con 0x7f9adc10e960 2024-04-22T07:14:53.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.895+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9adc1a7580 con 0x7f9adc10e960 2024-04-22T07:14:53.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.896+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9adc068550 con 0x7f9adc10e960 2024-04-22T07:14:53.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.897+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 23) ==== 138818+0+0 (secure 0 0 0) 0x7f9ad401c070 con 0x7f9adc10e960 2024-04-22T07:14:53.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.897+0000 7f9ad27fc640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ac80a3db0 0x7f9ac80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:53.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.898+0000 7f9ae1711640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ac80a3db0 0x7f9ac80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:53.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.898+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9ad40cfd90 con 0x7f9adc10e960 2024-04-22T07:14:53.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.898+0000 7f9ae1711640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ac80a3db0 0x7f9ac80a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80096c0 tx=0x7f9ad8000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:53.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:53.903+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ad409c400 con 0x7f9adc10e960 2024-04-22T07:14:54.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.074+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f9ad4098370 con 0x7f9adc10e960 2024-04-22T07:14:54.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.100+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9adc062390 con 0x7f9ac80a3db0 2024-04-22T07:14:54.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.107+0000 7f9ad27fc640 1 -- 172.21.15.113:0/3011432511 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257738 (secure 0 0 0) 0x7f9adc062390 con 0x7f9ac80a3db0 2024-04-22T07:14:54.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:54.110 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:14:54.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ac80a3db0 msgr2=0x7f9ac80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ac80a3db0 0x7f9ac80a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80096c0 tx=0x7f9ad8000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 msgr2=0x7f9adc19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 0x7f9adc19a870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400b9e0 tx=0x7f9ad400bea0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 shutdown_connections 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ac80a3db0 0x7f9ac80a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9adc10e960 0x7f9adc19a870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9adc06ac60 0x7f9adc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 --2- 172.21.15.113:0/3011432511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9adc06a340 0x7f9adc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 >> 172.21.15.113:0/3011432511 conn(0x7f9adc077f80 msgr2=0x7f9adc10fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 shutdown_connections 2024-04-22T07:14:54.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.110+0000 7f9ae3975640 1 -- 172.21.15.113:0/3011432511 wait complete. 2024-04-22T07:14:54.120 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T07:14:53.366437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187876,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":929,"num_read_kb":618,"num_write":467,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368420,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573404,"statfs":{"total":1151940427776,"available":1151563165696,"internally_reserved":0,"allocated":15433728,"data_stored":9202524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990817"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679807+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:14:49.679807+0000","last_peered":"2024-04-22T07:14:49.679807+0000","last_clean":"2024-04-22T07:14:49.679807+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:14:49.679807+0000","last_undegraded":"2024-04-22T07:14:49.679807+0000","last_fullsized":"2024-04-22T07:14:49.679807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566911+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:14:49.566911+0000","last_peered":"2024-04-22T07:14:49.566911+0000","last_clean":"2024-04-22T07:14:49.566911+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:14:49.566911+0000","last_undegraded":"2024-04-22T07:14:49.566911+0000","last_fullsized":"2024-04-22T07:14:49.566911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566794+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:14:49.566794+0000","last_peered":"2024-04-22T07:14:49.566794+0000","last_clean":"2024-04-22T07:14:49.566794+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:14:49.566794+0000","last_undegraded":"2024-04-22T07:14:49.566794+0000","last_fullsized":"2024-04-22T07:14:49.566794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522113+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:14:49.522113+0000","last_peered":"2024-04-22T07:14:49.522113+0000","last_clean":"2024-04-22T07:14:49.522113+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:14:49.522113+0000","last_undegraded":"2024-04-22T07:14:49.522113+0000","last_fullsized":"2024-04-22T07:14:49.522113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446077+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:14:49.446077+0000","last_peered":"2024-04-22T07:14:49.446077+0000","last_clean":"2024-04-22T07:14:49.446077+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:14:49.446077+0000","last_undegraded":"2024-04-22T07:14:49.446077+0000","last_fullsized":"2024-04-22T07:14:49.446077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454433+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:14:49.454433+0000","last_peered":"2024-04-22T07:14:49.454433+0000","last_clean":"2024-04-22T07:14:49.454433+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:14:49.454433+0000","last_undegraded":"2024-04-22T07:14:49.454433+0000","last_fullsized":"2024-04-22T07:14:49.454433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353645+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:14:49.353645+0000","last_peered":"2024-04-22T07:14:49.353645+0000","last_clean":"2024-04-22T07:14:49.353645+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:14:49.353645+0000","last_undegraded":"2024-04-22T07:14:49.353645+0000","last_fullsized":"2024-04-22T07:14:49.353645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446353+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:14:49.446353+0000","last_peered":"2024-04-22T07:14:49.446353+0000","last_clean":"2024-04-22T07:14:49.446353+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:14:49.446353+0000","last_undegraded":"2024-04-22T07:14:49.446353+0000","last_fullsized":"2024-04-22T07:14:49.446353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518246+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:14:49.518246+0000","last_peered":"2024-04-22T07:14:49.518246+0000","last_clean":"2024-04-22T07:14:49.518246+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:14:49.518246+0000","last_undegraded":"2024-04-22T07:14:49.518246+0000","last_fullsized":"2024-04-22T07:14:49.518246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454790+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:14:49.454790+0000","last_peered":"2024-04-22T07:14:49.454790+0000","last_clean":"2024-04-22T07:14:49.454790+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:14:49.454790+0000","last_undegraded":"2024-04-22T07:14:49.454790+0000","last_fullsized":"2024-04-22T07:14:49.454790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353879+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:14:49.353879+0000","last_peered":"2024-04-22T07:14:49.353879+0000","last_clean":"2024-04-22T07:14:49.353879+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:14:49.353879+0000","last_undegraded":"2024-04-22T07:14:49.353879+0000","last_fullsized":"2024-04-22T07:14:49.353879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521981+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:14:49.521981+0000","last_peered":"2024-04-22T07:14:49.521981+0000","last_clean":"2024-04-22T07:14:49.521981+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:14:49.521981+0000","last_undegraded":"2024-04-22T07:14:49.521981+0000","last_fullsized":"2024-04-22T07:14:49.521981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520446+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:14:49.520446+0000","last_peered":"2024-04-22T07:14:49.520446+0000","last_clean":"2024-04-22T07:14:49.520446+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:14:49.520446+0000","last_undegraded":"2024-04-22T07:14:49.520446+0000","last_fullsized":"2024-04-22T07:14:49.520446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567028+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:14:49.567028+0000","last_peered":"2024-04-22T07:14:49.567028+0000","last_clean":"2024-04-22T07:14:49.567028+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:14:49.567028+0000","last_undegraded":"2024-04-22T07:14:49.567028+0000","last_fullsized":"2024-04-22T07:14:49.567028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566423+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:14:49.566423+0000","last_peered":"2024-04-22T07:14:49.566423+0000","last_clean":"2024-04-22T07:14:49.566423+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:14:49.566423+0000","last_undegraded":"2024-04-22T07:14:49.566423+0000","last_fullsized":"2024-04-22T07:14:49.566423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.517925+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:14:49.517925+0000","last_peered":"2024-04-22T07:14:49.517925+0000","last_clean":"2024-04-22T07:14:49.517925+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:14:49.517925+0000","last_undegraded":"2024-04-22T07:14:49.517925+0000","last_fullsized":"2024-04-22T07:14:49.517925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679946+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:14:49.679946+0000","last_peered":"2024-04-22T07:14:49.679946+0000","last_clean":"2024-04-22T07:14:49.679946+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:14:49.679946+0000","last_undegraded":"2024-04-22T07:14:49.679946+0000","last_fullsized":"2024-04-22T07:14:49.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354137+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:14:49.354137+0000","last_peered":"2024-04-22T07:14:49.354137+0000","last_clean":"2024-04-22T07:14:49.354137+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:14:49.354137+0000","last_undegraded":"2024-04-22T07:14:49.354137+0000","last_fullsized":"2024-04-22T07:14:49.354137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356040+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:14:49.356040+0000","last_peered":"2024-04-22T07:14:49.356040+0000","last_clean":"2024-04-22T07:14:49.356040+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:14:49.356040+0000","last_undegraded":"2024-04-22T07:14:49.356040+0000","last_fullsized":"2024-04-22T07:14:49.356040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354290+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:14:49.354290+0000","last_peered":"2024-04-22T07:14:49.354290+0000","last_clean":"2024-04-22T07:14:49.354290+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:14:49.354290+0000","last_undegraded":"2024-04-22T07:14:49.354290+0000","last_fullsized":"2024-04-22T07:14:49.354290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447115+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:14:49.447115+0000","last_peered":"2024-04-22T07:14:49.447115+0000","last_clean":"2024-04-22T07:14:49.447115+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:14:49.447115+0000","last_undegraded":"2024-04-22T07:14:49.447115+0000","last_fullsized":"2024-04-22T07:14:49.447115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447288+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:14:49.447288+0000","last_peered":"2024-04-22T07:14:49.447288+0000","last_clean":"2024-04-22T07:14:49.447288+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:14:49.447288+0000","last_undegraded":"2024-04-22T07:14:49.447288+0000","last_fullsized":"2024-04-22T07:14:49.447288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567072+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:14:49.567072+0000","last_peered":"2024-04-22T07:14:49.567072+0000","last_clean":"2024-04-22T07:14:49.567072+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:14:49.567072+0000","last_undegraded":"2024-04-22T07:14:49.567072+0000","last_fullsized":"2024-04-22T07:14:49.567072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446441+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:14:49.446441+0000","last_peered":"2024-04-22T07:14:49.446441+0000","last_clean":"2024-04-22T07:14:49.446441+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:14:49.446441+0000","last_undegraded":"2024-04-22T07:14:49.446441+0000","last_fullsized":"2024-04-22T07:14:49.446441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518065+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:14:49.518065+0000","last_peered":"2024-04-22T07:14:49.518065+0000","last_clean":"2024-04-22T07:14:49.518065+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:14:49.518065+0000","last_undegraded":"2024-04-22T07:14:49.518065+0000","last_fullsized":"2024-04-22T07:14:49.518065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679550+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:14:49.679550+0000","last_peered":"2024-04-22T07:14:49.679550+0000","last_clean":"2024-04-22T07:14:49.679550+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:14:49.679550+0000","last_undegraded":"2024-04-22T07:14:49.679550+0000","last_fullsized":"2024-04-22T07:14:49.679550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451783+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:14:49.451783+0000","last_peered":"2024-04-22T07:14:49.451783+0000","last_clean":"2024-04-22T07:14:49.451783+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:14:49.451783+0000","last_undegraded":"2024-04-22T07:14:49.451783+0000","last_fullsized":"2024-04-22T07:14:49.451783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.500064+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:49.500064+0000","last_peered":"2024-04-22T07:14:49.500064+0000","last_clean":"2024-04-22T07:14:49.500064+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:49.500064+0000","last_undegraded":"2024-04-22T07:14:49.500064+0000","last_fullsized":"2024-04-22T07:14:49.500064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446021+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:14:49.446021+0000","last_peered":"2024-04-22T07:14:49.446021+0000","last_clean":"2024-04-22T07:14:49.446021+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:14:49.446021+0000","last_undegraded":"2024-04-22T07:14:49.446021+0000","last_fullsized":"2024-04-22T07:14:49.446021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454363+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:14:49.454363+0000","last_peered":"2024-04-22T07:14:49.454363+0000","last_clean":"2024-04-22T07:14:49.454363+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:14:49.454363+0000","last_undegraded":"2024-04-22T07:14:49.454363+0000","last_fullsized":"2024-04-22T07:14:49.454363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451662+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:14:49.451662+0000","last_peered":"2024-04-22T07:14:49.451662+0000","last_clean":"2024-04-22T07:14:49.451662+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:14:49.451662+0000","last_undegraded":"2024-04-22T07:14:49.451662+0000","last_fullsized":"2024-04-22T07:14:49.451662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520117+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:14:49.520117+0000","last_peered":"2024-04-22T07:14:49.520117+0000","last_clean":"2024-04-22T07:14:49.520117+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:14:49.520117+0000","last_undegraded":"2024-04-22T07:14:49.520117+0000","last_fullsized":"2024-04-22T07:14:49.520117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446512+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:14:49.446512+0000","last_peered":"2024-04-22T07:14:49.446512+0000","last_clean":"2024-04-22T07:14:49.446512+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:14:49.446512+0000","last_undegraded":"2024-04-22T07:14:49.446512+0000","last_fullsized":"2024-04-22T07:14:49.446512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451463+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:14:49.451463+0000","last_peered":"2024-04-22T07:14:49.451463+0000","last_clean":"2024-04-22T07:14:49.451463+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:14:49.451463+0000","last_undegraded":"2024-04-22T07:14:49.451463+0000","last_fullsized":"2024-04-22T07:14:49.451463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354021+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:14:49.354021+0000","last_peered":"2024-04-22T07:14:49.354021+0000","last_clean":"2024-04-22T07:14:49.354021+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:14:49.354021+0000","last_undegraded":"2024-04-22T07:14:49.354021+0000","last_fullsized":"2024-04-22T07:14:49.354021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454848+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:49.454848+0000","last_peered":"2024-04-22T07:14:49.454848+0000","last_clean":"2024-04-22T07:14:49.454848+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:49.454848+0000","last_undegraded":"2024-04-22T07:14:49.454848+0000","last_fullsized":"2024-04-22T07:14:49.454848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518517+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:14:49.518517+0000","last_peered":"2024-04-22T07:14:49.518517+0000","last_clean":"2024-04-22T07:14:49.518517+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:14:49.518517+0000","last_undegraded":"2024-04-22T07:14:49.518517+0000","last_fullsized":"2024-04-22T07:14:49.518517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353921+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:14:49.353921+0000","last_peered":"2024-04-22T07:14:49.353921+0000","last_clean":"2024-04-22T07:14:49.353921+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:14:49.353921+0000","last_undegraded":"2024-04-22T07:14:49.353921+0000","last_fullsized":"2024-04-22T07:14:49.353921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567045+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:14:49.567045+0000","last_peered":"2024-04-22T07:14:49.567045+0000","last_clean":"2024-04-22T07:14:49.567045+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:14:49.567045+0000","last_undegraded":"2024-04-22T07:14:49.567045+0000","last_fullsized":"2024-04-22T07:14:49.567045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451908+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:14:49.451908+0000","last_peered":"2024-04-22T07:14:49.451908+0000","last_clean":"2024-04-22T07:14:49.451908+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:14:49.451908+0000","last_undegraded":"2024-04-22T07:14:49.451908+0000","last_fullsized":"2024-04-22T07:14:49.451908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518495+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:14:49.518495+0000","last_peered":"2024-04-22T07:14:49.518495+0000","last_clean":"2024-04-22T07:14:49.518495+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:14:49.518495+0000","last_undegraded":"2024-04-22T07:14:49.518495+0000","last_fullsized":"2024-04-22T07:14:49.518495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.596414+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:49.596414+0000","last_peered":"2024-04-22T07:14:49.596414+0000","last_clean":"2024-04-22T07:14:49.596414+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:49.596414+0000","last_undegraded":"2024-04-22T07:14:49.596414+0000","last_fullsized":"2024-04-22T07:14:49.596414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":177,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446273+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:14:49.446273+0000","last_peered":"2024-04-22T07:14:49.446273+0000","last_clean":"2024-04-22T07:14:49.446273+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:14:49.446273+0000","last_undegraded":"2024-04-22T07:14:49.446273+0000","last_fullsized":"2024-04-22T07:14:49.446273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679544+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:14:49.679544+0000","last_peered":"2024-04-22T07:14:49.679544+0000","last_clean":"2024-04-22T07:14:49.679544+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:14:49.679544+0000","last_undegraded":"2024-04-22T07:14:49.679544+0000","last_fullsized":"2024-04-22T07:14:49.679544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520133+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:14:49.520133+0000","last_peered":"2024-04-22T07:14:49.520133+0000","last_clean":"2024-04-22T07:14:49.520133+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:14:49.520133+0000","last_undegraded":"2024-04-22T07:14:49.520133+0000","last_fullsized":"2024-04-22T07:14:49.520133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451635+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:14:49.451635+0000","last_peered":"2024-04-22T07:14:49.451635+0000","last_clean":"2024-04-22T07:14:49.451635+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:14:49.451635+0000","last_undegraded":"2024-04-22T07:14:49.451635+0000","last_fullsized":"2024-04-22T07:14:49.451635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353878+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:14:49.353878+0000","last_peered":"2024-04-22T07:14:49.353878+0000","last_clean":"2024-04-22T07:14:49.353878+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:14:49.353878+0000","last_undegraded":"2024-04-22T07:14:49.353878+0000","last_fullsized":"2024-04-22T07:14:49.353878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522205+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:49.522205+0000","last_peered":"2024-04-22T07:14:49.522205+0000","last_clean":"2024-04-22T07:14:49.522205+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:49.522205+0000","last_undegraded":"2024-04-22T07:14:49.522205+0000","last_fullsized":"2024-04-22T07:14:49.522205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353610+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:14:49.353610+0000","last_peered":"2024-04-22T07:14:49.353610+0000","last_clean":"2024-04-22T07:14:49.353610+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:14:49.353610+0000","last_undegraded":"2024-04-22T07:14:49.353610+0000","last_fullsized":"2024-04-22T07:14:49.353610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446009+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:14:49.446009+0000","last_peered":"2024-04-22T07:14:49.446009+0000","last_clean":"2024-04-22T07:14:49.446009+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:14:49.446009+0000","last_undegraded":"2024-04-22T07:14:49.446009+0000","last_fullsized":"2024-04-22T07:14:49.446009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451558+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:14:49.451558+0000","last_peered":"2024-04-22T07:14:49.451558+0000","last_clean":"2024-04-22T07:14:49.451558+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:14:49.451558+0000","last_undegraded":"2024-04-22T07:14:49.451558+0000","last_fullsized":"2024-04-22T07:14:49.451558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446380+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:14:49.446380+0000","last_peered":"2024-04-22T07:14:49.446380+0000","last_clean":"2024-04-22T07:14:49.446380+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:14:49.446380+0000","last_undegraded":"2024-04-22T07:14:49.446380+0000","last_fullsized":"2024-04-22T07:14:49.446380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679730+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:14:49.679730+0000","last_peered":"2024-04-22T07:14:49.679730+0000","last_clean":"2024-04-22T07:14:49.679730+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:14:49.679730+0000","last_undegraded":"2024-04-22T07:14:49.679730+0000","last_fullsized":"2024-04-22T07:14:49.679730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354097+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:14:49.354097+0000","last_peered":"2024-04-22T07:14:49.354097+0000","last_clean":"2024-04-22T07:14:49.354097+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:14:49.354097+0000","last_undegraded":"2024-04-22T07:14:49.354097+0000","last_fullsized":"2024-04-22T07:14:49.354097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454763+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:14:49.454763+0000","last_peered":"2024-04-22T07:14:49.454763+0000","last_clean":"2024-04-22T07:14:49.454763+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:14:49.454763+0000","last_undegraded":"2024-04-22T07:14:49.454763+0000","last_fullsized":"2024-04-22T07:14:49.454763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353758+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:14:49.353758+0000","last_peered":"2024-04-22T07:14:49.353758+0000","last_clean":"2024-04-22T07:14:49.353758+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:14:49.353758+0000","last_undegraded":"2024-04-22T07:14:49.353758+0000","last_fullsized":"2024-04-22T07:14:49.353758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454649+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:14:49.454649+0000","last_peered":"2024-04-22T07:14:49.454649+0000","last_clean":"2024-04-22T07:14:49.454649+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:14:49.454649+0000","last_undegraded":"2024-04-22T07:14:49.454649+0000","last_fullsized":"2024-04-22T07:14:49.454649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451592+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:14:49.451592+0000","last_peered":"2024-04-22T07:14:49.451592+0000","last_clean":"2024-04-22T07:14:49.451592+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:14:49.451592+0000","last_undegraded":"2024-04-22T07:14:49.451592+0000","last_fullsized":"2024-04-22T07:14:49.451592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":138,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520301+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:49.520301+0000","last_peered":"2024-04-22T07:14:49.520301+0000","last_clean":"2024-04-22T07:14:49.520301+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:49.520301+0000","last_undegraded":"2024-04-22T07:14:49.520301+0000","last_fullsized":"2024-04-22T07:14:49.520301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446669+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:14:49.446669+0000","last_peered":"2024-04-22T07:14:49.446669+0000","last_clean":"2024-04-22T07:14:49.446669+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:14:49.446669+0000","last_undegraded":"2024-04-22T07:14:49.446669+0000","last_fullsized":"2024-04-22T07:14:49.446669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521956+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:14:49.521956+0000","last_peered":"2024-04-22T07:14:49.521956+0000","last_clean":"2024-04-22T07:14:49.521956+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:14:49.521956+0000","last_undegraded":"2024-04-22T07:14:49.521956+0000","last_fullsized":"2024-04-22T07:14:49.521956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353712+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:14:49.353712+0000","last_peered":"2024-04-22T07:14:49.353712+0000","last_clean":"2024-04-22T07:14:49.353712+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:14:49.353712+0000","last_undegraded":"2024-04-22T07:14:49.353712+0000","last_fullsized":"2024-04-22T07:14:49.353712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.519969+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:14:49.519969+0000","last_peered":"2024-04-22T07:14:49.519969+0000","last_clean":"2024-04-22T07:14:49.519969+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:14:49.519969+0000","last_undegraded":"2024-04-22T07:14:49.519969+0000","last_fullsized":"2024-04-22T07:14:49.519969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353812+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:14:49.353812+0000","last_peered":"2024-04-22T07:14:49.353812+0000","last_clean":"2024-04-22T07:14:49.353812+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:14:49.353812+0000","last_undegraded":"2024-04-22T07:14:49.353812+0000","last_fullsized":"2024-04-22T07:14:49.353812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446310+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:14:49.446310+0000","last_peered":"2024-04-22T07:14:49.446310+0000","last_clean":"2024-04-22T07:14:49.446310+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:14:49.446310+0000","last_undegraded":"2024-04-22T07:14:49.446310+0000","last_fullsized":"2024-04-22T07:14:49.446310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520318+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:14:49.520318+0000","last_peered":"2024-04-22T07:14:49.520318+0000","last_clean":"2024-04-22T07:14:49.520318+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:14:49.520318+0000","last_undegraded":"2024-04-22T07:14:49.520318+0000","last_fullsized":"2024-04-22T07:14:49.520318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.559202+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:14:49.559202+0000","last_peered":"2024-04-22T07:14:49.559202+0000","last_clean":"2024-04-22T07:14:49.559202+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:14:49.559202+0000","last_undegraded":"2024-04-22T07:14:49.559202+0000","last_fullsized":"2024-04-22T07:14:49.559202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446988+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:14:49.446988+0000","last_peered":"2024-04-22T07:14:49.446988+0000","last_clean":"2024-04-22T07:14:49.446988+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:14:49.446988+0000","last_undegraded":"2024-04-22T07:14:49.446988+0000","last_fullsized":"2024-04-22T07:14:49.446988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522148+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:14:49.522148+0000","last_peered":"2024-04-22T07:14:49.522148+0000","last_clean":"2024-04-22T07:14:49.522148+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:14:49.522148+0000","last_undegraded":"2024-04-22T07:14:49.522148+0000","last_fullsized":"2024-04-22T07:14:49.522148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566877+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:14:49.566877+0000","last_peered":"2024-04-22T07:14:49.566877+0000","last_clean":"2024-04-22T07:14:49.566877+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:14:49.566877+0000","last_undegraded":"2024-04-22T07:14:49.566877+0000","last_fullsized":"2024-04-22T07:14:49.566877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518008+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:14:49.518008+0000","last_peered":"2024-04-22T07:14:49.518008+0000","last_clean":"2024-04-22T07:14:49.518008+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:14:49.518008+0000","last_undegraded":"2024-04-22T07:14:49.518008+0000","last_fullsized":"2024-04-22T07:14:49.518008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446005+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:14:49.446005+0000","last_peered":"2024-04-22T07:14:49.446005+0000","last_clean":"2024-04-22T07:14:49.446005+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:14:49.446005+0000","last_undegraded":"2024-04-22T07:14:49.446005+0000","last_fullsized":"2024-04-22T07:14:49.446005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446529+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:14:49.446529+0000","last_peered":"2024-04-22T07:14:49.446529+0000","last_clean":"2024-04-22T07:14:49.446529+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:14:49.446529+0000","last_undegraded":"2024-04-22T07:14:49.446529+0000","last_fullsized":"2024-04-22T07:14:49.446529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679532+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:14:49.679532+0000","last_peered":"2024-04-22T07:14:49.679532+0000","last_clean":"2024-04-22T07:14:49.679532+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:14:49.679532+0000","last_undegraded":"2024-04-22T07:14:49.679532+0000","last_fullsized":"2024-04-22T07:14:49.679532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355979+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:14:49.355979+0000","last_peered":"2024-04-22T07:14:49.355979+0000","last_clean":"2024-04-22T07:14:49.355979+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:14:49.355979+0000","last_undegraded":"2024-04-22T07:14:49.355979+0000","last_fullsized":"2024-04-22T07:14:49.355979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446724+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:14:49.446724+0000","last_peered":"2024-04-22T07:14:49.446724+0000","last_clean":"2024-04-22T07:14:49.446724+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:14:49.446724+0000","last_undegraded":"2024-04-22T07:14:49.446724+0000","last_fullsized":"2024-04-22T07:14:49.446724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446129+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:14:49.446129+0000","last_peered":"2024-04-22T07:14:49.446129+0000","last_clean":"2024-04-22T07:14:49.446129+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:14:49.446129+0000","last_undegraded":"2024-04-22T07:14:49.446129+0000","last_fullsized":"2024-04-22T07:14:49.446129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353868+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:14:49.353868+0000","last_peered":"2024-04-22T07:14:49.353868+0000","last_clean":"2024-04-22T07:14:49.353868+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:14:49.353868+0000","last_undegraded":"2024-04-22T07:14:49.353868+0000","last_fullsized":"2024-04-22T07:14:49.353868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356164+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:14:49.356164+0000","last_peered":"2024-04-22T07:14:49.356164+0000","last_clean":"2024-04-22T07:14:49.356164+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:14:49.356164+0000","last_undegraded":"2024-04-22T07:14:49.356164+0000","last_fullsized":"2024-04-22T07:14:49.356164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354047+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:14:49.354047+0000","last_peered":"2024-04-22T07:14:49.354047+0000","last_clean":"2024-04-22T07:14:49.354047+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:14:49.354047+0000","last_undegraded":"2024-04-22T07:14:49.354047+0000","last_fullsized":"2024-04-22T07:14:49.354047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.553901+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:14:49.553901+0000","last_peered":"2024-04-22T07:14:49.553901+0000","last_clean":"2024-04-22T07:14:49.553901+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:14:49.553901+0000","last_undegraded":"2024-04-22T07:14:49.553901+0000","last_fullsized":"2024-04-22T07:14:49.553901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353880+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:14:49.353880+0000","last_peered":"2024-04-22T07:14:49.353880+0000","last_clean":"2024-04-22T07:14:49.353880+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:14:49.353880+0000","last_undegraded":"2024-04-22T07:14:49.353880+0000","last_fullsized":"2024-04-22T07:14:49.353880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520280+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:14:49.520280+0000","last_peered":"2024-04-22T07:14:49.520280+0000","last_clean":"2024-04-22T07:14:49.520280+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:14:49.520280+0000","last_undegraded":"2024-04-22T07:14:49.520280+0000","last_fullsized":"2024-04-22T07:14:49.520280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354174+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:14:49.354174+0000","last_peered":"2024-04-22T07:14:49.354174+0000","last_clean":"2024-04-22T07:14:49.354174+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:14:49.354174+0000","last_undegraded":"2024-04-22T07:14:49.354174+0000","last_fullsized":"2024-04-22T07:14:49.354174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679633+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:14:49.679633+0000","last_peered":"2024-04-22T07:14:49.679633+0000","last_clean":"2024-04-22T07:14:49.679633+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:14:49.679633+0000","last_undegraded":"2024-04-22T07:14:49.679633+0000","last_fullsized":"2024-04-22T07:14:49.679633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355972+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:14:49.355972+0000","last_peered":"2024-04-22T07:14:49.355972+0000","last_clean":"2024-04-22T07:14:49.355972+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:14:49.355972+0000","last_undegraded":"2024-04-22T07:14:49.355972+0000","last_fullsized":"2024-04-22T07:14:49.355972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355819+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:14:49.355819+0000","last_peered":"2024-04-22T07:14:49.355819+0000","last_clean":"2024-04-22T07:14:49.355819+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:14:49.355819+0000","last_undegraded":"2024-04-22T07:14:49.355819+0000","last_fullsized":"2024-04-22T07:14:49.355819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566665+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:14:49.566665+0000","last_peered":"2024-04-22T07:14:49.566665+0000","last_clean":"2024-04-22T07:14:49.566665+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:14:49.566665+0000","last_undegraded":"2024-04-22T07:14:49.566665+0000","last_fullsized":"2024-04-22T07:14:49.566665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520295+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:14:49.520295+0000","last_peered":"2024-04-22T07:14:49.520295+0000","last_clean":"2024-04-22T07:14:49.520295+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:14:49.520295+0000","last_undegraded":"2024-04-22T07:14:49.520295+0000","last_fullsized":"2024-04-22T07:14:49.520295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679687+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:14:49.679687+0000","last_peered":"2024-04-22T07:14:49.679687+0000","last_clean":"2024-04-22T07:14:49.679687+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:14:49.679687+0000","last_undegraded":"2024-04-22T07:14:49.679687+0000","last_fullsized":"2024-04-22T07:14:49.679687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356248+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:14:49.356248+0000","last_peered":"2024-04-22T07:14:49.356248+0000","last_clean":"2024-04-22T07:14:49.356248+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:14:49.356248+0000","last_undegraded":"2024-04-22T07:14:49.356248+0000","last_fullsized":"2024-04-22T07:14:49.356248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354138+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:14:49.354138+0000","last_peered":"2024-04-22T07:14:49.354138+0000","last_clean":"2024-04-22T07:14:49.354138+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:14:49.354138+0000","last_undegraded":"2024-04-22T07:14:49.354138+0000","last_fullsized":"2024-04-22T07:14:49.354138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451527+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:14:49.451527+0000","last_peered":"2024-04-22T07:14:49.451527+0000","last_clean":"2024-04-22T07:14:49.451527+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:14:49.451527+0000","last_undegraded":"2024-04-22T07:14:49.451527+0000","last_fullsized":"2024-04-22T07:14:49.451527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446375+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:14:49.446375+0000","last_peered":"2024-04-22T07:14:49.446375+0000","last_clean":"2024-04-22T07:14:49.446375+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:14:49.446375+0000","last_undegraded":"2024-04-22T07:14:49.446375+0000","last_fullsized":"2024-04-22T07:14:49.446375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518310+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:14:49.518310+0000","last_peered":"2024-04-22T07:14:49.518310+0000","last_clean":"2024-04-22T07:14:49.518310+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:14:49.518310+0000","last_undegraded":"2024-04-22T07:14:49.518310+0000","last_fullsized":"2024-04-22T07:14:49.518310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520315+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:14:49.520315+0000","last_peered":"2024-04-22T07:14:49.520315+0000","last_clean":"2024-04-22T07:14:49.520315+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:14:49.520315+0000","last_undegraded":"2024-04-22T07:14:49.520315+0000","last_fullsized":"2024-04-22T07:14:49.520315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679771+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:14:49.679771+0000","last_peered":"2024-04-22T07:14:49.679771+0000","last_clean":"2024-04-22T07:14:49.679771+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:14:49.679771+0000","last_undegraded":"2024-04-22T07:14:49.679771+0000","last_fullsized":"2024-04-22T07:14:49.679771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597274,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":793,"num_read_kb":499,"num_write":284,"num_write_kb":1434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1261720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36688,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708464,"statfs":{"total":95995035648,"available":95957467136,"internally_reserved":0,"allocated":1380352,"data_stored":863942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1376256,"data_stored":863280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1581056,"data_stored":1063275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":538248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":471386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":539681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":995242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":470943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1728512,"data_stored":1194168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:14:54.124 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-22T07:14:54.124 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3992961634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb81010a0 msgr2=0x7fdbb8101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3992961634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb81010a0 0x7fdbb8101480 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3992961634 shutdown_connections 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3992961634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbb8106170 0x7fdbb810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3992961634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb8101a50 0x7fdbb8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3992961634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb81010a0 0x7fdbb8101480 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3992961634 >> 172.21.15.113:0/3992961634 conn(0x7fdbb80771d0 msgr2=0x7fdbb80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3992961634 shutdown_connections 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.262+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3992961634 wait complete. 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 Processor -- start 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 -- start start 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 0x7fdbb8197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb8106170 0x7fdbb8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbb819bf80 0x7fdbb819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81111f0 con 0x7fdbb8106170 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8111070 con 0x7fdbb81010a0 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbbd3b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8111370 con 0x7fdbb819bf80 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 0x7fdbb8197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb8106170 0x7fdbb8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 0x7fdbb8197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:55406/0 (socket says 172.21.15.113:55406) 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 -- 172.21.15.113:0/3611507202 learned_addr learned my addr 172.21.15.113:0/3611507202 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb77fe640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbb819bf80 0x7fdbb819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 -- 172.21.15.113:0/3611507202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbb819bf80 msgr2=0x7fdbb819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbb819bf80 0x7fdbb819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 -- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb8106170 msgr2=0x7fdbb8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb8106170 0x7fdbb8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb6ffd640 1 -- 172.21.15.113:0/3611507202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb819ca70 con 0x7fdbb81010a0 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.263+0000 7fdbb67fc640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb8106170 0x7fdbb8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.264+0000 7fdbb6ffd640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 0x7fdbb8197710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdba8006010 tx=0x7fdba802cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.264+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8041070 con 0x7fdbb81010a0 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.264+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb819cd00 con 0x7fdbb81010a0 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.264+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdba8004450 con 0x7fdbb81010a0 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.264+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb81a4550 con 0x7fdbb81010a0 2024-04-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.264+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba804b060 con 0x7fdbb81010a0 2024-04-22T07:14:54.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.265+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb8069230 con 0x7fdbb81010a0 2024-04-22T07:14:54.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.266+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fdba8044070 con 0x7fdbb81010a0 2024-04-22T07:14:54.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.266+0000 7fdb9ffff640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdba00a3e40 0x7fdba00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.267+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdba80f2960 con 0x7fdbb81010a0 2024-04-22T07:14:54.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.267+0000 7fdbb67fc640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdba00a3e40 0x7fdba00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.267+0000 7fdbb67fc640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdba00a3e40 0x7fdba00a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdbb81006e0 tx=0x7fdbac005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:54.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.270+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba80beef0 con 0x7fdbb81010a0 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Unable to set osd_memory_target on smithi154 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Unable to set osd_memory_target on smithi133 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Unable to set osd_memory_target on smithi113 to 184773120: error parsing value: Value '184773120' is below minimum 939524096 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-22T07:14:54.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3123166683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/65863107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:54 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Unable to set osd_memory_target on smithi154 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Unable to set osd_memory_target on smithi133 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-04-22T07:14:54.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Unable to set osd_memory_target on smithi113 to 184773120: error parsing value: Value '184773120' is below minimum 939524096 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3123166683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/65863107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:14:54.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:54.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:54.318 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:54 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-22T07:14:54.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Unable to set osd_memory_target on smithi154 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-22T07:14:54.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-22T07:14:54.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Unable to set osd_memory_target on smithi133 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Unable to set osd_memory_target on smithi113 to 184773120: error parsing value: Value '184773120' is below minimum 939524096 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Updating smithi113:/etc/ceph/ceph.conf 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Updating smithi154:/etc/ceph/ceph.conf 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3123166683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/65863107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:14:54.336 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:54 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:14:54.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.453+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdbb8101a50 con 0x7fdba00a3e40 2024-04-22T07:14:54.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.460+0000 7fdb9ffff640 1 -- 172.21.15.113:0/3611507202 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257738 (secure 0 0 0) 0x7fdbb8101a50 con 0x7fdba00a3e40 2024-04-22T07:14:54.461 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:54.463 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:14:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdba00a3e40 msgr2=0x7fdba00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdba00a3e40 0x7fdba00a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdbb81006e0 tx=0x7fdbac005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 msgr2=0x7fdbb8197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 0x7fdbb8197710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdba8006010 tx=0x7fdba802cc40 comp rx=0 tx=0).stop 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 shutdown_connections 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdba00a3e40 0x7fdba00a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbb819bf80 0x7fdbb819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbb8106170 0x7fdbb8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 --2- 172.21.15.113:0/3611507202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbb81010a0 0x7fdbb8197710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 >> 172.21.15.113:0/3611507202 conn(0x7fdbb80771d0 msgr2=0x7fdbb80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 shutdown_connections 2024-04-22T07:14:54.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.463+0000 7fdbbd3b2640 1 -- 172.21.15.113:0/3611507202 wait complete. 2024-04-22T07:14:54.473 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T07:14:53.366437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187876,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":929,"num_read_kb":618,"num_write":467,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368420,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573404,"statfs":{"total":1151940427776,"available":1151563165696,"internally_reserved":0,"allocated":15433728,"data_stored":9202524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990817"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679807+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:14:49.679807+0000","last_peered":"2024-04-22T07:14:49.679807+0000","last_clean":"2024-04-22T07:14:49.679807+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:14:49.679807+0000","last_undegraded":"2024-04-22T07:14:49.679807+0000","last_fullsized":"2024-04-22T07:14:49.679807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566911+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:14:49.566911+0000","last_peered":"2024-04-22T07:14:49.566911+0000","last_clean":"2024-04-22T07:14:49.566911+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:14:49.566911+0000","last_undegraded":"2024-04-22T07:14:49.566911+0000","last_fullsized":"2024-04-22T07:14:49.566911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566794+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:14:49.566794+0000","last_peered":"2024-04-22T07:14:49.566794+0000","last_clean":"2024-04-22T07:14:49.566794+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:14:49.566794+0000","last_undegraded":"2024-04-22T07:14:49.566794+0000","last_fullsized":"2024-04-22T07:14:49.566794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522113+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:14:49.522113+0000","last_peered":"2024-04-22T07:14:49.522113+0000","last_clean":"2024-04-22T07:14:49.522113+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:14:49.522113+0000","last_undegraded":"2024-04-22T07:14:49.522113+0000","last_fullsized":"2024-04-22T07:14:49.522113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446077+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:14:49.446077+0000","last_peered":"2024-04-22T07:14:49.446077+0000","last_clean":"2024-04-22T07:14:49.446077+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:14:49.446077+0000","last_undegraded":"2024-04-22T07:14:49.446077+0000","last_fullsized":"2024-04-22T07:14:49.446077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454433+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:14:49.454433+0000","last_peered":"2024-04-22T07:14:49.454433+0000","last_clean":"2024-04-22T07:14:49.454433+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:14:49.454433+0000","last_undegraded":"2024-04-22T07:14:49.454433+0000","last_fullsized":"2024-04-22T07:14:49.454433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353645+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:14:49.353645+0000","last_peered":"2024-04-22T07:14:49.353645+0000","last_clean":"2024-04-22T07:14:49.353645+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:14:49.353645+0000","last_undegraded":"2024-04-22T07:14:49.353645+0000","last_fullsized":"2024-04-22T07:14:49.353645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446353+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:14:49.446353+0000","last_peered":"2024-04-22T07:14:49.446353+0000","last_clean":"2024-04-22T07:14:49.446353+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:14:49.446353+0000","last_undegraded":"2024-04-22T07:14:49.446353+0000","last_fullsized":"2024-04-22T07:14:49.446353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518246+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:14:49.518246+0000","last_peered":"2024-04-22T07:14:49.518246+0000","last_clean":"2024-04-22T07:14:49.518246+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:14:49.518246+0000","last_undegraded":"2024-04-22T07:14:49.518246+0000","last_fullsized":"2024-04-22T07:14:49.518246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454790+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:14:49.454790+0000","last_peered":"2024-04-22T07:14:49.454790+0000","last_clean":"2024-04-22T07:14:49.454790+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:14:49.454790+0000","last_undegraded":"2024-04-22T07:14:49.454790+0000","last_fullsized":"2024-04-22T07:14:49.454790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353879+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:14:49.353879+0000","last_peered":"2024-04-22T07:14:49.353879+0000","last_clean":"2024-04-22T07:14:49.353879+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:14:49.353879+0000","last_undegraded":"2024-04-22T07:14:49.353879+0000","last_fullsized":"2024-04-22T07:14:49.353879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521981+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:14:49.521981+0000","last_peered":"2024-04-22T07:14:49.521981+0000","last_clean":"2024-04-22T07:14:49.521981+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:14:49.521981+0000","last_undegraded":"2024-04-22T07:14:49.521981+0000","last_fullsized":"2024-04-22T07:14:49.521981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520446+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:14:49.520446+0000","last_peered":"2024-04-22T07:14:49.520446+0000","last_clean":"2024-04-22T07:14:49.520446+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:14:49.520446+0000","last_undegraded":"2024-04-22T07:14:49.520446+0000","last_fullsized":"2024-04-22T07:14:49.520446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567028+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:14:49.567028+0000","last_peered":"2024-04-22T07:14:49.567028+0000","last_clean":"2024-04-22T07:14:49.567028+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:14:49.567028+0000","last_undegraded":"2024-04-22T07:14:49.567028+0000","last_fullsized":"2024-04-22T07:14:49.567028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566423+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:14:49.566423+0000","last_peered":"2024-04-22T07:14:49.566423+0000","last_clean":"2024-04-22T07:14:49.566423+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:14:49.566423+0000","last_undegraded":"2024-04-22T07:14:49.566423+0000","last_fullsized":"2024-04-22T07:14:49.566423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.517925+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:14:49.517925+0000","last_peered":"2024-04-22T07:14:49.517925+0000","last_clean":"2024-04-22T07:14:49.517925+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:14:49.517925+0000","last_undegraded":"2024-04-22T07:14:49.517925+0000","last_fullsized":"2024-04-22T07:14:49.517925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679946+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:14:49.679946+0000","last_peered":"2024-04-22T07:14:49.679946+0000","last_clean":"2024-04-22T07:14:49.679946+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:14:49.679946+0000","last_undegraded":"2024-04-22T07:14:49.679946+0000","last_fullsized":"2024-04-22T07:14:49.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354137+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:14:49.354137+0000","last_peered":"2024-04-22T07:14:49.354137+0000","last_clean":"2024-04-22T07:14:49.354137+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:14:49.354137+0000","last_undegraded":"2024-04-22T07:14:49.354137+0000","last_fullsized":"2024-04-22T07:14:49.354137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356040+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:14:49.356040+0000","last_peered":"2024-04-22T07:14:49.356040+0000","last_clean":"2024-04-22T07:14:49.356040+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:14:49.356040+0000","last_undegraded":"2024-04-22T07:14:49.356040+0000","last_fullsized":"2024-04-22T07:14:49.356040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354290+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:14:49.354290+0000","last_peered":"2024-04-22T07:14:49.354290+0000","last_clean":"2024-04-22T07:14:49.354290+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:14:49.354290+0000","last_undegraded":"2024-04-22T07:14:49.354290+0000","last_fullsized":"2024-04-22T07:14:49.354290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447115+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:14:49.447115+0000","last_peered":"2024-04-22T07:14:49.447115+0000","last_clean":"2024-04-22T07:14:49.447115+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:14:49.447115+0000","last_undegraded":"2024-04-22T07:14:49.447115+0000","last_fullsized":"2024-04-22T07:14:49.447115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447288+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:14:49.447288+0000","last_peered":"2024-04-22T07:14:49.447288+0000","last_clean":"2024-04-22T07:14:49.447288+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:14:49.447288+0000","last_undegraded":"2024-04-22T07:14:49.447288+0000","last_fullsized":"2024-04-22T07:14:49.447288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567072+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:14:49.567072+0000","last_peered":"2024-04-22T07:14:49.567072+0000","last_clean":"2024-04-22T07:14:49.567072+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:14:49.567072+0000","last_undegraded":"2024-04-22T07:14:49.567072+0000","last_fullsized":"2024-04-22T07:14:49.567072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446441+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:14:49.446441+0000","last_peered":"2024-04-22T07:14:49.446441+0000","last_clean":"2024-04-22T07:14:49.446441+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:14:49.446441+0000","last_undegraded":"2024-04-22T07:14:49.446441+0000","last_fullsized":"2024-04-22T07:14:49.446441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518065+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:14:49.518065+0000","last_peered":"2024-04-22T07:14:49.518065+0000","last_clean":"2024-04-22T07:14:49.518065+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:14:49.518065+0000","last_undegraded":"2024-04-22T07:14:49.518065+0000","last_fullsized":"2024-04-22T07:14:49.518065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679550+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:14:49.679550+0000","last_peered":"2024-04-22T07:14:49.679550+0000","last_clean":"2024-04-22T07:14:49.679550+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:14:49.679550+0000","last_undegraded":"2024-04-22T07:14:49.679550+0000","last_fullsized":"2024-04-22T07:14:49.679550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451783+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:14:49.451783+0000","last_peered":"2024-04-22T07:14:49.451783+0000","last_clean":"2024-04-22T07:14:49.451783+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:14:49.451783+0000","last_undegraded":"2024-04-22T07:14:49.451783+0000","last_fullsized":"2024-04-22T07:14:49.451783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.500064+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:49.500064+0000","last_peered":"2024-04-22T07:14:49.500064+0000","last_clean":"2024-04-22T07:14:49.500064+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:49.500064+0000","last_undegraded":"2024-04-22T07:14:49.500064+0000","last_fullsized":"2024-04-22T07:14:49.500064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446021+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:14:49.446021+0000","last_peered":"2024-04-22T07:14:49.446021+0000","last_clean":"2024-04-22T07:14:49.446021+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:14:49.446021+0000","last_undegraded":"2024-04-22T07:14:49.446021+0000","last_fullsized":"2024-04-22T07:14:49.446021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454363+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:14:49.454363+0000","last_peered":"2024-04-22T07:14:49.454363+0000","last_clean":"2024-04-22T07:14:49.454363+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:14:49.454363+0000","last_undegraded":"2024-04-22T07:14:49.454363+0000","last_fullsized":"2024-04-22T07:14:49.454363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451662+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:14:49.451662+0000","last_peered":"2024-04-22T07:14:49.451662+0000","last_clean":"2024-04-22T07:14:49.451662+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:14:49.451662+0000","last_undegraded":"2024-04-22T07:14:49.451662+0000","last_fullsized":"2024-04-22T07:14:49.451662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520117+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:14:49.520117+0000","last_peered":"2024-04-22T07:14:49.520117+0000","last_clean":"2024-04-22T07:14:49.520117+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:14:49.520117+0000","last_undegraded":"2024-04-22T07:14:49.520117+0000","last_fullsized":"2024-04-22T07:14:49.520117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446512+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:14:49.446512+0000","last_peered":"2024-04-22T07:14:49.446512+0000","last_clean":"2024-04-22T07:14:49.446512+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:14:49.446512+0000","last_undegraded":"2024-04-22T07:14:49.446512+0000","last_fullsized":"2024-04-22T07:14:49.446512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451463+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:14:49.451463+0000","last_peered":"2024-04-22T07:14:49.451463+0000","last_clean":"2024-04-22T07:14:49.451463+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:14:49.451463+0000","last_undegraded":"2024-04-22T07:14:49.451463+0000","last_fullsized":"2024-04-22T07:14:49.451463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354021+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:14:49.354021+0000","last_peered":"2024-04-22T07:14:49.354021+0000","last_clean":"2024-04-22T07:14:49.354021+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:14:49.354021+0000","last_undegraded":"2024-04-22T07:14:49.354021+0000","last_fullsized":"2024-04-22T07:14:49.354021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454848+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:49.454848+0000","last_peered":"2024-04-22T07:14:49.454848+0000","last_clean":"2024-04-22T07:14:49.454848+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:49.454848+0000","last_undegraded":"2024-04-22T07:14:49.454848+0000","last_fullsized":"2024-04-22T07:14:49.454848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518517+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:14:49.518517+0000","last_peered":"2024-04-22T07:14:49.518517+0000","last_clean":"2024-04-22T07:14:49.518517+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:14:49.518517+0000","last_undegraded":"2024-04-22T07:14:49.518517+0000","last_fullsized":"2024-04-22T07:14:49.518517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353921+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:14:49.353921+0000","last_peered":"2024-04-22T07:14:49.353921+0000","last_clean":"2024-04-22T07:14:49.353921+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:14:49.353921+0000","last_undegraded":"2024-04-22T07:14:49.353921+0000","last_fullsized":"2024-04-22T07:14:49.353921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567045+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:14:49.567045+0000","last_peered":"2024-04-22T07:14:49.567045+0000","last_clean":"2024-04-22T07:14:49.567045+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:14:49.567045+0000","last_undegraded":"2024-04-22T07:14:49.567045+0000","last_fullsized":"2024-04-22T07:14:49.567045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451908+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:14:49.451908+0000","last_peered":"2024-04-22T07:14:49.451908+0000","last_clean":"2024-04-22T07:14:49.451908+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:14:49.451908+0000","last_undegraded":"2024-04-22T07:14:49.451908+0000","last_fullsized":"2024-04-22T07:14:49.451908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518495+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:14:49.518495+0000","last_peered":"2024-04-22T07:14:49.518495+0000","last_clean":"2024-04-22T07:14:49.518495+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:14:49.518495+0000","last_undegraded":"2024-04-22T07:14:49.518495+0000","last_fullsized":"2024-04-22T07:14:49.518495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.596414+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:49.596414+0000","last_peered":"2024-04-22T07:14:49.596414+0000","last_clean":"2024-04-22T07:14:49.596414+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:49.596414+0000","last_undegraded":"2024-04-22T07:14:49.596414+0000","last_fullsized":"2024-04-22T07:14:49.596414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":177,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446273+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:14:49.446273+0000","last_peered":"2024-04-22T07:14:49.446273+0000","last_clean":"2024-04-22T07:14:49.446273+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:14:49.446273+0000","last_undegraded":"2024-04-22T07:14:49.446273+0000","last_fullsized":"2024-04-22T07:14:49.446273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679544+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:14:49.679544+0000","last_peered":"2024-04-22T07:14:49.679544+0000","last_clean":"2024-04-22T07:14:49.679544+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:14:49.679544+0000","last_undegraded":"2024-04-22T07:14:49.679544+0000","last_fullsized":"2024-04-22T07:14:49.679544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520133+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:14:49.520133+0000","last_peered":"2024-04-22T07:14:49.520133+0000","last_clean":"2024-04-22T07:14:49.520133+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:14:49.520133+0000","last_undegraded":"2024-04-22T07:14:49.520133+0000","last_fullsized":"2024-04-22T07:14:49.520133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451635+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:14:49.451635+0000","last_peered":"2024-04-22T07:14:49.451635+0000","last_clean":"2024-04-22T07:14:49.451635+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:14:49.451635+0000","last_undegraded":"2024-04-22T07:14:49.451635+0000","last_fullsized":"2024-04-22T07:14:49.451635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353878+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:14:49.353878+0000","last_peered":"2024-04-22T07:14:49.353878+0000","last_clean":"2024-04-22T07:14:49.353878+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:14:49.353878+0000","last_undegraded":"2024-04-22T07:14:49.353878+0000","last_fullsized":"2024-04-22T07:14:49.353878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522205+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:49.522205+0000","last_peered":"2024-04-22T07:14:49.522205+0000","last_clean":"2024-04-22T07:14:49.522205+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:49.522205+0000","last_undegraded":"2024-04-22T07:14:49.522205+0000","last_fullsized":"2024-04-22T07:14:49.522205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353610+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:14:49.353610+0000","last_peered":"2024-04-22T07:14:49.353610+0000","last_clean":"2024-04-22T07:14:49.353610+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:14:49.353610+0000","last_undegraded":"2024-04-22T07:14:49.353610+0000","last_fullsized":"2024-04-22T07:14:49.353610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446009+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:14:49.446009+0000","last_peered":"2024-04-22T07:14:49.446009+0000","last_clean":"2024-04-22T07:14:49.446009+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:14:49.446009+0000","last_undegraded":"2024-04-22T07:14:49.446009+0000","last_fullsized":"2024-04-22T07:14:49.446009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451558+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:14:49.451558+0000","last_peered":"2024-04-22T07:14:49.451558+0000","last_clean":"2024-04-22T07:14:49.451558+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:14:49.451558+0000","last_undegraded":"2024-04-22T07:14:49.451558+0000","last_fullsized":"2024-04-22T07:14:49.451558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446380+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:14:49.446380+0000","last_peered":"2024-04-22T07:14:49.446380+0000","last_clean":"2024-04-22T07:14:49.446380+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:14:49.446380+0000","last_undegraded":"2024-04-22T07:14:49.446380+0000","last_fullsized":"2024-04-22T07:14:49.446380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679730+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:14:49.679730+0000","last_peered":"2024-04-22T07:14:49.679730+0000","last_clean":"2024-04-22T07:14:49.679730+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:14:49.679730+0000","last_undegraded":"2024-04-22T07:14:49.679730+0000","last_fullsized":"2024-04-22T07:14:49.679730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354097+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:14:49.354097+0000","last_peered":"2024-04-22T07:14:49.354097+0000","last_clean":"2024-04-22T07:14:49.354097+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:14:49.354097+0000","last_undegraded":"2024-04-22T07:14:49.354097+0000","last_fullsized":"2024-04-22T07:14:49.354097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454763+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:14:49.454763+0000","last_peered":"2024-04-22T07:14:49.454763+0000","last_clean":"2024-04-22T07:14:49.454763+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:14:49.454763+0000","last_undegraded":"2024-04-22T07:14:49.454763+0000","last_fullsized":"2024-04-22T07:14:49.454763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353758+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:14:49.353758+0000","last_peered":"2024-04-22T07:14:49.353758+0000","last_clean":"2024-04-22T07:14:49.353758+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:14:49.353758+0000","last_undegraded":"2024-04-22T07:14:49.353758+0000","last_fullsized":"2024-04-22T07:14:49.353758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454649+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:14:49.454649+0000","last_peered":"2024-04-22T07:14:49.454649+0000","last_clean":"2024-04-22T07:14:49.454649+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:14:49.454649+0000","last_undegraded":"2024-04-22T07:14:49.454649+0000","last_fullsized":"2024-04-22T07:14:49.454649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451592+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:14:49.451592+0000","last_peered":"2024-04-22T07:14:49.451592+0000","last_clean":"2024-04-22T07:14:49.451592+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:14:49.451592+0000","last_undegraded":"2024-04-22T07:14:49.451592+0000","last_fullsized":"2024-04-22T07:14:49.451592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":138,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520301+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:49.520301+0000","last_peered":"2024-04-22T07:14:49.520301+0000","last_clean":"2024-04-22T07:14:49.520301+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:49.520301+0000","last_undegraded":"2024-04-22T07:14:49.520301+0000","last_fullsized":"2024-04-22T07:14:49.520301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446669+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:14:49.446669+0000","last_peered":"2024-04-22T07:14:49.446669+0000","last_clean":"2024-04-22T07:14:49.446669+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:14:49.446669+0000","last_undegraded":"2024-04-22T07:14:49.446669+0000","last_fullsized":"2024-04-22T07:14:49.446669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521956+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:14:49.521956+0000","last_peered":"2024-04-22T07:14:49.521956+0000","last_clean":"2024-04-22T07:14:49.521956+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:14:49.521956+0000","last_undegraded":"2024-04-22T07:14:49.521956+0000","last_fullsized":"2024-04-22T07:14:49.521956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353712+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:14:49.353712+0000","last_peered":"2024-04-22T07:14:49.353712+0000","last_clean":"2024-04-22T07:14:49.353712+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:14:49.353712+0000","last_undegraded":"2024-04-22T07:14:49.353712+0000","last_fullsized":"2024-04-22T07:14:49.353712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.519969+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:14:49.519969+0000","last_peered":"2024-04-22T07:14:49.519969+0000","last_clean":"2024-04-22T07:14:49.519969+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:14:49.519969+0000","last_undegraded":"2024-04-22T07:14:49.519969+0000","last_fullsized":"2024-04-22T07:14:49.519969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353812+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:14:49.353812+0000","last_peered":"2024-04-22T07:14:49.353812+0000","last_clean":"2024-04-22T07:14:49.353812+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:14:49.353812+0000","last_undegraded":"2024-04-22T07:14:49.353812+0000","last_fullsized":"2024-04-22T07:14:49.353812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446310+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:14:49.446310+0000","last_peered":"2024-04-22T07:14:49.446310+0000","last_clean":"2024-04-22T07:14:49.446310+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:14:49.446310+0000","last_undegraded":"2024-04-22T07:14:49.446310+0000","last_fullsized":"2024-04-22T07:14:49.446310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520318+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:14:49.520318+0000","last_peered":"2024-04-22T07:14:49.520318+0000","last_clean":"2024-04-22T07:14:49.520318+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:14:49.520318+0000","last_undegraded":"2024-04-22T07:14:49.520318+0000","last_fullsized":"2024-04-22T07:14:49.520318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.559202+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:14:49.559202+0000","last_peered":"2024-04-22T07:14:49.559202+0000","last_clean":"2024-04-22T07:14:49.559202+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:14:49.559202+0000","last_undegraded":"2024-04-22T07:14:49.559202+0000","last_fullsized":"2024-04-22T07:14:49.559202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446988+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:14:49.446988+0000","last_peered":"2024-04-22T07:14:49.446988+0000","last_clean":"2024-04-22T07:14:49.446988+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:14:49.446988+0000","last_undegraded":"2024-04-22T07:14:49.446988+0000","last_fullsized":"2024-04-22T07:14:49.446988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522148+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:14:49.522148+0000","last_peered":"2024-04-22T07:14:49.522148+0000","last_clean":"2024-04-22T07:14:49.522148+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:14:49.522148+0000","last_undegraded":"2024-04-22T07:14:49.522148+0000","last_fullsized":"2024-04-22T07:14:49.522148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566877+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:14:49.566877+0000","last_peered":"2024-04-22T07:14:49.566877+0000","last_clean":"2024-04-22T07:14:49.566877+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:14:49.566877+0000","last_undegraded":"2024-04-22T07:14:49.566877+0000","last_fullsized":"2024-04-22T07:14:49.566877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518008+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:14:49.518008+0000","last_peered":"2024-04-22T07:14:49.518008+0000","last_clean":"2024-04-22T07:14:49.518008+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:14:49.518008+0000","last_undegraded":"2024-04-22T07:14:49.518008+0000","last_fullsized":"2024-04-22T07:14:49.518008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446005+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:14:49.446005+0000","last_peered":"2024-04-22T07:14:49.446005+0000","last_clean":"2024-04-22T07:14:49.446005+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:14:49.446005+0000","last_undegraded":"2024-04-22T07:14:49.446005+0000","last_fullsized":"2024-04-22T07:14:49.446005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446529+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:14:49.446529+0000","last_peered":"2024-04-22T07:14:49.446529+0000","last_clean":"2024-04-22T07:14:49.446529+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:14:49.446529+0000","last_undegraded":"2024-04-22T07:14:49.446529+0000","last_fullsized":"2024-04-22T07:14:49.446529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679532+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:14:49.679532+0000","last_peered":"2024-04-22T07:14:49.679532+0000","last_clean":"2024-04-22T07:14:49.679532+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:14:49.679532+0000","last_undegraded":"2024-04-22T07:14:49.679532+0000","last_fullsized":"2024-04-22T07:14:49.679532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355979+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:14:49.355979+0000","last_peered":"2024-04-22T07:14:49.355979+0000","last_clean":"2024-04-22T07:14:49.355979+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:14:49.355979+0000","last_undegraded":"2024-04-22T07:14:49.355979+0000","last_fullsized":"2024-04-22T07:14:49.355979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446724+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:14:49.446724+0000","last_peered":"2024-04-22T07:14:49.446724+0000","last_clean":"2024-04-22T07:14:49.446724+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:14:49.446724+0000","last_undegraded":"2024-04-22T07:14:49.446724+0000","last_fullsized":"2024-04-22T07:14:49.446724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446129+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:14:49.446129+0000","last_peered":"2024-04-22T07:14:49.446129+0000","last_clean":"2024-04-22T07:14:49.446129+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:14:49.446129+0000","last_undegraded":"2024-04-22T07:14:49.446129+0000","last_fullsized":"2024-04-22T07:14:49.446129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353868+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:14:49.353868+0000","last_peered":"2024-04-22T07:14:49.353868+0000","last_clean":"2024-04-22T07:14:49.353868+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:14:49.353868+0000","last_undegraded":"2024-04-22T07:14:49.353868+0000","last_fullsized":"2024-04-22T07:14:49.353868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356164+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:14:49.356164+0000","last_peered":"2024-04-22T07:14:49.356164+0000","last_clean":"2024-04-22T07:14:49.356164+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:14:49.356164+0000","last_undegraded":"2024-04-22T07:14:49.356164+0000","last_fullsized":"2024-04-22T07:14:49.356164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354047+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:14:49.354047+0000","last_peered":"2024-04-22T07:14:49.354047+0000","last_clean":"2024-04-22T07:14:49.354047+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:14:49.354047+0000","last_undegraded":"2024-04-22T07:14:49.354047+0000","last_fullsized":"2024-04-22T07:14:49.354047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.553901+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:14:49.553901+0000","last_peered":"2024-04-22T07:14:49.553901+0000","last_clean":"2024-04-22T07:14:49.553901+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:14:49.553901+0000","last_undegraded":"2024-04-22T07:14:49.553901+0000","last_fullsized":"2024-04-22T07:14:49.553901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353880+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:14:49.353880+0000","last_peered":"2024-04-22T07:14:49.353880+0000","last_clean":"2024-04-22T07:14:49.353880+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:14:49.353880+0000","last_undegraded":"2024-04-22T07:14:49.353880+0000","last_fullsized":"2024-04-22T07:14:49.353880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520280+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:14:49.520280+0000","last_peered":"2024-04-22T07:14:49.520280+0000","last_clean":"2024-04-22T07:14:49.520280+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:14:49.520280+0000","last_undegraded":"2024-04-22T07:14:49.520280+0000","last_fullsized":"2024-04-22T07:14:49.520280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354174+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:14:49.354174+0000","last_peered":"2024-04-22T07:14:49.354174+0000","last_clean":"2024-04-22T07:14:49.354174+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:14:49.354174+0000","last_undegraded":"2024-04-22T07:14:49.354174+0000","last_fullsized":"2024-04-22T07:14:49.354174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679633+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:14:49.679633+0000","last_peered":"2024-04-22T07:14:49.679633+0000","last_clean":"2024-04-22T07:14:49.679633+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:14:49.679633+0000","last_undegraded":"2024-04-22T07:14:49.679633+0000","last_fullsized":"2024-04-22T07:14:49.679633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355972+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:14:49.355972+0000","last_peered":"2024-04-22T07:14:49.355972+0000","last_clean":"2024-04-22T07:14:49.355972+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:14:49.355972+0000","last_undegraded":"2024-04-22T07:14:49.355972+0000","last_fullsized":"2024-04-22T07:14:49.355972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355819+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:14:49.355819+0000","last_peered":"2024-04-22T07:14:49.355819+0000","last_clean":"2024-04-22T07:14:49.355819+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:14:49.355819+0000","last_undegraded":"2024-04-22T07:14:49.355819+0000","last_fullsized":"2024-04-22T07:14:49.355819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566665+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:14:49.566665+0000","last_peered":"2024-04-22T07:14:49.566665+0000","last_clean":"2024-04-22T07:14:49.566665+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:14:49.566665+0000","last_undegraded":"2024-04-22T07:14:49.566665+0000","last_fullsized":"2024-04-22T07:14:49.566665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520295+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:14:49.520295+0000","last_peered":"2024-04-22T07:14:49.520295+0000","last_clean":"2024-04-22T07:14:49.520295+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:14:49.520295+0000","last_undegraded":"2024-04-22T07:14:49.520295+0000","last_fullsized":"2024-04-22T07:14:49.520295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679687+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:14:49.679687+0000","last_peered":"2024-04-22T07:14:49.679687+0000","last_clean":"2024-04-22T07:14:49.679687+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:14:49.679687+0000","last_undegraded":"2024-04-22T07:14:49.679687+0000","last_fullsized":"2024-04-22T07:14:49.679687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356248+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:14:49.356248+0000","last_peered":"2024-04-22T07:14:49.356248+0000","last_clean":"2024-04-22T07:14:49.356248+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:14:49.356248+0000","last_undegraded":"2024-04-22T07:14:49.356248+0000","last_fullsized":"2024-04-22T07:14:49.356248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354138+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:14:49.354138+0000","last_peered":"2024-04-22T07:14:49.354138+0000","last_clean":"2024-04-22T07:14:49.354138+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:14:49.354138+0000","last_undegraded":"2024-04-22T07:14:49.354138+0000","last_fullsized":"2024-04-22T07:14:49.354138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451527+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:14:49.451527+0000","last_peered":"2024-04-22T07:14:49.451527+0000","last_clean":"2024-04-22T07:14:49.451527+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:14:49.451527+0000","last_undegraded":"2024-04-22T07:14:49.451527+0000","last_fullsized":"2024-04-22T07:14:49.451527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446375+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:14:49.446375+0000","last_peered":"2024-04-22T07:14:49.446375+0000","last_clean":"2024-04-22T07:14:49.446375+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:14:49.446375+0000","last_undegraded":"2024-04-22T07:14:49.446375+0000","last_fullsized":"2024-04-22T07:14:49.446375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518310+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:14:49.518310+0000","last_peered":"2024-04-22T07:14:49.518310+0000","last_clean":"2024-04-22T07:14:49.518310+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:14:49.518310+0000","last_undegraded":"2024-04-22T07:14:49.518310+0000","last_fullsized":"2024-04-22T07:14:49.518310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520315+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:14:49.520315+0000","last_peered":"2024-04-22T07:14:49.520315+0000","last_clean":"2024-04-22T07:14:49.520315+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:14:49.520315+0000","last_undegraded":"2024-04-22T07:14:49.520315+0000","last_fullsized":"2024-04-22T07:14:49.520315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679771+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:14:49.679771+0000","last_peered":"2024-04-22T07:14:49.679771+0000","last_clean":"2024-04-22T07:14:49.679771+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:14:49.679771+0000","last_undegraded":"2024-04-22T07:14:49.679771+0000","last_fullsized":"2024-04-22T07:14:49.679771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597274,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":793,"num_read_kb":499,"num_write":284,"num_write_kb":1434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1261720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36688,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708464,"statfs":{"total":95995035648,"available":95957467136,"internally_reserved":0,"allocated":1380352,"data_stored":863942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1376256,"data_stored":863280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1581056,"data_stored":1063275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":538248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":471386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":539681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":995242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":470943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1728512,"data_stored":1194168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":68340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:14:54.477 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 -- 172.21.15.113:0/2924004356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210107f70 msgr2=0x7f6210108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 --2- 172.21.15.113:0/2924004356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210107f70 0x7f6210108350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6204009930 tx=0x7f620402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 -- 172.21.15.113:0/2924004356 shutdown_connections 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 --2- 172.21.15.113:0/2924004356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210069a80 0x7f621010ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 --2- 172.21.15.113:0/2924004356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 0x7f6210069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 --2- 172.21.15.113:0/2924004356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210107f70 0x7f6210108350 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 -- 172.21.15.113:0/2924004356 >> 172.21.15.113:0/2924004356 conn(0x7f6210077e30 msgr2=0x7f6210078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 -- 172.21.15.113:0/2924004356 shutdown_connections 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.617+0000 7f6215e06640 1 -- 172.21.15.113:0/2924004356 wait complete. 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.618+0000 7f6215e06640 1 Processor -- start 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.618+0000 7f6215e06640 1 -- start start 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.618+0000 7f6215e06640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 0x7f6210101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.618+0000 7f6215e06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 0x7f62101017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f6215e06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210107f70 0x7f6210101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f6215e06640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f62101122d0 con 0x7f6210069a80 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f6215e06640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6210112150 con 0x7f6210107f70 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f6215e06640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6210112450 con 0x7f62100690e0 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 0x7f62101017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 0x7f62101017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53050/0 (socket says 172.21.15.113:53050) 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 -- 172.21.15.113:0/3153639105 learned_addr learned my addr 172.21.15.113:0/3153639105 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620f7fe640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 0x7f6210101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620ffff640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210107f70 0x7f6210101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 -- 172.21.15.113:0/3153639105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 msgr2=0x7f6210101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 0x7f6210101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 -- 172.21.15.113:0/3153639105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210107f70 msgr2=0x7f6210101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210107f70 0x7f6210101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 -- 172.21.15.113:0/3153639105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62101a6c10 con 0x7f6210069a80 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620f7fe640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 0x7f6210101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620ffff640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210107f70 0x7f6210101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620effd640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 0x7f62101017b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f620000e9f0 tx=0x7f620000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000cd80 con 0x7f6210069a80 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f62000054f0 con 0x7f6210069a80 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62101a6f00 con 0x7f6210069a80 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200010690 con 0x7f6210069a80 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.619+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62101a7370 con 0x7f6210069a80 2024-04-22T07:14:54.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.620+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6210068de0 con 0x7f6210069a80 2024-04-22T07:14:54.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.621+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f62000046c0 con 0x7f6210069a80 2024-04-22T07:14:54.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.622+0000 7f620cff9640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61f80a3e70 0x7f61f80a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.622+0000 7f620f7fe640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61f80a3e70 0x7f61f80a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.622+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f62000d0ed0 con 0x7f6210069a80 2024-04-22T07:14:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.623+0000 7f620f7fe640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61f80a3e70 0x7f61f80a6330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6204006010 tx=0x7f620402c6b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.627+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f620009d350 con 0x7f6210069a80 2024-04-22T07:14:54.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.809+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f621002d5c0 con 0x7f6210069a80 2024-04-22T07:14:54.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.810+0000 7f620cff9640 1 -- 172.21.15.113:0/3153639105 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7f620009cc50 con 0x7f6210069a80 2024-04-22T07:14:54.809 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:54.809 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:54.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61f80a3e70 msgr2=0x7f61f80a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61f80a3e70 0x7f61f80a6330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6204006010 tx=0x7f620402c6b0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 msgr2=0x7f62101017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 0x7f62101017b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f620000e9f0 tx=0x7f620000edf0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 shutdown_connections 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61f80a3e70 0x7f61f80a6330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6210107f70 0x7f6210101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6210069a80 0x7f62101017b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 --2- 172.21.15.113:0/3153639105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62100690e0 0x7f6210101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.812+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 >> 172.21.15.113:0/3153639105 conn(0x7f6210077e30 msgr2=0x7f62100fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.813+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 shutdown_connections 2024-04-22T07:14:54.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.813+0000 7f6215e06640 1 -- 172.21.15.113:0/3153639105 wait complete. 2024-04-22T07:14:54.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:54.823 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/1494215219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8107270 msgr2=0x7f7bb810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/1494215219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8107270 0x7f7bb810ee40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009ca0 tx=0x7f7bb002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/1494215219 shutdown_connections 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/1494215219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8107270 0x7f7bb810ee40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/1494215219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb8102b50 0x7f7bb8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/1494215219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb81021a0 0x7f7bb8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/1494215219 >> 172.21.15.113:0/1494215219 conn(0x7f7bb80fd9c0 msgr2=0x7f7bb80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/1494215219 shutdown_connections 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/1494215219 wait complete. 2024-04-22T07:14:54.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.925+0000 7f7bbd3b6640 1 Processor -- start 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 -- start start 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb81021a0 0x7f7bb819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb8107270 0x7f7bb819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 0x7f7bb8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8112310 con 0x7f7bb8198650 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8112190 con 0x7f7bb81021a0 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bbd3b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8112490 con 0x7f7bb8107270 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 0x7f7bb8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 0x7f7bb8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53064/0 (socket says 172.21.15.113:53064) 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 -- 172.21.15.113:0/864883207 learned_addr learned my addr 172.21.15.113:0/864883207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb67fc640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb8107270 0x7f7bb819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb6ffd640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb81021a0 0x7f7bb819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 -- 172.21.15.113:0/864883207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb8107270 msgr2=0x7f7bb819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb8107270 0x7f7bb819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 -- 172.21.15.113:0/864883207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb81021a0 msgr2=0x7f7bb819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb81021a0 0x7f7bb819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 -- 172.21.15.113:0/864883207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb8199250 con 0x7f7bb8198650 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.926+0000 7f7bb77fe640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 0x7f7bb8198ad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bb002c700 tx=0x7f7bb0035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.927+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0039040 con 0x7f7bb8198650 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.927+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7bb0044070 con 0x7f7bb8198650 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.927+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0004330 con 0x7f7bb8198650 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.927+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb81994e0 con 0x7f7bb8198650 2024-04-22T07:14:54.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.927+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb81a5620 con 0x7f7bb8198650 2024-04-22T07:14:54.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.928+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8103670 con 0x7f7bb8198650 2024-04-22T07:14:54.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.929+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f7bb00044d0 con 0x7f7bb8198650 2024-04-22T07:14:54.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.930+0000 7f7baffff640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7ba40a3e40 0x7f7ba40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:54.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.930+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f7bb00f3440 con 0x7f7bb8198650 2024-04-22T07:14:54.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.930+0000 7f7bb6ffd640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7ba40a3e40 0x7f7ba40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:54.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.931+0000 7f7bb6ffd640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7ba40a3e40 0x7f7ba40a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009bd0 tx=0x7f7ba0005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:54.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:54.934+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bb00bfaf0 con 0x7f7bb8198650 2024-04-22T07:14:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T07:14:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:55.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:55 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3153639105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:55.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.116+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7bb8102b50 con 0x7f7bb8198650 2024-04-22T07:14:55.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.117+0000 7f7baffff640 1 -- 172.21.15.113:0/864883207 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f7bb00bf3f0 con 0x7f7bb8198650 2024-04-22T07:14:55.115 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7ba40a3e40 msgr2=0x7f7ba40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7ba40a3e40 0x7f7ba40a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009bd0 tx=0x7f7ba0005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 msgr2=0x7f7bb8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 0x7f7bb8198ad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bb002c700 tx=0x7f7bb0035d80 comp rx=0 tx=0).stop 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 shutdown_connections 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7ba40a3e40 0x7f7ba40a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb8198650 0x7f7bb8198ad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bb8107270 0x7f7bb819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 --2- 172.21.15.113:0/864883207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bb81021a0 0x7f7bb819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 >> 172.21.15.113:0/864883207 conn(0x7f7bb80fd9c0 msgr2=0x7f7bb80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:55.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 shutdown_connections 2024-04-22T07:14:55.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.119+0000 7f7bbd3b6640 1 -- 172.21.15.113:0/864883207 wait complete. 2024-04-22T07:14:55.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:55.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 -- 172.21.15.113:0/732453225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14106160 msgr2=0x7fde1410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 --2- 172.21.15.113:0/732453225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14106160 0x7fde1410dd30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009ca0 tx=0x7fde0c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 -- 172.21.15.113:0/732453225 shutdown_connections 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 --2- 172.21.15.113:0/732453225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14106160 0x7fde1410dd30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 --2- 172.21.15.113:0/732453225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14101a40 0x7fde14105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 --2- 172.21.15.113:0/732453225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde14101090 0x7fde14101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 -- 172.21.15.113:0/732453225 >> 172.21.15.113:0/732453225 conn(0x7fde14077f50 msgr2=0x7fde14078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 -- 172.21.15.113:0/732453225 shutdown_connections 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.235+0000 7fde199bd640 1 -- 172.21.15.113:0/732453225 wait complete. 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 Processor -- start 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 -- start start 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde14101090 0x7fde14197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 0x7fde14197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 0x7fde141a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fde141111e0 con 0x7fde14197db0 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde14111060 con 0x7fde14106160 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde199bd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde14111360 con 0x7fde14101090 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde137fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 0x7fde141a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 0x7fde14197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde137fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 0x7fde141a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53084/0 (socket says 172.21.15.113:53084) 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 0x7fde14197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:55474/0 (socket says 172.21.15.113:55474) 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde137fe640 1 -- 172.21.15.113:0/4057051084 learned_addr learned my addr 172.21.15.113:0/4057051084 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.236+0000 7fde12ffd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde14101090 0x7fde14197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde137fe640 1 -- 172.21.15.113:0/4057051084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde14101090 msgr2=0x7fde14197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde137fe640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde14101090 0x7fde14197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde137fe640 1 -- 172.21.15.113:0/4057051084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 msgr2=0x7fde14197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde137fe640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 0x7fde14197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde137fe640 1 -- 172.21.15.113:0/4057051084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde141a6fc0 con 0x7fde14197db0 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde0bfff640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 0x7fde14197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde137fe640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 0x7fde141a69b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009dc0 tx=0x7fde0c035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c039040 con 0x7fde14197db0 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fde0c044070 con 0x7fde14197db0 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde141a7250 con 0x7fde14197db0 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c004370 con 0x7fde14197db0 2024-04-22T07:14:55.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.237+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde141a7740 con 0x7fde14197db0 2024-04-22T07:14:55.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.238+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde14068d70 con 0x7fde14197db0 2024-04-22T07:14:55.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.239+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fde0c004510 con 0x7fde14197db0 2024-04-22T07:14:55.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.240+0000 7fde189bb640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fde000a3e40 0x7fde000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.240+0000 7fde12ffd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fde000a3e40 0x7fde000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.240+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fde0c0f2e90 con 0x7fde14197db0 2024-04-22T07:14:55.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.240+0000 7fde12ffd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fde000a3e40 0x7fde000a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde0400abe0 tx=0x7fde0400a2d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:55.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.244+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde0c0bf420 con 0x7fde14197db0 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:55.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T07:14:55.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:55.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:55 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3153639105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: Updating smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: Updating smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: Updating smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:55.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:55 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3153639105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.425+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde14101b50 con 0x7fde14197db0 2024-04-22T07:14:55.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.425+0000 7fde189bb640 1 -- 172.21.15.113:0/4057051084 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fde0c0bed20 con 0x7fde14197db0 2024-04-22T07:14:55.424 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fde000a3e40 msgr2=0x7fde000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fde000a3e40 0x7fde000a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde0400abe0 tx=0x7fde0400a2d0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 msgr2=0x7fde141a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 0x7fde141a69b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009dc0 tx=0x7fde0c035d80 comp rx=0 tx=0).stop 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 shutdown_connections 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fde000a3e40 0x7fde000a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14197db0 0x7fde141a69b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde14106160 0x7fde14197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 --2- 172.21.15.113:0/4057051084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde14101090 0x7fde14197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 >> 172.21.15.113:0/4057051084 conn(0x7fde14077f50 msgr2=0x7fde140fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 shutdown_connections 2024-04-22T07:14:55.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.427+0000 7fde199bd640 1 -- 172.21.15.113:0/4057051084 wait complete. 2024-04-22T07:14:55.437 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 -- 172.21.15.113:0/1186132318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c101a40 msgr2=0x7f2f4c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 --2- 172.21.15.113:0/1186132318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c101a40 0x7f2f4c105a80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009900 tx=0x7f2f3c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 -- 172.21.15.113:0/1186132318 shutdown_connections 2024-04-22T07:14:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 --2- 172.21.15.113:0/1186132318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4c106160 0x7f2f4c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 --2- 172.21.15.113:0/1186132318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c101a40 0x7f2f4c105a80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 --2- 172.21.15.113:0/1186132318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 0x7f2f4c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 -- 172.21.15.113:0/1186132318 >> 172.21.15.113:0/1186132318 conn(0x7f2f4c077f50 msgr2=0x7f2f4c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 -- 172.21.15.113:0/1186132318 shutdown_connections 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 -- 172.21.15.113:0/1186132318 wait complete. 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 Processor -- start 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.540+0000 7f2f52877640 1 -- start start 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 0x7f2f4c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 0x7f2f4c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4c1975a0 0x7f2f4c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c111240 con 0x7f2f4c106160 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1110c0 con 0x7f2f4c1975a0 2024-04-22T07:14:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1113c0 con 0x7f2f4c101090 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 0x7f2f4c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 0x7f2f4c19d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53108/0 (socket says 172.21.15.113:53108) 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 -- 172.21.15.113:0/4239673417 learned_addr learned my addr 172.21.15.113:0/4239673417 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4bfff640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 0x7f2f4c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 -- 172.21.15.113:0/4239673417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 msgr2=0x7f2f4c19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 0x7f2f4c19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 -- 172.21.15.113:0/4239673417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4c1975a0 msgr2=0x7f2f4c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4c1975a0 0x7f2f4c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 -- 172.21.15.113:0/4239673417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c1980c0 con 0x7f2f4c106160 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4bfff640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 0x7f2f4c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f4b7fe640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 0x7f2f4c19d930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c000c00 tx=0x7f2f3c02cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c041070 con 0x7f2f4c106160 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2f3c0044d0 con 0x7f2f4c106160 2024-04-22T07:14:55.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c198350 con 0x7f2f4c106160 2024-04-22T07:14:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c003d50 con 0x7f2f4c106160 2024-04-22T07:14:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.541+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c1a45a0 con 0x7f2f4c106160 2024-04-22T07:14:55.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.542+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c068d70 con 0x7f2f4c106160 2024-04-22T07:14:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.543+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f2f3c044070 con 0x7f2f4c106160 2024-04-22T07:14:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.544+0000 7f2f497fa640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2f340a3e40 0x7f2f340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.544+0000 7f2f4bfff640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2f340a3e40 0x7f2f340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.544+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2f3c0f2500 con 0x7f2f4c106160 2024-04-22T07:14:55.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.544+0000 7f2f4bfff640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2f340a3e40 0x7f2f340a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009930 tx=0x7f2f40008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:55.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.548+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f3c0beb20 con 0x7f2f4c106160 2024-04-22T07:14:55.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.725+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f4c101a40 con 0x7f2f4c106160 2024-04-22T07:14:55.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.725+0000 7f2f497fa640 1 -- 172.21.15.113:0/4239673417 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f2f3c0be420 con 0x7f2f4c106160 2024-04-22T07:14:55.724 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2f340a3e40 msgr2=0x7f2f340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2f340a3e40 0x7f2f340a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009930 tx=0x7f2f40008040 comp rx=0 tx=0).stop 2024-04-22T07:14:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 msgr2=0x7f2f4c19d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 0x7f2f4c19d930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c000c00 tx=0x7f2f3c02cdf0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 shutdown_connections 2024-04-22T07:14:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2f340a3e40 0x7f2f340a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4c1975a0 0x7f2f4c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f4c106160 0x7f2f4c19d930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 --2- 172.21.15.113:0/4239673417 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c101090 0x7f2f4c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 >> 172.21.15.113:0/4239673417 conn(0x7f2f4c077f50 msgr2=0x7f2f4c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 shutdown_connections 2024-04-22T07:14:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.727+0000 7f2f52877640 1 -- 172.21.15.113:0/4239673417 wait complete. 2024-04-22T07:14:55.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1403564082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4102b40 msgr2=0x7f5eb4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1403564082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4102b40 0x7f5eb4106b80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009900 tx=0x7f5ea402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1403564082 shutdown_connections 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1403564082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eb4107260 0x7f5eb410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1403564082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4102b40 0x7f5eb4106b80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1403564082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb4102190 0x7f5eb4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1403564082 >> 172.21.15.113:0/1403564082 conn(0x7f5eb40793e0 msgr2=0x7f5eb40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1403564082 shutdown_connections 2024-04-22T07:14:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.836+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1403564082 wait complete. 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 Processor -- start 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 -- start start 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eb4102190 0x7f5eb419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb4107260 0x7f5eb419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 0x7f5eb4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41122e0 con 0x7f5eb4198620 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4112160 con 0x7f5eb4102190 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5ebc3e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4112460 con 0x7f5eb4107260 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 0x7f5eb4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 0x7f5eb4198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56220/0 (socket says 172.21.15.113:56220) 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 -- 172.21.15.113:0/1862628768 learned_addr learned my addr 172.21.15.113:0/1862628768 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba182640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eb4102190 0x7f5eb419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eb9981640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb4107260 0x7f5eb419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 -- 172.21.15.113:0/1862628768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb4107260 msgr2=0x7f5eb419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb4107260 0x7f5eb419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 -- 172.21.15.113:0/1862628768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eb4102190 msgr2=0x7f5eb419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eb4102190 0x7f5eb419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.837+0000 7f5eba983640 1 -- 172.21.15.113:0/1862628768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb4199140 con 0x7f5eb4198620 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.838+0000 7f5eba983640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 0x7f5eb4198ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00eaa0 tx=0x7f5eac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.838+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00cdb0 con 0x7f5eb4198620 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.838+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5eac005020 con 0x7f5eb4198620 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.838+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb4199430 con 0x7f5eb4198620 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.838+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0116d0 con 0x7f5eb4198620 2024-04-22T07:14:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.838+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb41a5630 con 0x7f5eb4198620 2024-04-22T07:14:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.839+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb4069230 con 0x7f5eb4198620 2024-04-22T07:14:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.840+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f5eac0051d0 con 0x7f5eb4198620 2024-04-22T07:14:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.840+0000 7f5eab7fe640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ea00a3e40 0x7f5ea00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.840+0000 7f5eba182640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ea00a3e40 0x7f5ea00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.840+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5eac0d0cb0 con 0x7f5eb4198620 2024-04-22T07:14:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.841+0000 7f5eba182640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ea00a3e40 0x7f5ea00a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005fd0 tx=0x7f5eb000b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:55.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:55.845+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5eac016030 con 0x7f5eb4198620 2024-04-22T07:14:56.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.059+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eb410a490 con 0x7f5eb4198620 2024-04-22T07:14:56.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.060+0000 7f5eab7fe640 1 -- 172.21.15.113:0/1862628768 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f5eac09d130 con 0x7f5eb4198620 2024-04-22T07:14:56.059 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:56.059 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:14:56.059 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ea00a3e40 msgr2=0x7f5ea00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ea00a3e40 0x7f5ea00a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0005fd0 tx=0x7f5eb000b390 comp rx=0 tx=0).stop 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 msgr2=0x7f5eb4198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 0x7f5eb4198ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00eaa0 tx=0x7f5eac00ef60 comp rx=0 tx=0).stop 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 shutdown_connections 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ea00a3e40 0x7f5ea00a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb4198620 0x7f5eb4198ad0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb4107260 0x7f5eb419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 --2- 172.21.15.113:0/1862628768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eb4102190 0x7f5eb419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 >> 172.21.15.113:0/1862628768 conn(0x7f5eb40793e0 msgr2=0x7f5eb40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 shutdown_connections 2024-04-22T07:14:56.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.062+0000 7f5ebc3e6640 1 -- 172.21.15.113:0/1862628768 wait complete. 2024-04-22T07:14:56.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:56.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 -- 172.21.15.113:0/679249876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc101b60 msgr2=0x7f41dc10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 --2- 172.21.15.113:0/679249876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc101b60 0x7f41dc10dc90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009930 tx=0x7f41d802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 -- 172.21.15.113:0/679249876 shutdown_connections 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 --2- 172.21.15.113:0/679249876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc101b60 0x7f41dc10dc90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 --2- 172.21.15.113:0/679249876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 0x7f41dc101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 --2- 172.21.15.113:0/679249876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41dc105e30 0x7f41dc106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 -- 172.21.15.113:0/679249876 >> 172.21.15.113:0/679249876 conn(0x7f41dc077e30 msgr2=0x7f41dc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 -- 172.21.15.113:0/679249876 shutdown_connections 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 -- 172.21.15.113:0/679249876 wait complete. 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 Processor -- start 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.171+0000 7f41e4070640 1 -- start start 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 0x7f41dc197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41dc101b60 0x7f41dc197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 0x7f41dc19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc111240 con 0x7f41dc105e30 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1110c0 con 0x7f41dc1011a0 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1113c0 con 0x7f41dc101b60 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 0x7f41dc19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 0x7f41dc19bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56244/0 (socket says 172.21.15.113:56244) 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 -- 172.21.15.113:0/485950019 learned_addr learned my addr 172.21.15.113:0/485950019 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e160b640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41dc101b60 0x7f41dc197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e1e0c640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 0x7f41dc197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 -- 172.21.15.113:0/485950019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41dc101b60 msgr2=0x7f41dc197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41dc101b60 0x7f41dc197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 -- 172.21.15.113:0/485950019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 msgr2=0x7f41dc197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 0x7f41dc197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 -- 172.21.15.113:0/485950019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc19c670 con 0x7f41dc105e30 2024-04-22T07:14:56.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e1e0c640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 0x7f41dc197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:56.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e260d640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 0x7f41dc19bf40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f41d8005940 tx=0x7f41d802cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8043020 con 0x7f41dc105e30 2024-04-22T07:14:56.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f41d8004310 con 0x7f41dc105e30 2024-04-22T07:14:56.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc19c900 con 0x7f41dc105e30 2024-04-22T07:14:56.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8003950 con 0x7f41dc105e30 2024-04-22T07:14:56.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.172+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a41f0 con 0x7f41dc105e30 2024-04-22T07:14:56.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.173+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc068de0 con 0x7f41dc105e30 2024-04-22T07:14:56.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.175+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f41d8046020 con 0x7f41dc105e30 2024-04-22T07:14:56.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.175+0000 7f41d2ffd640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f41c80a3e40 0x7f41c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.175+0000 7f41e1e0c640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f41c80a3e40 0x7f41c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.175+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f41d80f2b70 con 0x7f41dc105e30 2024-04-22T07:14:56.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.176+0000 7f41e1e0c640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f41c80a3e40 0x7f41c80a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005fd0 tx=0x7f41d4009520 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.179+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41d804d070 con 0x7f41dc105e30 2024-04-22T07:14:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:56 smithi133 ceph-mon[31198]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/864883207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4057051084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:56 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4239673417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:56 smithi113 ceph-mon[27933]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:56.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/864883207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4057051084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:56 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4239673417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.360+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41dc109330 con 0x7f41dc105e30 2024-04-22T07:14:56.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.361+0000 7f41d2ffd640 1 -- 172.21.15.113:0/485950019 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7f41d80bf100 con 0x7f41dc105e30 2024-04-22T07:14:56.360 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:56.360 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f41c80a3e40 msgr2=0x7f41c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f41c80a3e40 0x7f41c80a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005fd0 tx=0x7f41d4009520 comp rx=0 tx=0).stop 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 msgr2=0x7f41dc19bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 0x7f41dc19bf40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f41d8005940 tx=0x7f41d802cb90 comp rx=0 tx=0).stop 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 shutdown_connections 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f41c80a3e40 0x7f41c80a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41dc105e30 0x7f41dc19bf40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41dc101b60 0x7f41dc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 --2- 172.21.15.113:0/485950019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc1011a0 0x7f41dc197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 >> 172.21.15.113:0/485950019 conn(0x7f41dc077e30 msgr2=0x7f41dc0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 shutdown_connections 2024-04-22T07:14:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.363+0000 7f41e4070640 1 -- 172.21.15.113:0/485950019 wait complete. 2024-04-22T07:14:56.373 DEBUG:tasks.fwd_scrub:Yielding 2024-04-22T07:14:56.373 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T07:14:56.384 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T07:14:56.384 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:14:56.384 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-22T07:14:56.384 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-22T07:14:56.385 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:14:56.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:14:56.385 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:14:56.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 -- 172.21.15.113:0/3734042342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4870102190 msgr2=0x7f4870102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 --2- 172.21.15.113:0/3734042342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4870102190 0x7f4870102570 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:56.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 -- 172.21.15.113:0/1282407748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c101aa0 msgr2=0x7f283c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 --2- 172.21.15.113:0/1282407748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c101aa0 0x7f283c105a90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f282c009900 tx=0x7f282c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:56.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 -- 172.21.15.113:0/3734042342 shutdown_connections 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 --2- 172.21.15.113:0/3734042342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 0x7f487010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 --2- 172.21.15.113:0/3734042342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4870102b40 0x7f4870106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 --2- 172.21.15.113:0/3734042342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4870102190 0x7f4870102570 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 -- 172.21.15.113:0/3734042342 >> 172.21.15.113:0/3734042342 conn(0x7f48700fd9f0 msgr2=0x7f48700ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 -- 172.21.15.113:0/1282407748 shutdown_connections 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 --2- 172.21.15.113:0/1282407748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c106170 0x7f283c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 --2- 172.21.15.113:0/1282407748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c101aa0 0x7f283c105a90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 -- 172.21.15.113:0/3734042342 shutdown_connections 2024-04-22T07:14:56.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 --2- 172.21.15.113:0/1282407748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f283c1010f0 0x7f283c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 -- 172.21.15.113:0/1282407748 >> 172.21.15.113:0/1282407748 conn(0x7f283c077fd0 msgr2=0x7f283c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 -- 172.21.15.113:0/3734042342 wait complete. 2024-04-22T07:14:56.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 -- 172.21.15.113:0/1282407748 shutdown_connections 2024-04-22T07:14:56.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 -- 172.21.15.113:0/1282407748 wait complete. 2024-04-22T07:14:56.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 Processor -- start 2024-04-22T07:14:56.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 Processor -- start 2024-04-22T07:14:56.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f487857e640 1 -- start start 2024-04-22T07:14:56.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.489+0000 7f2844611640 1 -- start start 2024-04-22T07:14:56.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487857e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4870102190 0x7f487019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487857e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 0x7f487019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487857e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 0x7f4870198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487857e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48701122e0 con 0x7f48701986e0 2024-04-22T07:14:56.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487857e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4870112160 con 0x7f4870107260 2024-04-22T07:14:56.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487857e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4870112460 con 0x7f4870102190 2024-04-22T07:14:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2844611640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c1010f0 0x7f283c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2844611640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f283c106170 0x7f283c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2844611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 0x7f283c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2844611640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f283c112300 con 0x7f283c1985e0 2024-04-22T07:14:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2844611640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f283c112180 con 0x7f283c1010f0 2024-04-22T07:14:56.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2844611640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f283c112480 con 0x7f283c106170 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 0x7f4870198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 0x7f4870198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56276/0 (socket says 172.21.15.113:56276) 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f487631a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4870102190 0x7f487019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 -- 172.21.15.113:0/3923713111 learned_addr learned my addr 172.21.15.113:0/3923713111 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 0x7f283c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 0x7f283c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56286/0 (socket says 172.21.15.113:56286) 2024-04-22T07:14:56.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 -- 172.21.15.113:0/74029061 learned_addr learned my addr 172.21.15.113:0/74029061 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2841bac640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f283c106170 0x7f283c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f28423ad640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c1010f0 0x7f283c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4875b19640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 0x7f487019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 -- 172.21.15.113:0/3923713111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4870102190 msgr2=0x7f487019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4870102190 0x7f487019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 -- 172.21.15.113:0/3923713111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 msgr2=0x7f487019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 0x7f487019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 -- 172.21.15.113:0/3923713111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48701992e0 con 0x7f48701986e0 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4875b19640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 0x7f487019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 -- 172.21.15.113:0/74029061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f283c106170 msgr2=0x7f283c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f283c106170 0x7f283c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 -- 172.21.15.113:0/74029061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c1010f0 msgr2=0x7f283c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c1010f0 0x7f283c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 -- 172.21.15.113:0/74029061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c199100 con 0x7f283c1985e0 2024-04-22T07:14:56.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f4876b1b640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 0x7f4870198b60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4868009fd0 tx=0x7f486800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f2842bae640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 0x7f283c198a90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f283400cce0 tx=0x7f283400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f48677fe640 1 -- 172.21.15.113:0/3923713111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868017070 con 0x7f48701986e0 2024-04-22T07:14:56.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f48677fe640 1 -- 172.21.15.113:0/3923713111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4868003e40 con 0x7f48701986e0 2024-04-22T07:14:56.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48701a5370 con 0x7f48701986e0 2024-04-22T07:14:56.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.490+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400f450 con 0x7f283c1985e0 2024-04-22T07:14:56.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f48677fe640 1 -- 172.21.15.113:0/3923713111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868004820 con 0x7f48701986e0 2024-04-22T07:14:56.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c1993f0 con 0x7f283c1985e0 2024-04-22T07:14:56.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f283400fac0 con 0x7f283c1985e0 2024-04-22T07:14:56.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48701a5720 con 0x7f48701986e0 2024-04-22T07:14:56.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834018be0 con 0x7f283c1985e0 2024-04-22T07:14:56.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.491+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a55f0 con 0x7f283c1985e0 2024-04-22T07:14:56.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.492+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c102570 con 0x7f283c1985e0 2024-04-22T07:14:56.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.493+0000 7f48677fe640 1 -- 172.21.15.113:0/3923713111 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f4868003970 con 0x7f48701986e0 2024-04-22T07:14:56.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.493+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f283400fc70 con 0x7f283c1985e0 2024-04-22T07:14:56.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.493+0000 7f487b16e740 1 Processor -- start 2024-04-22T07:14:56.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 -- start start 2024-04-22T07:14:56.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f28337fe640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f28280a3e50 0x7f28280a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350d740 0x5633e350db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 0x5633e3516c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f48677fe640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f485c0a3e40 0x7f485c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e35176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f28340d0f90 con 0x7f283c1985e0 2024-04-22T07:14:56.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5633e3471f00 con 0x5633e350e0f0 2024-04-22T07:14:56.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5633e340f590 con 0x5633e350d740 2024-04-22T07:14:56.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487b16e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5633e346dec0 con 0x5633e3517250 2024-04-22T07:14:56.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f28423ad640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f28280a3e50 0x7f28280a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f4875b19640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f485c0a3e40 0x7f485c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487631a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 0x5633e3516c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487631a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 0x5633e3516c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56298/0 (socket says 172.21.15.113:56298) 2024-04-22T07:14:56.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f487631a640 1 -- 172.21.15.113:0/697017772 learned_addr learned my addr 172.21.15.113:0/697017772 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f48677fe640 1 -- 172.21.15.113:0/3923713111 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f48680d07e0 con 0x7f48701986e0 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f4876b1b640 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e35176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.494+0000 7f4875b19640 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350d740 0x5633e350db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f487631a640 1 -- 172.21.15.113:0/697017772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 msgr2=0x5633e35176d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f487631a640 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e35176d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f487631a640 1 -- 172.21.15.113:0/697017772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350d740 msgr2=0x5633e350db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f487631a640 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350d740 0x5633e350db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f487631a640 1 -- 172.21.15.113:0/697017772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e324c7e0 con 0x5633e350e0f0 2024-04-22T07:14:56.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f28423ad640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f28280a3e50 0x7f28280a6310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28380058c0 tx=0x7f283800a250 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f4876b1b640 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e35176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f4875b19640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f485c0a3e40 0x7f485c0a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4870199ee0 tx=0x7f486000a770 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.495+0000 7f487631a640 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 0x5633e3516c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad01c277e922023 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.498+0000 7f4864bfe640 1 -- 172.21.15.113:0/697017772 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c02cb10 con 0x5633e350e0f0 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f4864bfe640 1 -- 172.21.15.113:0/697017772 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f486c02ccb0 con 0x5633e350e0f0 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f4864bfe640 1 -- 172.21.15.113:0/697017772 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c004920 con 0x5633e350e0f0 2024-04-22T07:14:56.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2834014030 con 0x7f283c1985e0 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 -- 172.21.15.113:0/697017772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 msgr2=0x5633e3516c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 0x5633e3516c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02c680 comp rx=0 tx=0).stop 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 -- 172.21.15.113:0/697017772 shutdown_connections 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e35176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350e0f0 0x5633e3516c30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 --2- 172.21.15.113:0/697017772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350d740 0x5633e350db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.499+0000 7f487b16e740 1 -- 172.21.15.113:0/697017772 >> 172.21.15.113:0/697017772 conn(0x5633e3507fb0 msgr2=0x5633e3508390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 -- 172.21.15.113:0/697017772 shutdown_connections 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 -- 172.21.15.113:0/697017772 wait complete. 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 -- start start 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 Processor -- start 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e351ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e351b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e351b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5633e351e170 con 0x5633e350d740 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5633e342b580 con 0x5633e350e0f0 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5633e351e2f0 con 0x5633e3517250 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487b16e740 10 client.0 _ll_register_callbacks cb 0x5633e3431650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487631a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e351ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487631a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e351ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56314/0 (socket says 172.21.15.113:56314) 2024-04-22T07:14:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.500+0000 7f487631a640 1 -- 172.21.15.113:0/2931003841 learned_addr learned my addr 172.21.15.113:0/2931003841 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4875b19640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e351b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487631a640 1 -- 172.21.15.113:0/2931003841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 msgr2=0x5633e351b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4876b1b640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e351b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487631a640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e351b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487631a640 1 -- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 msgr2=0x5633e351b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487631a640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e351b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487631a640 1 -- 172.21.15.113:0/2931003841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e32acf70 con 0x5633e350d740 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4875b19640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e351b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487631a640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e351ac80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f486c0096c0 tx=0x7f486c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b007cd83076b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4859ffb640 1 -- 172.21.15.113:0/2931003841 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c04b070 con 0x5633e350d740 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633e347f920 con 0x5633e350d740 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4859ffb640 1 -- 172.21.15.113:0/2931003841 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f486c034c40 con 0x5633e350d740 2024-04-22T07:14:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4859ffb640 1 -- 172.21.15.113:0/2931003841 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f486c034e40 con 0x5633e350d740 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487b16e740 10 client.15483 fetch_fsmap learned FSMap version 22 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5633e3144170 con 0x5633e350d740 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4859ffb640 1 -- 172.21.15.113:0/2931003841 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x5633e347f920 con 0x5633e350d740 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f4859ffb640 1 -- 172.21.15.113:0/2931003841 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f486c045780 con 0x5633e350d740 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487b16e740 10 client.15483 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487b16e740 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.501+0000 7f487b16e740 20 client.15483 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.502+0000 7f487b16e740 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e352b530 0x5633e352d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.502+0000 7f487b16e740 4 client.15483 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.502+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e3479790 con 0x5633e352b530 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.502+0000 7f4875b19640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e352b530 0x5633e352d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.502+0000 7f4859ffb640 10 client.15483 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.502+0000 7f4875b19640 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e352b530 0x5633e352d920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4261d7b441aac0b8 server_cookie=e2c4b01832fae9ae in_seq=0 out_seq=0 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f4859ffb640 1 -- 172.21.15.113:0/2931003841 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633e3479790 con 0x5633e352b530 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f4859ffb640 10 client.15483 handle_command_reply: tid=0 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 client.15483 shutdown 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 msgr2=0x5633e351ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e351ac80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f486c0096c0 tx=0x7f486c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 shutdown_connections 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e351b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e351b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e352b530 0x5633e352d920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 --2- 172.21.15.113:0/2931003841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e351ac80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f4859ffb640 0 client.15483 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 >> 172.21.15.113:0/2931003841 conn(0x5633e3507fb0 msgr2=0x5633e3509d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 shutdown_connections 2024-04-22T07:14:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 1 -- 172.21.15.113:0/2931003841 wait complete. 2024-04-22T07:14:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.503+0000 7f487b16e740 20 client.15483 trim_cache size 0 max 16384 2024-04-22T07:14:56.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 Processor -- start 2024-04-22T07:14:56.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 -- start start 2024-04-22T07:14:56.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e350d740 0x5633e351d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e3518640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 0x5633e3526c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48701ae2a0 con 0x5633e3517250 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f48701abb70 con 0x5633e350e0f0 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.520+0000 7f487b16e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a99c0 con 0x5633e350d740 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 0x5633e3526c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 0x5633e3526c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56316/0 (socket says 172.21.15.113:56316) 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 -- 172.21.15.113:0/1322259752 learned_addr learned my addr 172.21.15.113:0/1322259752 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4875b19640 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e3518640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 -- 172.21.15.113:0/1322259752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e350d740 msgr2=0x5633e351d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e350d740 0x5633e351d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 -- 172.21.15.113:0/1322259752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 msgr2=0x5633e3518640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e3518640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 -- 172.21.15.113:0/1322259752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e350de90 con 0x5633e3517250 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4875b19640 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e3518640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f4876b1b640 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 0x5633e3526c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f48680ce560 tx=0x7f486809c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213b425e6e17ee8e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f485bfff640 1 -- 172.21.15.113:0/1322259752 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868020040 con 0x5633e3517250 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f485bfff640 1 -- 172.21.15.113:0/1322259752 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f486809c560 con 0x5633e3517250 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.521+0000 7f485bfff640 1 -- 172.21.15.113:0/1322259752 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868098bd0 con 0x5633e3517250 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- 172.21.15.113:0/1322259752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 msgr2=0x5633e3526c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 0x5633e3526c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f48680ce560 tx=0x7f486809c450 comp rx=0 tx=0).stop 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- 172.21.15.113:0/1322259752 shutdown_connections 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e3517250 0x5633e3526c90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e3518640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- 172.21.15.113:0/1322259752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e350d740 0x5633e351d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- 172.21.15.113:0/1322259752 >> 172.21.15.113:0/1322259752 conn(0x5633e3507fb0 msgr2=0x5633e3526580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- 172.21.15.113:0/1322259752 shutdown_connections 2024-04-22T07:14:56.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- 172.21.15.113:0/1322259752 wait complete. 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- start start 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 Processor -- start 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e352ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e352f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e358a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48701aa030 con 0x5633e350d740 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f48701ab4a0 con 0x5633e350e0f0 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f48701aa700 con 0x5633e3517250 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.522+0000 7f487b16e740 10 client.0 _ll_register_callbacks cb 0x7f48680043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e352ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e352ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56324/0 (socket says 172.21.15.113:56324) 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 -- 172.21.15.113:0/759995292 learned_addr learned my addr 172.21.15.113:0/759995292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4875b19640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e352f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4876b1b640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e358a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 -- 172.21.15.113:0/759995292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 msgr2=0x5633e358a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e358a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 -- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 msgr2=0x5633e352f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e352f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 -- 172.21.15.113:0/759995292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e358acd0 con 0x5633e350d740 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487631a640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e352ebe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f486c02f020 tx=0x7f486c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9274aab689171e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4859ffb640 1 -- 172.21.15.113:0/759995292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c05d070 con 0x5633e350d740 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4859ffb640 1 -- 172.21.15.113:0/759995292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f486c004440 con 0x5633e350d740 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486809c560 con 0x5633e350d740 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4859ffb640 1 -- 172.21.15.113:0/759995292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f486c066050 con 0x5633e350d740 2024-04-22T07:14:56.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487b16e740 10 client.15489 fetch_fsmap learned FSMap version 22 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5633e3592090 con 0x5633e350d740 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.523+0000 7f4859ffb640 1 -- 172.21.15.113:0/759995292 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f486809c560 con 0x5633e350d740 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f4859ffb640 1 -- 172.21.15.113:0/759995292 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f486c060020 con 0x5633e350d740 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f487b16e740 10 client.15489 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f487b16e740 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f487b16e740 20 client.15489 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f487b16e740 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e3592780 0x5633e3594ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f487b16e740 4 client.15489 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5633e33f5c50 con 0x5633e3592780 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f4875b19640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e3592780 0x5633e3594ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f4859ffb640 10 client.15489 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.524+0000 7f4875b19640 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e3592780 0x5633e3594ba0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ee7764c00afd85 server_cookie=63c97724f036500 in_seq=0 out_seq=0 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f4859ffb640 1 -- 172.21.15.113:0/759995292 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5633e33f5c50 con 0x5633e3592780 2024-04-22T07:14:56.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f4859ffb640 10 client.15489 handle_command_reply: tid=0 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 client.15489 shutdown 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 msgr2=0x5633e352ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e352ebe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f486c02f020 tx=0x7f486c009520 comp rx=0 tx=0).stop 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 shutdown_connections 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633e3517250 0x5633e358a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633e350e0f0 0x5633e352f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5633e3592780 0x5633e3594ba0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 --2- 172.21.15.113:0/759995292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5633e350d740 0x5633e352ebe0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 >> 172.21.15.113:0/759995292 conn(0x5633e3507fb0 msgr2=0x5633e351b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f4859ffb640 0 client.15489 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 shutdown_connections 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 1 -- 172.21.15.113:0/759995292 wait complete. 2024-04-22T07:14:56.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.526+0000 7f487b16e740 20 client.15489 trim_cache size 0 max 16384 2024-04-22T07:14:56.533 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:56.533 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:14:56.533 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "12b1c8a4-eb3b-45c0-8600-57f9920974f4", 2024-04-22T07:14:56.533 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:14:56.533 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:56.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f485c0a3e40 msgr2=0x7f485c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f485c0a3e40 0x7f485c0a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4870199ee0 tx=0x7f486000a770 comp rx=0 tx=0).stop 2024-04-22T07:14:56.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 msgr2=0x7f4870198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 0x7f4870198b60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4868009fd0 tx=0x7f486800b260 comp rx=0 tx=0).stop 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 shutdown_connections 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f485c0a3e40 0x7f485c0a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48701986e0 0x7f4870198b60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4870107260 0x7f487019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 --2- 172.21.15.113:0/3923713111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4870102190 0x7f487019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 >> 172.21.15.113:0/3923713111 conn(0x7f48700fd9f0 msgr2=0x7f48700fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 shutdown_connections 2024-04-22T07:14:56.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.527+0000 7f487857e640 1 -- 172.21.15.113:0/3923713111 wait complete. 2024-04-22T07:14:56.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:14:56.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:14:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:56 smithi154 ceph-mon[31862]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:14:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/864883207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4057051084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:56 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4239673417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:14:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 -- 172.21.15.113:0/3955748547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84101a30 msgr2=0x7f8f84105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 --2- 172.21.15.113:0/3955748547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84101a30 0x7f8f84105a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009900 tx=0x7f8f7402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 -- 172.21.15.113:0/3955748547 shutdown_connections 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 --2- 172.21.15.113:0/3955748547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 0x7f8f8410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 --2- 172.21.15.113:0/3955748547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f84101a30 0x7f8f84105a70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 --2- 172.21.15.113:0/3955748547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 0x7f8f84101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 -- 172.21.15.113:0/3955748547 >> 172.21.15.113:0/3955748547 conn(0x7f8f84078000 msgr2=0x7f8f84078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 -- 172.21.15.113:0/3955748547 shutdown_connections 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.642+0000 7f8f8923f640 1 -- 172.21.15.113:0/3955748547 wait complete. 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 Processor -- start 2024-04-22T07:14:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 -- start start 2024-04-22T07:14:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 0x7f8f841a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 0x7f8f841a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 0x7f8f8419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841111d0 con 0x7f8f8419a7d0 2024-04-22T07:14:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84111050 con 0x7f8f84101080 2024-04-22T07:14:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8923f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f84111350 con 0x7f8f84106150 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.643+0000 7f8f8359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 0x7f8f8419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 0x7f8f8419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56342/0 (socket says 172.21.15.113:56342) 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 -- 172.21.15.113:0/1469073652 learned_addr learned my addr 172.21.15.113:0/1469073652 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 0x7f8f841a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 0x7f8f841a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 -- 172.21.15.113:0/1469073652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 msgr2=0x7f8f841a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 0x7f8f841a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 -- 172.21.15.113:0/1469073652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 msgr2=0x7f8f841a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 0x7f8f841a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 -- 172.21.15.113:0/1469073652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8419b360 con 0x7f8f8419a7d0 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 0x7f8f841a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 0x7f8f841a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8359e640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 0x7f8f8419ac80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00eaa0 tx=0x7f8f7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f6bfff640 1 -- 172.21.15.113:0/1469073652 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00cdb0 con 0x7f8f8419a7d0 2024-04-22T07:14:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8419b650 con 0x7f8f8419a7d0 2024-04-22T07:14:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f6bfff640 1 -- 172.21.15.113:0/1469073652 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8f7c005020 con 0x7f8f8419a7d0 2024-04-22T07:14:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f6bfff640 1 -- 172.21.15.113:0/1469073652 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0116b0 con 0x7f8f8419a7d0 2024-04-22T07:14:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.644+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f841a7880 con 0x7f8f8419a7d0 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.646+0000 7f8f6bfff640 1 -- 172.21.15.113:0/1469073652 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f8f7c0051d0 con 0x7f8f8419a7d0 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.647+0000 7f8f6bfff640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8f6c0a3e40 0x7f8f6c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.647+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8f6c0a3e40 0x7f8f6c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.647+0000 7f8f6bfff640 1 -- 172.21.15.113:0/1469073652 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8f7c0d0cc0 con 0x7f8f8419a7d0 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.647+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8f6c0a3e40 0x7f8f6c0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009c20 tx=0x7f8f78005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 Processor -- start 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 -- start start 2024-04-22T07:14:56.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac34540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac36e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559acab901e0 con 0x559acac2bea0 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559acab2d390 con 0x559acac34a80 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559acab8c1a0 con 0x559acac2b580 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac34540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac34540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56356/0 (socket says 172.21.15.113:56356) 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 -- 172.21.15.113:0/1419221198 learned_addr learned my addr 172.21.15.113:0/1419221198 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8359e640 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac36e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 -- 172.21.15.113:0/1419221198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 msgr2=0x559acac2b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac2b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 -- 172.21.15.113:0/1419221198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 msgr2=0x559acac36e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac36e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 -- 172.21.15.113:0/1419221198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aca96c7d0 con 0x559acac2bea0 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.648+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac34540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006010 tx=0x7f8f74005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb32de37c470d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f697fa640 1 -- 172.21.15.113:0/1419221198 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7402ca50 con 0x559acac2bea0 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f697fa640 1 -- 172.21.15.113:0/1419221198 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8f7402cbf0 con 0x559acac2bea0 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f697fa640 1 -- 172.21.15.113:0/1419221198 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74005650 con 0x559acac2bea0 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1419221198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 msgr2=0x559acac34540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac34540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006010 tx=0x7f8f74005dd0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1419221198 shutdown_connections 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac36e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac34540 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1419221198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1419221198 >> 172.21.15.113:0/1419221198 conn(0x559acac25cb0 msgr2=0x559acac26090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1419221198 shutdown_connections 2024-04-22T07:14:56.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.649+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1419221198 wait complete. 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 -- start start 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 Processor -- start 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac31ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac2bea0 0x559acac32020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac34a80 0x559acac32560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559acac37810 con 0x559acac2b580 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559acab492f0 con 0x559acac2bea0 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559acac37990 con 0x559acac34a80 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8be2c740 10 client.0 _ll_register_callbacks cb 0x559acab56580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f82d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac31ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f82d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac31ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56366/0 (socket says 172.21.15.113:56366) 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f82d9d640 1 -- 172.21.15.113:0/3775151400 learned_addr learned my addr 172.21.15.113:0/3775151400 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.650+0000 7f8f8359e640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac34a80 0x559acac32560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8259c640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac2bea0 0x559acac32020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f82d9d640 1 -- 172.21.15.113:0/3775151400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac34a80 msgr2=0x559acac32560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac34a80 0x559acac32560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f82d9d640 1 -- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac2bea0 msgr2=0x559acac32020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac2bea0 0x559acac32020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f82d9d640 1 -- 172.21.15.113:0/3775151400 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aca8a23f0 con 0x559acac2b580 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8259c640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac2bea0 0x559acac32020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac31ae0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f780045c0 tx=0x7f8f78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfbf12b433db9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f66ff5640 1 -- 172.21.15.113:0/3775151400 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78024070 con 0x559acac2b580 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f66ff5640 1 -- 172.21.15.113:0/3775151400 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8f78012860 con 0x559acac2b580 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7402cbf0 con 0x559acac2b580 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f66ff5640 1 -- 172.21.15.113:0/3775151400 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8f78030050 con 0x559acac2b580 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8be2c740 10 client.15501 fetch_fsmap learned FSMap version 22 2024-04-22T07:14:56.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559aca8620f0 con 0x559acac2b580 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f66ff5640 1 -- 172.21.15.113:0/3775151400 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8f7402cbf0 con 0x559acac2b580 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f66ff5640 1 -- 172.21.15.113:0/3775151400 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8f78020c30 con 0x559acac2b580 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8be2c740 10 client.15501 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8be2c740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.651+0000 7f8f8be2c740 20 client.15501 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.652+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acac44a90 0x559acac46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.652+0000 7f8f8be2c740 4 client.15501 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.652+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559acab9dc00 con 0x559acac44a90 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.652+0000 7f8f8259c640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acac44a90 0x559acac46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.652+0000 7f8f66ff5640 10 client.15501 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.652+0000 7f8f8259c640 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acac44a90 0x559acac46f50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b99fe5527afba6 server_cookie=b6ad32ab6f0130a5 in_seq=0 out_seq=0 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f66ff5640 1 -- 172.21.15.113:0/3775151400 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559acab9dc00 con 0x559acac44a90 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f66ff5640 10 client.15501 handle_command_reply: tid=0 2024-04-22T07:14:56.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 client.15501 shutdown 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 msgr2=0x559acac31ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac31ae0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f780045c0 tx=0x7f8f78004730 comp rx=0 tx=0).stop 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 shutdown_connections 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac34a80 0x559acac32560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac2bea0 0x559acac32020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acac44a90 0x559acac46f50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/3775151400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac31ae0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f66ff5640 0 client.15501 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 >> 172.21.15.113:0/3775151400 conn(0x559acac25cb0 msgr2=0x559acac27a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 shutdown_connections 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 1 -- 172.21.15.113:0/3775151400 wait complete. 2024-04-22T07:14:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.653+0000 7f8f8be2c740 20 client.15501 trim_cache size 0 max 16384 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 Processor -- start 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 -- start start 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac30a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841b2750 con 0x559acac2bea0 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841b0020 con 0x559acac34a80 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.670+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841ade70 con 0x559acac2b580 2024-04-22T07:14:56.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac30a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac30a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56370/0 (socket says 172.21.15.113:56370) 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 -- 172.21.15.113:0/1905222117 learned_addr learned my addr 172.21.15.113:0/1905222117 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8359e640 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 -- 172.21.15.113:0/1905222117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 msgr2=0x559acac34210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac34210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 -- 172.21.15.113:0/1905222117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 msgr2=0x559acac406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 -- 172.21.15.113:0/1905222117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559acac30050 con 0x559acac2bea0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8359e640 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f8259c640 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac30a40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009900 tx=0x7f8f74005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3c47ed763eb730 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f68ff9640 1 -- 172.21.15.113:0/1905222117 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7402c2b0 con 0x559acac2bea0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f68ff9640 1 -- 172.21.15.113:0/1905222117 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8f7402ca30 con 0x559acac2bea0 2024-04-22T07:14:56.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.671+0000 7f8f68ff9640 1 -- 172.21.15.113:0/1905222117 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74003850 con 0x559acac2bea0 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1905222117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 msgr2=0x559acac30a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac30a40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009900 tx=0x7f8f74005a20 comp rx=0 tx=0).stop 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1905222117 shutdown_connections 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2bea0 0x559acac30a40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/1905222117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2b580 0x559acac34210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1905222117 >> 172.21.15.113:0/1905222117 conn(0x559acac25cb0 msgr2=0x559acac27970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1905222117 shutdown_connections 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- 172.21.15.113:0/1905222117 wait complete. 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- start start 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 Processor -- start 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac3d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2bea0 0x559acac39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841ae4e0 con 0x559acac2b580 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841af950 con 0x559acac34a80 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8f841aebb0 con 0x559acac2bea0 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f8be2c740 10 client.0 _ll_register_callbacks cb 0x7f8f78009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.672+0000 7f8f82d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac3d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac3d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56386/0 (socket says 172.21.15.113:56386) 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 -- 172.21.15.113:0/123861818 learned_addr learned my addr 172.21.15.113:0/123861818 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f8259c640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2bea0 0x559acac39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f8359e640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac3a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 -- 172.21.15.113:0/123861818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2bea0 msgr2=0x559acac39bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2bea0 0x559acac39bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 -- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 msgr2=0x559acac3a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac3a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 -- 172.21.15.113:0/123861818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559acacab0a0 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f82d9d640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac3d120 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005eb0 tx=0x7f8f78003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7faf6181a7c34d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f66ff5640 1 -- 172.21.15.113:0/123861818 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78024040 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f66ff5640 1 -- 172.21.15.113:0/123861818 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8f78012bb0 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7402ca30 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f66ff5640 1 -- 172.21.15.113:0/123861818 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8f78012db0 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f8be2c740 10 client.15507 fetch_fsmap learned FSMap version 22 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559acacac250 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.673+0000 7f8f66ff5640 1 -- 172.21.15.113:0/123861818 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8f7402ca30 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f66ff5640 1 -- 172.21.15.113:0/123861818 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8f78020660 con 0x559acac2b580 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8be2c740 10 client.15507 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8be2c740 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:14:56.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8be2c740 20 client.15507 populate_metadata read hostname 'smithi113' 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acacac9f0 0x559acacaee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8be2c740 4 client.15507 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559acab97a70 con 0x559acacac9f0 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8259c640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acacac9f0 0x559acacaee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f66ff5640 10 client.15507 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.674+0000 7f8f8259c640 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acacac9f0 0x559acacaee10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d44f1623d74943e server_cookie=8c5a691055e82bc8 in_seq=0 out_seq=0 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f66ff5640 1 -- 172.21.15.113:0/123861818 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559acab97a70 con 0x559acacac9f0 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f66ff5640 10 client.15507 handle_command_reply: tid=0 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 client.15507 shutdown 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 msgr2=0x559acac3d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac3d120 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005eb0 tx=0x7f8f78003d90 comp rx=0 tx=0).stop 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 shutdown_connections 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559acac34a80 0x559acac3a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559acac2bea0 0x559acac39bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559acacac9f0 0x559acacaee10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 --2- 172.21.15.113:0/123861818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559acac2b580 0x559acac3d120 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f66ff5640 0 client.15507 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 >> 172.21.15.113:0/123861818 conn(0x559acac25cb0 msgr2=0x559acac270e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 shutdown_connections 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 1 -- 172.21.15.113:0/123861818 wait complete. 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8be2c740 20 client.15507 trim_cache size 0 max 16384 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:14:56.681 INFO:teuthology.orchestra.run.smithi113.stdout: "12b1c8a4-eb3b-45c0-8600-57f9920974f4": { 2024-04-22T07:14:56.682 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:14:56.682 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "12b1c8a4-eb3b-45c0-8600-57f9920974f4", 2024-04-22T07:14:56.682 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:14:56.682 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:14:56.682 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:14:56.682 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8f6c0a3e40 msgr2=0x7f8f6c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8f6c0a3e40 0x7f8f6c0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009c20 tx=0x7f8f78005f90 comp rx=0 tx=0).stop 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 msgr2=0x7f8f8419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 0x7f8f8419ac80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00eaa0 tx=0x7f8f7c00ef60 comp rx=0 tx=0).stop 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 shutdown_connections 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8f6c0a3e40 0x7f8f6c0a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f8419a7d0 0x7f8f8419ac80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f84106150 0x7f8f841a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.675+0000 7f8f8923f640 1 --2- 172.21.15.113:0/1469073652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f84101080 0x7f8f841a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.676+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 >> 172.21.15.113:0/1469073652 conn(0x7f8f84078000 msgr2=0x7f8f840692a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.676+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 shutdown_connections 2024-04-22T07:14:56.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.676+0000 7f8f8923f640 1 -- 172.21.15.113:0/1469073652 wait complete. 2024-04-22T07:14:56.685 INFO:tasks.cephfs.filesystem:scrub status for tag:12b1c8a4-eb3b-45c0-8600-57f9920974f4 - {'path': '/', 'tag': '12b1c8a4-eb3b-45c0-8600-57f9920974f4', 'options': 'recursive,force'} 2024-04-22T07:14:56.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.720+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f283c199ac0 con 0x7f28280a3e50 2024-04-22T07:14:56.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.791+0000 7f28337fe640 1 -- 172.21.15.113:0/74029061 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f283c199ac0 con 0x7f28280a3e50 2024-04-22T07:14:56.790 INFO:teuthology.orchestra.run.smithi113.stdout:Schedule set for path / 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f28280a3e50 msgr2=0x7f28280a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f28280a3e50 0x7f28280a6310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28380058c0 tx=0x7f283800a250 comp rx=0 tx=0).stop 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 msgr2=0x7f283c198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 0x7f283c198a90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f283400cce0 tx=0x7f283400a620 comp rx=0 tx=0).stop 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 shutdown_connections 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f28280a3e50 0x7f28280a6310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f283c1985e0 0x7f283c198a90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f283c106170 0x7f283c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 --2- 172.21.15.113:0/74029061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c1010f0 0x7f283c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 >> 172.21.15.113:0/74029061 conn(0x7f283c077fd0 msgr2=0x7f283c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 shutdown_connections 2024-04-22T07:14:56.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.793+0000 7f2844611640 1 -- 172.21.15.113:0/74029061 wait complete. 2024-04-22T07:14:56.802 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-22T07:14:56.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 -- 172.21.15.113:0/4109924485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94106fb0 msgr2=0x7f2c94107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 --2- 172.21.15.113:0/4109924485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94106fb0 0x7f2c94107390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402bc80 comp rx=0 tx=0).stop 2024-04-22T07:14:56.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 -- 172.21.15.113:0/4109924485 shutdown_connections 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 --2- 172.21.15.113:0/4109924485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94101b60 0x7f2c9410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 --2- 172.21.15.113:0/4109924485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c941011a0 0x7f2c94101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 --2- 172.21.15.113:0/4109924485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94106fb0 0x7f2c94107390 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 -- 172.21.15.113:0/4109924485 >> 172.21.15.113:0/4109924485 conn(0x7f2c94076c50 msgr2=0x7f2c94077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 -- 172.21.15.113:0/4109924485 shutdown_connections 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.944+0000 7f2c9a631640 1 -- 172.21.15.113:0/4109924485 wait complete. 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 Processor -- start 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 -- start start 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c941011a0 0x7f2c941986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 0x7f2c94198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94106fb0 0x7f2c9419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94111220 con 0x7f2c94101b60 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941110a0 con 0x7f2c941011a0 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c9a631640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941113a0 con 0x7f2c94106fb0 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 0x7f2c94198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 0x7f2c94198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56402/0 (socket says 172.21.15.113:56402) 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c937fe640 1 -- 172.21.15.113:0/2902541431 learned_addr learned my addr 172.21.15.113:0/2902541431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c93fff640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c941011a0 0x7f2c941986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.945+0000 7f2c98bce640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94106fb0 0x7f2c9419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c937fe640 1 -- 172.21.15.113:0/2902541431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94106fb0 msgr2=0x7f2c9419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c937fe640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94106fb0 0x7f2c9419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c937fe640 1 -- 172.21.15.113:0/2902541431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c941011a0 msgr2=0x7f2c941986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c937fe640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c941011a0 0x7f2c941986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c937fe640 1 -- 172.21.15.113:0/2902541431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9419d6a0 con 0x7f2c94101b60 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c98bce640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94106fb0 0x7f2c9419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:56.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c937fe640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 0x7f2c94198bf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c920 tx=0x7f2c8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88007c70 con 0x7f2c94101b60 2024-04-22T07:14:56.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2c88007e10 con 0x7f2c94101b60 2024-04-22T07:14:56.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9419d990 con 0x7f2c94101b60 2024-04-22T07:14:56.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880188c0 con 0x7f2c94101b60 2024-04-22T07:14:56.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.946+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941a5220 con 0x7f2c94101b60 2024-04-22T07:14:56.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.947+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c94102550 con 0x7f2c94101b60 2024-04-22T07:14:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.948+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f2c88018a60 con 0x7f2c94101b60 2024-04-22T07:14:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.948+0000 7f2c917fa640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2c7c0a3ee0 0x7f2c7c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.949+0000 7f2c93fff640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2c7c0a3ee0 0x7f2c7c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.949+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2c880d0e80 con 0x7f2c94101b60 2024-04-22T07:14:56.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.949+0000 7f2c93fff640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2c7c0a3ee0 0x7f2c7c0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402efd0 tx=0x7f2c84005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:56.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:56.952+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c88014030 con 0x7f2c94101b60 2024-04-22T07:14:57.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.153+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f2c94104c50 con 0x7f2c7c0a3ee0 2024-04-22T07:14:57.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.214+0000 7f2c917fa640 1 -- 172.21.15.113:0/2902541431 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f2c94104c50 con 0x7f2c7c0a3ee0 2024-04-22T07:14:57.213 INFO:teuthology.orchestra.run.smithi113.stdout:Retention added to path / 2024-04-22T07:14:57.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.216+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2c7c0a3ee0 msgr2=0x7f2c7c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.216+0000 7f2c9a631640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2c7c0a3ee0 0x7f2c7c0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402efd0 tx=0x7f2c84005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.216+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 msgr2=0x7f2c94198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.216+0000 7f2c9a631640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 0x7f2c94198bf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c920 tx=0x7f2c8800cde0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 shutdown_connections 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2c7c0a3ee0 0x7f2c7c0a63a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c94106fb0 0x7f2c9419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c94101b60 0x7f2c94198bf0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 --2- 172.21.15.113:0/2902541431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c941011a0 0x7f2c941986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 >> 172.21.15.113:0/2902541431 conn(0x7f2c94076c50 msgr2=0x7f2c94100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 shutdown_connections 2024-04-22T07:14:57.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.217+0000 7f2c9a631640 1 -- 172.21.15.113:0/2902541431 wait complete. 2024-04-22T07:14:57.226 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-22T07:14:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:57 smithi154 ceph-mon[31862]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T07:14:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:57 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1862628768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:57 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/485950019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:57 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:14:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:57 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:14:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:57 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:14:57.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 -- 172.21.15.113:0/78066455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 msgr2=0x7fa360101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 --2- 172.21.15.113:0/78066455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa360101620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa35002bc80 comp rx=0 tx=0).stop 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 -- 172.21.15.113:0/78066455 shutdown_connections 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 --2- 172.21.15.113:0/78066455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa360101b60 0x7fa36010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 --2- 172.21.15.113:0/78066455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa360101620 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 --2- 172.21.15.113:0/78066455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa360106fb0 0x7fa360107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 -- 172.21.15.113:0/78066455 >> 172.21.15.113:0/78066455 conn(0x7fa360076c50 msgr2=0x7fa360077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 -- 172.21.15.113:0/78066455 shutdown_connections 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.325+0000 7fa367da4640 1 -- 172.21.15.113:0/78066455 wait complete. 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 Processor -- start 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 -- start start 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa36019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa360101b60 0x7fa36019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa360106fb0 0x7fa360198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa360111220 con 0x7fa3601011a0 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601110a0 con 0x7fa360106fb0 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa367da4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601113a0 con 0x7fa360101b60 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa36019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa36019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56412/0 (socket says 172.21.15.113:56412) 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 -- 172.21.15.113:0/2801230409 learned_addr learned my addr 172.21.15.113:0/2801230409 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa36533f640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa360101b60 0x7fa36019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa366341640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa360106fb0 0x7fa360198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 -- 172.21.15.113:0/2801230409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa360101b60 msgr2=0x7fa36019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa360101b60 0x7fa36019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 -- 172.21.15.113:0/2801230409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa360106fb0 msgr2=0x7fa360198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa360106fb0 0x7fa360198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 -- 172.21.15.113:0/2801230409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa360198d10 con 0x7fa3601011a0 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa365b40640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa36019e3e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00ba50 tx=0x7fa35c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.326+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c019070 con 0x7fa3601011a0 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.327+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa35c00cbe0 con 0x7fa3601011a0 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.327+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa360199000 con 0x7fa3601011a0 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.327+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0076e0 con 0x7fa3601011a0 2024-04-22T07:14:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.327+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3601a5320 con 0x7fa3601011a0 2024-04-22T07:14:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.327+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa360102550 con 0x7fa3601011a0 2024-04-22T07:14:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.329+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fa35c01c070 con 0x7fa3601011a0 2024-04-22T07:14:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.329+0000 7fa356ffd640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa34c0a3ee0 0x7fa34c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.329+0000 7fa36533f640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa34c0a3ee0 0x7fa34c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.329+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa35c0cffc0 con 0x7fa3601011a0 2024-04-22T07:14:57.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.330+0000 7fa36533f640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa34c0a3ee0 0x7fa34c0a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa35002efd0 tx=0x7fa350005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:57.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.334+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa35c09c550 con 0x7fa3601011a0 2024-04-22T07:14:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:57 smithi113 ceph-mon[27933]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T07:14:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:57 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1862628768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:57 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/485950019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:57 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:14:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:57 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:14:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:57 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:14:57.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.531+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa360104c50 con 0x7fa34c0a3ee0 2024-04-22T07:14:57.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.539+0000 7fa356ffd640 1 -- 172.21.15.113:0/2801230409 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa360104c50 con 0x7fa34c0a3ee0 2024-04-22T07:14:57.537 INFO:teuthology.orchestra.run.smithi113.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-22T00:00:00", "created": "2024-04-22T07:14:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-22T07:14:57.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.540+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa34c0a3ee0 msgr2=0x7fa34c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.540+0000 7fa367da4640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa34c0a3ee0 0x7fa34c0a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa35002efd0 tx=0x7fa350005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.540+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 msgr2=0x7fa36019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.540+0000 7fa367da4640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa36019e3e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00ba50 tx=0x7fa35c00bf10 comp rx=0 tx=0).stop 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 shutdown_connections 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa34c0a3ee0 0x7fa34c0a63a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa360106fb0 0x7fa360198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa360101b60 0x7fa36019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 --2- 172.21.15.113:0/2801230409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3601011a0 0x7fa36019e3e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 >> 172.21.15.113:0/2801230409 conn(0x7fa360076c50 msgr2=0x7fa3601004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 shutdown_connections 2024-04-22T07:14:57.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.541+0000 7fa367da4640 1 -- 172.21.15.113:0/2801230409 wait complete. 2024-04-22T07:14:57.550 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-22T07:14:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:57 smithi133 ceph-mon[31198]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T07:14:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:57 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1862628768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:14:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:57 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/485950019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:57 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:14:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:57 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:14:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:57 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- 172.21.15.113:0/1684265015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 msgr2=0x7f342c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- 172.21.15.113:0/1684265015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c06ac20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f341c009930 tx=0x7f341c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- 172.21.15.113:0/1684265015 shutdown_connections 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- 172.21.15.113:0/1684265015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c06b160 0x7f342c10edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- 172.21.15.113:0/1684265015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c06ac20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- 172.21.15.113:0/1684265015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 0x7f342c1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- 172.21.15.113:0/1684265015 >> 172.21.15.113:0/1684265015 conn(0x7f342c077ba0 msgr2=0x7f342c077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- 172.21.15.113:0/1684265015 shutdown_connections 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- 172.21.15.113:0/1684265015 wait complete. 2024-04-22T07:14:57.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 Processor -- start 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- start start 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c06b160 0x7f342c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 0x7f342c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.651+0000 7f3433a3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f342c112320 con 0x7f342c06a7c0 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3433a3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1121a0 con 0x7f342c06b160 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3433a3f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1124a0 con 0x7f342c107fc0 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c19e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56430/0 (socket says 172.21.15.113:56430) 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 -- 172.21.15.113:0/3789974173 learned_addr learned my addr 172.21.15.113:0/3789974173 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3430fda640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c06b160 0x7f342c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3431fdc640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 0x7f342c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 -- 172.21.15.113:0/3789974173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 msgr2=0x7f342c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 0x7f342c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 -- 172.21.15.113:0/3789974173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c06b160 msgr2=0x7f342c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c06b160 0x7f342c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 -- 172.21.15.113:0/3789974173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c198d40 con 0x7f342c06a7c0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3431fdc640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 0x7f342c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34317db640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c19e410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f342800d770 tx=0x7f342800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428017070 con 0x7f342c06a7c0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f34280054f0 con 0x7f342c06a7c0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c199030 con 0x7f342c06a7c0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428004de0 con 0x7f342c06a7c0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.652+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a52d0 con 0x7f342c06a7c0 2024-04-22T07:14:57.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.653+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c103640 con 0x7f342c06a7c0 2024-04-22T07:14:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.654+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f3428004f80 con 0x7f342c06a7c0 2024-04-22T07:14:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.655+0000 7f34227fc640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f34180a3ee0 0x7f34180a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.655+0000 7f3430fda640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f34180a3ee0 0x7f34180a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.655+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f34280d0910 con 0x7f342c06a7c0 2024-04-22T07:14:57.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.655+0000 7f3430fda640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f34180a3ee0 0x7f34180a63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:57.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.659+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f342809cd90 con 0x7f342c06a7c0 2024-04-22T07:14:57.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.862+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f342c10a480 con 0x7f34180a3ee0 2024-04-22T07:14:57.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.869+0000 7f34227fc640 1 -- 172.21.15.113:0/3789974173 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f342c10a480 con 0x7f34180a3ee0 2024-04-22T07:14:57.868 INFO:teuthology.orchestra.run.smithi113.stdout:/ 1m 6m3h 2024-04-22T07:14:57.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f34180a3ee0 msgr2=0x7f34180a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f34180a3ee0 0x7f34180a63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 msgr2=0x7f342c19e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c19e410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f342800d770 tx=0x7f342800dc30 comp rx=0 tx=0).stop 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 shutdown_connections 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f34180a3ee0 0x7f34180a63a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f342c107fc0 0x7f342c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c06b160 0x7f342c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 --2- 172.21.15.113:0/3789974173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f342c06a7c0 0x7f342c19e410 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 >> 172.21.15.113:0/3789974173 conn(0x7f342c077ba0 msgr2=0x7f342c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 shutdown_connections 2024-04-22T07:14:57.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:57.872+0000 7f3433a3f640 1 -- 172.21.15.113:0/3789974173 wait complete. 2024-04-22T07:14:57.881 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-22T07:14:57.905 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-22T07:14:57.918 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-22T07:14:57.957 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 -- 172.21.15.113:0/828476240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 msgr2=0x7f050c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 --2- 172.21.15.113:0/828476240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c10dcb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 -- 172.21.15.113:0/828476240 shutdown_connections 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 --2- 172.21.15.113:0/828476240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c10dcb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 --2- 172.21.15.113:0/828476240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 0x7f050c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 --2- 172.21.15.113:0/828476240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 0x7f050c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 -- 172.21.15.113:0/828476240 >> 172.21.15.113:0/828476240 conn(0x7f050c077e30 msgr2=0x7f050c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 -- 172.21.15.113:0/828476240 shutdown_connections 2024-04-22T07:14:58.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 -- 172.21.15.113:0/828476240 wait complete. 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 Processor -- start 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.056+0000 7f051339d640 1 -- start start 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 0x7f050c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 0x7f050c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f050c112370 con 0x7f050c101ad0 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1121f0 con 0x7f050c101130 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1124f0 con 0x7f050c106f10 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56444/0 (socket says 172.21.15.113:56444) 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 -- 172.21.15.113:0/930419819 learned_addr learned my addr 172.21.15.113:0/930419819 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051193a640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 0x7f050c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0511139640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 0x7f050c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 -- 172.21.15.113:0/930419819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 msgr2=0x7f050c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 0x7f050c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 -- 172.21.15.113:0/930419819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 msgr2=0x7f050c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 0x7f050c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 -- 172.21.15.113:0/930419819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c198d90 con 0x7f050c101ad0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051193a640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 0x7f050c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0511139640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 0x7f050c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f0510938640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c19e9a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f050000db80 tx=0x7f050000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500017070 con 0x7f050c101ad0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0500003b70 con 0x7f050c101ad0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05000106c0 con 0x7f050c101ad0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c199080 con 0x7f050c101ad0 2024-04-22T07:14:58.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.057+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c1a5320 con 0x7f050c101ad0 2024-04-22T07:14:58.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.058+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050c069230 con 0x7f050c101ad0 2024-04-22T07:14:58.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.059+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f0500004020 con 0x7f050c101ad0 2024-04-22T07:14:58.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.060+0000 7f050a7fc640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04f40a3e40 0x7f04f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.060+0000 7f0511139640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04f40a3e40 0x7f04f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.060+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f05000d0930 con 0x7f050c101ad0 2024-04-22T07:14:58.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.060+0000 7f0511139640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04f40a3e40 0x7f04f40a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f0504005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.064+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0500020070 con 0x7f050c101ad0 2024-04-22T07:14:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.243+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f050c109370 con 0x7f050c101ad0 2024-04-22T07:14:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.245+0000 7f050a7fc640 1 -- 172.21.15.113:0/930419819 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7f0500004310 con 0x7f050c101ad0 2024-04-22T07:14:58.244 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:58.244 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04f40a3e40 msgr2=0x7f04f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04f40a3e40 0x7f04f40a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f0504005850 comp rx=0 tx=0).stop 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 msgr2=0x7f050c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c19e9a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f050000db80 tx=0x7f050000b630 comp rx=0 tx=0).stop 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 shutdown_connections 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04f40a3e40 0x7f04f40a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f050c106f10 0x7f050c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f050c101ad0 0x7f050c19e9a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 --2- 172.21.15.113:0/930419819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f050c101130 0x7f050c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 >> 172.21.15.113:0/930419819 conn(0x7f050c077e30 msgr2=0x7f050c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 shutdown_connections 2024-04-22T07:14:58.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.247+0000 7f051339d640 1 -- 172.21.15.113:0/930419819 wait complete. 2024-04-22T07:14:58.255 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:58 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:14:58.255 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:58 smithi113 ceph-mon[27933]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:58.255 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:58 smithi113 ceph-mon[27933]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:58.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:58.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 -- 172.21.15.113:0/3559504701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0101090 msgr2=0x7fc5f0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 --2- 172.21.15.113:0/3559504701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0101090 0x7fc5f0101470 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009900 tx=0x7fc5e002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 -- 172.21.15.113:0/3559504701 shutdown_connections 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 --2- 172.21.15.113:0/3559504701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0106160 0x7fc5f010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 --2- 172.21.15.113:0/3559504701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5f0101a40 0x7fc5f0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 --2- 172.21.15.113:0/3559504701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0101090 0x7fc5f0101470 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 -- 172.21.15.113:0/3559504701 >> 172.21.15.113:0/3559504701 conn(0x7fc5f0077f50 msgr2=0x7fc5f0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 -- 172.21.15.113:0/3559504701 shutdown_connections 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.359+0000 7fc5f664a640 1 -- 172.21.15.113:0/3559504701 wait complete. 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 Processor -- start 2024-04-22T07:14:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 -- start start 2024-04-22T07:14:58.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0101090 0x7fc5f0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 0x7fc5f0199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5f019a160 0x7fc5f01a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111200 con 0x7fc5f0106160 2024-04-22T07:14:58.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111080 con 0x7fc5f0101090 2024-04-22T07:14:58.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.360+0000 7fc5f664a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111380 con 0x7fc5f019a160 2024-04-22T07:14:58.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 0x7fc5f0199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 0x7fc5f0199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56460/0 (socket says 172.21.15.113:56460) 2024-04-22T07:14:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 -- 172.21.15.113:0/401631938 learned_addr learned my addr 172.21.15.113:0/401631938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5effff640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0101090 0x7fc5f0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5f4be7640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5f019a160 0x7fc5f01a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 -- 172.21.15.113:0/401631938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5f019a160 msgr2=0x7fc5f01a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5f019a160 0x7fc5f01a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 -- 172.21.15.113:0/401631938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0101090 msgr2=0x7fc5f0199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0101090 0x7fc5f0199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 -- 172.21.15.113:0/401631938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f01a4ee0 con 0x7fc5f0106160 2024-04-22T07:14:58.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ef7fe640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 0x7fc5f0199b80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400e9f0 tx=0x7fc5e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.361+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e400ccc0 con 0x7fc5f0106160 2024-04-22T07:14:58.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.362+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc5e40054f0 con 0x7fc5f0106160 2024-04-22T07:14:58.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.362+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f01a51d0 con 0x7fc5f0106160 2024-04-22T07:14:58.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.362+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4010690 con 0x7fc5f0106160 2024-04-22T07:14:58.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.362+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f01a56d0 con 0x7fc5f0106160 2024-04-22T07:14:58.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.363+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0068d70 con 0x7fc5f0106160 2024-04-22T07:14:58.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.364+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fc5e40046c0 con 0x7fc5f0106160 2024-04-22T07:14:58.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.365+0000 7fc5ed7fa640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc5d80a3e40 0x7fc5d80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.365+0000 7fc5effff640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc5d80a3e40 0x7fc5d80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.365+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc5e40d1110 con 0x7fc5f0106160 2024-04-22T07:14:58.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.365+0000 7fc5effff640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc5d80a3e40 0x7fc5d80a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009230 tx=0x7fc5e0005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.369+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5e4053ff0 con 0x7fc5f0106160 2024-04-22T07:14:58.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.544+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5f0109390 con 0x7fc5f0106160 2024-04-22T07:14:58.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.544+0000 7fc5ed7fa640 1 -- 172.21.15.113:0/401631938 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fc5e409d590 con 0x7fc5f0106160 2024-04-22T07:14:58.543 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc5d80a3e40 msgr2=0x7fc5d80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc5d80a3e40 0x7fc5d80a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009230 tx=0x7fc5e0005fd0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 msgr2=0x7fc5f0199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 0x7fc5f0199b80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400e9f0 tx=0x7fc5e400eeb0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 shutdown_connections 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc5d80a3e40 0x7fc5d80a6300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5f019a160 0x7fc5f01a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5f0106160 0x7fc5f0199b80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 --2- 172.21.15.113:0/401631938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0101090 0x7fc5f0199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 >> 172.21.15.113:0/401631938 conn(0x7fc5f0077f50 msgr2=0x7fc5f00ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 shutdown_connections 2024-04-22T07:14:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.546+0000 7fc5f664a640 1 -- 172.21.15.113:0/401631938 wait complete. 2024-04-22T07:14:58.556 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:58.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:58 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:14:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:58 smithi133 ceph-mon[31198]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:58.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:58 smithi133 ceph-mon[31198]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:58.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:58 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:14:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:58 smithi154 ceph-mon[31862]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:58.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:58 smithi154 ceph-mon[31862]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 -- 172.21.15.113:0/568100859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c1014a0 msgr2=0x7f998c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 --2- 172.21.15.113:0/568100859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c1014a0 0x7f998c106bd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 -- 172.21.15.113:0/568100859 shutdown_connections 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 --2- 172.21.15.113:0/568100859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 0x7f998c10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 --2- 172.21.15.113:0/568100859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c1014a0 0x7f998c106bd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 --2- 172.21.15.113:0/568100859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c100af0 0x7f998c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 -- 172.21.15.113:0/568100859 >> 172.21.15.113:0/568100859 conn(0x7f998c0760e0 msgr2=0x7f998c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 -- 172.21.15.113:0/568100859 shutdown_connections 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.658+0000 7f999154b640 1 -- 172.21.15.113:0/568100859 wait complete. 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 Processor -- start 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 -- start start 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 0x7f998c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 0x7f998c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c198730 0x7f998c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1123d0 con 0x7f998c100af0 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112250 con 0x7f998c1072b0 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f999154b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f998c112550 con 0x7f998c198730 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f998affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 0x7f998c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f998affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 0x7f998c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56486/0 (socket says 172.21.15.113:56486) 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f998affd640 1 -- 172.21.15.113:0/724033084 learned_addr learned my addr 172.21.15.113:0/724033084 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f9983fff640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 0x7f998c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f998b7fe640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c198730 0x7f998c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.659+0000 7f998affd640 1 -- 172.21.15.113:0/724033084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c198730 msgr2=0x7f998c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f998affd640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c198730 0x7f998c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f998affd640 1 -- 172.21.15.113:0/724033084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 msgr2=0x7f998c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f998affd640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 0x7f998c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f998affd640 1 -- 172.21.15.113:0/724033084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c199330 con 0x7f998c100af0 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f998b7fe640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c198730 0x7f998c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f9983fff640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 0x7f998c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f998affd640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 0x7f998c19e670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f997c00b4a0 tx=0x7f997c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c014070 con 0x7f998c100af0 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f997c010ae0 con 0x7f998c100af0 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c017ce0 con 0x7f998c100af0 2024-04-22T07:14:58.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c1a5410 con 0x7f998c100af0 2024-04-22T07:14:58.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.660+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1a5810 con 0x7f998c100af0 2024-04-22T07:14:58.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.661+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c06a030 con 0x7f998c100af0 2024-04-22T07:14:58.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.662+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f997c017070 con 0x7f998c100af0 2024-04-22T07:14:58.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.663+0000 7f9988ff9640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f99780a3ec0 0x7f99780a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.663+0000 7f9983fff640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f99780a3ec0 0x7f99780a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.663+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f997c0d0030 con 0x7f998c100af0 2024-04-22T07:14:58.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.663+0000 7f9983fff640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f99780a3ec0 0x7f99780a6380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9974006010 tx=0x7f997402c7b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.666+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f997c09c5c0 con 0x7f998c100af0 2024-04-22T07:14:58.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.843+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f998c0691c0 con 0x7f998c100af0 2024-04-22T07:14:58.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.844+0000 7f9988ff9640 1 -- 172.21.15.113:0/724033084 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f997c09bec0 con 0x7f998c100af0 2024-04-22T07:14:58.842 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:14:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f99780a3ec0 msgr2=0x7f99780a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f99780a3ec0 0x7f99780a6380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9974006010 tx=0x7f997402c7b0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 msgr2=0x7f998c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 0x7f998c19e670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f997c00b4a0 tx=0x7f997c00b960 comp rx=0 tx=0).stop 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 shutdown_connections 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f99780a3ec0 0x7f99780a6380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f998c198730 0x7f998c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f998c1072b0 0x7f998c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 --2- 172.21.15.113:0/724033084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f998c100af0 0x7f998c19e670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.845+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 >> 172.21.15.113:0/724033084 conn(0x7f998c0760e0 msgr2=0x7f998c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.846+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 shutdown_connections 2024-04-22T07:14:58.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.846+0000 7f999154b640 1 -- 172.21.15.113:0/724033084 wait complete. 2024-04-22T07:14:58.855 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:14:58.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/255110549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102b40 msgr2=0x7f1f48106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/255110549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102b40 0x7f1f48106b80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009900 tx=0x7f1f3802bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:58.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/255110549 shutdown_connections 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/255110549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f48107260 0x7f1f4810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/255110549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102b40 0x7f1f48106b80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/255110549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48102190 0x7f1f48102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/255110549 >> 172.21.15.113:0/255110549 conn(0x7f1f480fd990 msgr2=0x7f1f480ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/255110549 shutdown_connections 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/255110549 wait complete. 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 Processor -- start 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.958+0000 7f1f4d6b3640 1 -- start start 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 0x7f1f4819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48107260 0x7f1f4819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f481986a0 0x7f1f48198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481122e0 con 0x7f1f48102190 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48112160 con 0x7f1f481986a0 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48112460 con 0x7f1f48107260 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 0x7f1f4819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 0x7f1f4819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56500/0 (socket says 172.21.15.113:56500) 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 -- 172.21.15.113:0/3901744772 learned_addr learned my addr 172.21.15.113:0/3901744772 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f467fc640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48107260 0x7f1f4819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 -- 172.21.15.113:0/3901744772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48107260 msgr2=0x7f1f4819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f477fe640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f481986a0 0x7f1f48198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48107260 0x7f1f4819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 -- 172.21.15.113:0/3901744772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f481986a0 msgr2=0x7f1f48198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f481986a0 0x7f1f48198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 -- 172.21.15.113:0/3901744772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f481992a0 con 0x7f1f48102190 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f467fc640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48107260 0x7f1f4819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:14:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f46ffd640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 0x7f1f4819e5e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c890 tx=0x7f1f3c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c004220 con 0x7f1f48102190 2024-04-22T07:14:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f481a5380 con 0x7f1f48102190 2024-04-22T07:14:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f3c0043c0 con 0x7f1f48102190 2024-04-22T07:14:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0049e0 con 0x7f1f48102190 2024-04-22T07:14:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.959+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481a5780 con 0x7f1f48102190 2024-04-22T07:14:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.960+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f48103660 con 0x7f1f48102190 2024-04-22T07:14:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.962+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f1f3c005140 con 0x7f1f48102190 2024-04-22T07:14:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.962+0000 7f1f2ffff640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f300a3e40 0x7f1f300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.962+0000 7f1f467fc640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f300a3e40 0x7f1f300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.962+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f1f3c0d0c80 con 0x7f1f48102190 2024-04-22T07:14:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.962+0000 7f1f467fc640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f300a3e40 0x7f1f300a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006010 tx=0x7f1f3802c930 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:58.966+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f3c014030 con 0x7f1f48102190 2024-04-22T07:14:59.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.146+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f480691c0 con 0x7f1f48102190 2024-04-22T07:14:59.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.146+0000 7f1f2ffff640 1 -- 172.21.15.113:0/3901744772 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f1f3c09d0a0 con 0x7f1f48102190 2024-04-22T07:14:59.145 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:14:59.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f300a3e40 msgr2=0x7f1f300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f300a3e40 0x7f1f300a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006010 tx=0x7f1f3802c930 comp rx=0 tx=0).stop 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 msgr2=0x7f1f4819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 0x7f1f4819e5e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c890 tx=0x7f1f3c00cd50 comp rx=0 tx=0).stop 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 shutdown_connections 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f300a3e40 0x7f1f300a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f481986a0 0x7f1f48198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f48107260 0x7f1f4819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 --2- 172.21.15.113:0/3901744772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f48102190 0x7f1f4819e5e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 >> 172.21.15.113:0/3901744772 conn(0x7f1f480fd990 msgr2=0x7f1f480fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 shutdown_connections 2024-04-22T07:14:59.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.148+0000 7f1f4d6b3640 1 -- 172.21.15.113:0/3901744772 wait complete. 2024-04-22T07:14:59.156 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:59 smithi113 ceph-mon[27933]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-22T07:14:59.156 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:59 smithi113 ceph-mon[27933]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:59.156 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:59 smithi113 ceph-mon[27933]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:59.156 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/930419819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:59.156 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/401631938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:59.157 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:14:59 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/724033084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:59.158 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:14:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 -- 172.21.15.113:0/1821748598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584102b40 msgr2=0x7f2584106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 --2- 172.21.15.113:0/1821748598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584102b40 0x7f2584106b80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f256c009900 tx=0x7f256c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 -- 172.21.15.113:0/1821748598 shutdown_connections 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 --2- 172.21.15.113:0/1821748598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2584107260 0x7f258410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 --2- 172.21.15.113:0/1821748598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584102b40 0x7f2584106b80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 --2- 172.21.15.113:0/1821748598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2584102190 0x7f2584102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.256+0000 7f2589dce640 1 -- 172.21.15.113:0/1821748598 >> 172.21.15.113:0/1821748598 conn(0x7f25840793e0 msgr2=0x7f25840797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 -- 172.21.15.113:0/1821748598 shutdown_connections 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 -- 172.21.15.113:0/1821748598 wait complete. 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 Processor -- start 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 -- start start 2024-04-22T07:14:59.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2584102190 0x7f258419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2584107260 0x7f258419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 0x7f2584198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f25841122e0 con 0x7f2584198660 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2584112160 con 0x7f2584102190 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2589dce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2584112460 con 0x7f2584107260 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2583fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 0x7f2584198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2583fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 0x7f2584198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56526/0 (socket says 172.21.15.113:56526) 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f2583fff640 1 -- 172.21.15.113:0/2181972733 learned_addr learned my addr 172.21.15.113:0/2181972733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.257+0000 7f25837fe640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2584102190 0x7f258419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2583fff640 1 -- 172.21.15.113:0/2181972733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2584107260 msgr2=0x7f258419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2583fff640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2584107260 0x7f258419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2583fff640 1 -- 172.21.15.113:0/2181972733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2584102190 msgr2=0x7f258419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2583fff640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2584102190 0x7f258419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2583fff640 1 -- 172.21.15.113:0/2181972733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2584199260 con 0x7f2584198660 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2583fff640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 0x7f2584198ae0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f257c00cad0 tx=0x7f257c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00ae20 con 0x7f2584198660 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f257c005020 con 0x7f2584198660 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25841a5340 con 0x7f2584198660 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00f6b0 con 0x7f2584198660 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.258+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a5740 con 0x7f2584198660 2024-04-22T07:14:59.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.259+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2584103660 con 0x7f2584198660 2024-04-22T07:14:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.260+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f257c0051d0 con 0x7f2584198660 2024-04-22T07:14:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.260+0000 7f25817fa640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f25700a3e40 0x7f25700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.261+0000 7f25837fe640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f25700a3e40 0x7f25700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.261+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f257c0d0af0 con 0x7f2584198660 2024-04-22T07:14:59.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.261+0000 7f25837fe640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f25700a3e40 0x7f25700a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2574005fd0 tx=0x7f2574005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:59.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.265+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f257c014030 con 0x7f2584198660 2024-04-22T07:14:59.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.445+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f258410a4f0 con 0x7f2584198660 2024-04-22T07:14:59.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.447+0000 7f25817fa640 1 -- 172.21.15.113:0/2181972733 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7f257c09cf70 con 0x7f2584198660 2024-04-22T07:14:59.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:14:59.446 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f25700a3e40 msgr2=0x7f25700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f25700a3e40 0x7f25700a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2574005fd0 tx=0x7f2574005850 comp rx=0 tx=0).stop 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 msgr2=0x7f2584198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 0x7f2584198ae0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f257c00cad0 tx=0x7f257c00cf90 comp rx=0 tx=0).stop 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 shutdown_connections 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f25700a3e40 0x7f25700a6300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2584198660 0x7f2584198ae0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2584107260 0x7f258419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 --2- 172.21.15.113:0/2181972733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2584102190 0x7f258419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 >> 172.21.15.113:0/2181972733 conn(0x7f25840793e0 msgr2=0x7f25840ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 shutdown_connections 2024-04-22T07:14:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.449+0000 7f2589dce640 1 -- 172.21.15.113:0/2181972733 wait complete. 2024-04-22T07:14:59.459 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:14:59.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:14:59.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 -- 172.21.15.113:0/3561335311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec101090 msgr2=0x7fe6ec101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/3561335311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec101090 0x7fe6ec101470 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009900 tx=0x7fe6dc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:59.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 -- 172.21.15.113:0/3561335311 shutdown_connections 2024-04-22T07:14:59.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/3561335311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec106160 0x7fe6ec10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/3561335311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6ec101a40 0x7fe6ec105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/3561335311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec101090 0x7fe6ec101470 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 -- 172.21.15.113:0/3561335311 >> 172.21.15.113:0/3561335311 conn(0x7fe6ec077f50 msgr2=0x7fe6ec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 -- 172.21.15.113:0/3561335311 shutdown_connections 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 -- 172.21.15.113:0/3561335311 wait complete. 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 Processor -- start 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.562+0000 7fe6f20c8640 1 -- start start 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec101090 0x7fe6ec19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 0x7fe6ec19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6ec198650 0x7fe6ec198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1111e0 con 0x7fe6ec106160 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec111060 con 0x7fe6ec101090 2024-04-22T07:14:59.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec111360 con 0x7fe6ec198650 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 0x7fe6ec19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 0x7fe6ec19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56548/0 (socket says 172.21.15.113:56548) 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 -- 172.21.15.113:0/193880353 learned_addr learned my addr 172.21.15.113:0/193880353 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6ebfff640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6ec198650 0x7fe6ec198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6eb7fe640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec101090 0x7fe6ec19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 -- 172.21.15.113:0/193880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6ec198650 msgr2=0x7fe6ec198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6ec198650 0x7fe6ec198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 -- 172.21.15.113:0/193880353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec101090 msgr2=0x7fe6ec19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec101090 0x7fe6ec19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 -- 172.21.15.113:0/193880353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec199250 con 0x7fe6ec106160 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6eb7fe640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec101090 0x7fe6ec19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e3fff640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 0x7fe6ec19ead0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c970 tx=0x7fe6d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4007bb0 con 0x7fe6ec106160 2024-04-22T07:14:59.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec1a5330 con 0x7fe6ec106160 2024-04-22T07:14:59.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe6d4007d50 con 0x7fe6ec106160 2024-04-22T07:14:59.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400f450 con 0x7fe6ec106160 2024-04-22T07:14:59.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.563+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec1a5730 con 0x7fe6ec106160 2024-04-22T07:14:59.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.564+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec102560 con 0x7fe6ec106160 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:59 smithi154 ceph-mon[31862]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:59 smithi154 ceph-mon[31862]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:59 smithi154 ceph-mon[31862]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/930419819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/401631938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:14:59 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/724033084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:59 smithi133 ceph-mon[31198]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:59 smithi133 ceph-mon[31198]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:59 smithi133 ceph-mon[31198]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/930419819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/401631938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:14:59.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:14:59 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/724033084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:14:59.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.565+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fe6d4004f60 con 0x7fe6ec106160 2024-04-22T07:14:59.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.566+0000 7fe6e97fa640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe6d80a3e40 0x7fe6d80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.566+0000 7fe6eb7fe640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe6d80a3e40 0x7fe6d80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.566+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe6d40d0880 con 0x7fe6ec106160 2024-04-22T07:14:59.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.566+0000 7fe6eb7fe640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe6d80a3e40 0x7fe6d80a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc036040 tx=0x7fe6dc000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:59.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.570+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6d4014030 con 0x7fe6ec106160 2024-04-22T07:14:59.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.746+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6ec068d70 con 0x7fe6ec106160 2024-04-22T07:14:59.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.746+0000 7fe6e97fa640 1 -- 172.21.15.113:0/193880353 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fe6d409cd00 con 0x7fe6ec106160 2024-04-22T07:14:59.745 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe6d80a3e40 msgr2=0x7fe6d80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe6d80a3e40 0x7fe6d80a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc036040 tx=0x7fe6dc000bc0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 msgr2=0x7fe6ec19ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 0x7fe6ec19ead0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c970 tx=0x7fe6d400ce30 comp rx=0 tx=0).stop 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 shutdown_connections 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe6d80a3e40 0x7fe6d80a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6ec198650 0x7fe6ec198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6ec106160 0x7fe6ec19ead0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 --2- 172.21.15.113:0/193880353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6ec101090 0x7fe6ec19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 >> 172.21.15.113:0/193880353 conn(0x7fe6ec077f50 msgr2=0x7fe6ec0694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 shutdown_connections 2024-04-22T07:14:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.748+0000 7fe6f20c8640 1 -- 172.21.15.113:0/193880353 wait complete. 2024-04-22T07:14:59.758 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 -- 172.21.15.113:0/4131865468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60101a50 msgr2=0x7f4f60105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 --2- 172.21.15.113:0/4131865468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60101a50 0x7f4f60105a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4f50009900 tx=0x7f4f5002bc30 comp rx=0 tx=0).stop 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 -- 172.21.15.113:0/4131865468 shutdown_connections 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 --2- 172.21.15.113:0/4131865468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f60106170 0x7f4f6010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 --2- 172.21.15.113:0/4131865468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60101a50 0x7f4f60105a90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 --2- 172.21.15.113:0/4131865468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f601010a0 0x7f4f60101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 -- 172.21.15.113:0/4131865468 >> 172.21.15.113:0/4131865468 conn(0x7f4f60077eb0 msgr2=0x7f4f600782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 -- 172.21.15.113:0/4131865468 shutdown_connections 2024-04-22T07:14:59.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 -- 172.21.15.113:0/4131865468 wait complete. 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 Processor -- start 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.860+0000 7f4f66552640 1 -- start start 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f601010a0 0x7f4f601996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 0x7f4f60199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6019a180 0x7f4f601a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60111250 con 0x7f4f60106170 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f601110d0 con 0x7f4f6019a180 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f601113d0 con 0x7f4f601010a0 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 0x7f4f60199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 0x7f4f60199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56568/0 (socket says 172.21.15.113:56568) 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 -- 172.21.15.113:0/2512189029 learned_addr learned my addr 172.21.15.113:0/2512189029 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5ffff640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f601010a0 0x7f4f601996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 -- 172.21.15.113:0/2512189029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f601010a0 msgr2=0x7f4f601996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f601010a0 0x7f4f601996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 -- 172.21.15.113:0/2512189029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6019a180 msgr2=0x7f4f601a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f64aef640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6019a180 0x7f4f601a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6019a180 0x7f4f601a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 -- 172.21.15.113:0/2512189029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f601a4f30 con 0x7f4f60106170 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5ffff640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f601010a0 0x7f4f601996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f64aef640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6019a180 0x7f4f601a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5f7fe640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 0x7f4f60199bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f5002cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50041070 con 0x7f4f60106170 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4f500044d0 con 0x7f4f60106170 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f601a51c0 con 0x7f4f60106170 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50003d50 con 0x7f4f60106170 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.861+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f601a5710 con 0x7f4f60106170 2024-04-22T07:14:59.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.862+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f600691c0 con 0x7f4f60106170 2024-04-22T07:14:59.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.863+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f4f50044070 con 0x7f4f60106170 2024-04-22T07:14:59.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.864+0000 7f4f5d7fa640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4f4c0a3e40 0x7f4f4c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:14:59.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.864+0000 7f4f5ffff640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4f4c0a3e40 0x7f4f4c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:14:59.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.864+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f4f500f2500 con 0x7f4f60106170 2024-04-22T07:14:59.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.865+0000 7f4f5ffff640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4f4c0a3e40 0x7f4f4c0a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009b40 tx=0x7f4f48005e50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:14:59.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:14:59.869+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f500beb20 con 0x7f4f60106170 2024-04-22T07:15:00.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.045+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f60109400 con 0x7f4f60106170 2024-04-22T07:15:00.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.046+0000 7f4f5d7fa640 1 -- 172.21.15.113:0/2512189029 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f4f500be420 con 0x7f4f60106170 2024-04-22T07:15:00.044 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:15:00.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.047+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4f4c0a3e40 msgr2=0x7f4f4c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.047+0000 7f4f66552640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4f4c0a3e40 0x7f4f4c0a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009b40 tx=0x7f4f48005e50 comp rx=0 tx=0).stop 2024-04-22T07:15:00.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.047+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 msgr2=0x7f4f60199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.047+0000 7f4f66552640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 0x7f4f60199bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f5002cdf0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 shutdown_connections 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4f4c0a3e40 0x7f4f4c0a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6019a180 0x7f4f601a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f60106170 0x7f4f60199bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 --2- 172.21.15.113:0/2512189029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f601010a0 0x7f4f601996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 >> 172.21.15.113:0/2512189029 conn(0x7f4f60077eb0 msgr2=0x7f4f60100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 shutdown_connections 2024-04-22T07:15:00.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.048+0000 7f4f66552640 1 -- 172.21.15.113:0/2512189029 wait complete. 2024-04-22T07:15:00.057 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:15:00.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 -- 172.21.15.113:0/3381269806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 msgr2=0x7f8b1c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 --2- 172.21.15.113:0/3381269806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c102210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009900 tx=0x7f8b0c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:00.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 -- 172.21.15.113:0/3381269806 shutdown_connections 2024-04-22T07:15:00.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 --2- 172.21.15.113:0/3381269806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c1072f0 0x7f8b1c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 --2- 172.21.15.113:0/3381269806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1027e0 0x7f8b1c106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 --2- 172.21.15.113:0/3381269806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c102210 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 -- 172.21.15.113:0/3381269806 >> 172.21.15.113:0/3381269806 conn(0x7f8b1c076150 msgr2=0x7f8b1c076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:00.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 -- 172.21.15.113:0/3381269806 shutdown_connections 2024-04-22T07:15:00.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 -- 172.21.15.113:0/3381269806 wait complete. 2024-04-22T07:15:00.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 Processor -- start 2024-04-22T07:15:00.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.162+0000 7f8b2093d640 1 -- start start 2024-04-22T07:15:00.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1072f0 0x7f8b1c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c197f60 0x7f8b1c1a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c111250 con 0x7f8b1c101e30 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c1110d0 con 0x7f8b1c197f60 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c1113d0 con 0x7f8b1c1072f0 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56586/0 (socket says 172.21.15.113:56586) 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 -- 172.21.15.113:0/484608611 learned_addr learned my addr 172.21.15.113:0/484608611 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 -- 172.21.15.113:0/484608611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1072f0 msgr2=0x7f8b1c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1ad9d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c197f60 0x7f8b1c1a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b13fff640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1072f0 0x7f8b1c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1072f0 0x7f8b1c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 -- 172.21.15.113:0/484608611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c197f60 msgr2=0x7f8b1c1a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c197f60 0x7f8b1c1a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 -- 172.21.15.113:0/484608611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c1a7120 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b13fff640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1072f0 0x7f8b1c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1ad9d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c197f60 0x7f8b1c1a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b1a59c640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c1974e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c036040 tx=0x7f8b0c02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c041070 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c1a73b0 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b0c0052f0 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c003db0 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.163+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c1a78f0 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.164+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c0691c0 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.166+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f8b0c044070 con 0x7f8b1c101e30 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.166+0000 7f8b137fe640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b080a3e40 0x7f8b080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.167+0000 7f8b13fff640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b080a3e40 0x7f8b080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.167+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8b0c0f29a0 con 0x7f8b1c101e30 2024-04-22T07:15:00.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.167+0000 7f8b13fff640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b080a3e40 0x7f8b080a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006fd0 tx=0x7f8b04008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:00.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.170+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b0c0bef30 con 0x7f8b1c101e30 2024-04-22T07:15:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3901744772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2181972733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/193880353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2512189029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:15:00.224+0000 7f1389a37700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:15:00.224+0000 7f1389a37700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:15:00.224+0000 7f1389a37700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:15:00.224+0000 7f1389a37700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:15:00.335 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:15:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:15:00.224+0000 7f1389a37700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T07:15:00.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.365+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b1c109300 con 0x7f8b1c101e30 2024-04-22T07:15:00.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.365+0000 7f8b137fe640 1 -- 172.21.15.113:0/484608611 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f8b0c0be830 con 0x7f8b1c101e30 2024-04-22T07:15:00.363 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b080a3e40 msgr2=0x7f8b080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b080a3e40 0x7f8b080a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006fd0 tx=0x7f8b04008040 comp rx=0 tx=0).stop 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 msgr2=0x7f8b1c1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c1974e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c036040 tx=0x7f8b0c02c710 comp rx=0 tx=0).stop 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 shutdown_connections 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b080a3e40 0x7f8b080a6300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1c197f60 0x7f8b1c1a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b1c1072f0 0x7f8b1c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 --2- 172.21.15.113:0/484608611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b1c101e30 0x7f8b1c1974e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 >> 172.21.15.113:0/484608611 conn(0x7f8b1c076150 msgr2=0x7f8b1c104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.367+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 shutdown_connections 2024-04-22T07:15:00.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.368+0000 7f8b2093d640 1 -- 172.21.15.113:0/484608611 wait complete. 2024-04-22T07:15:00.377 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-22T07:15:00.377 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 -- 172.21.15.113:0/2242415600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74101b60 msgr2=0x7fea7410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 --2- 172.21.15.113:0/2242415600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74101b60 0x7fea7410dc90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009cd0 tx=0x7fea6c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 -- 172.21.15.113:0/2242415600 shutdown_connections 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 --2- 172.21.15.113:0/2242415600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74101b60 0x7fea7410dc90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 --2- 172.21.15.113:0/2242415600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea741011a0 0x7fea74101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 --2- 172.21.15.113:0/2242415600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea74105e30 0x7fea74106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 -- 172.21.15.113:0/2242415600 >> 172.21.15.113:0/2242415600 conn(0x7fea74077e30 msgr2=0x7fea74078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 -- 172.21.15.113:0/2242415600 shutdown_connections 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.478+0000 7fea7a352640 1 -- 172.21.15.113:0/2242415600 wait complete. 2024-04-22T07:15:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 Processor -- start 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 -- start start 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741011a0 0x7fea741995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea74101b60 0x7fea74199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 0x7fea741a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111200 con 0x7fea74105e30 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111080 con 0x7fea741011a0 2024-04-22T07:15:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.479+0000 7fea7a352640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea74111380 con 0x7fea74101b60 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 0x7fea741a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 0x7fea741a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56628/0 (socket says 172.21.15.113:56628) 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 -- 172.21.15.113:0/1286533662 learned_addr learned my addr 172.21.15.113:0/1286533662 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea737fe640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea74101b60 0x7fea74199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea73fff640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741011a0 0x7fea741995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 -- 172.21.15.113:0/1286533662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea74101b60 msgr2=0x7fea74199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea74101b60 0x7fea74199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 -- 172.21.15.113:0/1286533662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741011a0 msgr2=0x7fea741995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741011a0 0x7fea741995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 -- 172.21.15.113:0/1286533662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea741a4b80 con 0x7fea74105e30 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea73fff640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741011a0 0x7fea741995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea788ef640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 0x7fea741a4640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ca0 tx=0x7fea6c035d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:00.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c039040 con 0x7fea74105e30 2024-04-22T07:15:00.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea6c044070 con 0x7fea74105e30 2024-04-22T07:15:00.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea741a4d50 con 0x7fea74105e30 2024-04-22T07:15:00.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0043c0 con 0x7fea74105e30 2024-04-22T07:15:00.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.480+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a51c0 con 0x7fea74105e30 2024-04-22T07:15:00.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.481+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea74068de0 con 0x7fea74105e30 2024-04-22T07:15:00.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.482+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fea6c004560 con 0x7fea74105e30 2024-04-22T07:15:00.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.483+0000 7fea717fa640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea5c0a3e40 0x7fea5c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.483+0000 7fea73fff640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea5c0a3e40 0x7fea5c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.483+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fea6c0f2fc0 con 0x7fea74105e30 2024-04-22T07:15:00.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.483+0000 7fea73fff640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea5c0a3e40 0x7fea5c0a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea64009ad0 tx=0x7fea64005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:00.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.487+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea6c0bf550 con 0x7fea74105e30 2024-04-22T07:15:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:00 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3901744772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:00 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2181972733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:00 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:15:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:00 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/193880353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:00 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2512189029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:00 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3901744772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:00 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2181972733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:00 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:15:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:00 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/193880353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:00 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2512189029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:15:00.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.665+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea74062390 con 0x7fea5c0a3e40 2024-04-22T07:15:00.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.672+0000 7fea717fa640 1 -- 172.21.15.113:0/1286533662 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257749 (secure 0 0 0) 0x7fea74062390 con 0x7fea5c0a3e40 2024-04-22T07:15:00.673 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:00.674 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:15:00.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.675+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea5c0a3e40 msgr2=0x7fea5c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.675+0000 7fea7a352640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea5c0a3e40 0x7fea5c0a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea64009ad0 tx=0x7fea64005f90 comp rx=0 tx=0).stop 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.675+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 msgr2=0x7fea741a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.675+0000 7fea7a352640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 0x7fea741a4640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009ca0 tx=0x7fea6c035d00 comp rx=0 tx=0).stop 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 shutdown_connections 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea5c0a3e40 0x7fea5c0a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea74105e30 0x7fea741a4640 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea74101b60 0x7fea74199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 --2- 172.21.15.113:0/1286533662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea741011a0 0x7fea741995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 >> 172.21.15.113:0/1286533662 conn(0x7fea74077e30 msgr2=0x7fea740feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 shutdown_connections 2024-04-22T07:15:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.676+0000 7fea7a352640 1 -- 172.21.15.113:0/1286533662 wait complete. 2024-04-22T07:15:00.684 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-22T07:14:59.367509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191569,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1063,"num_read_kb":853,"num_write":493,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368416,"kb_used_data":15084,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573408,"statfs":{"total":1151940427776,"available":1151563169792,"internally_reserved":0,"allocated":15446016,"data_stored":9213603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3693,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":235,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991889"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679807+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:14:49.679807+0000","last_peered":"2024-04-22T07:14:49.679807+0000","last_clean":"2024-04-22T07:14:49.679807+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:14:49.679807+0000","last_undegraded":"2024-04-22T07:14:49.679807+0000","last_fullsized":"2024-04-22T07:14:49.679807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566911+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:14:49.566911+0000","last_peered":"2024-04-22T07:14:49.566911+0000","last_clean":"2024-04-22T07:14:49.566911+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:14:49.566911+0000","last_undegraded":"2024-04-22T07:14:49.566911+0000","last_fullsized":"2024-04-22T07:14:49.566911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566794+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:14:49.566794+0000","last_peered":"2024-04-22T07:14:49.566794+0000","last_clean":"2024-04-22T07:14:49.566794+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:14:49.566794+0000","last_undegraded":"2024-04-22T07:14:49.566794+0000","last_fullsized":"2024-04-22T07:14:49.566794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522113+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:14:49.522113+0000","last_peered":"2024-04-22T07:14:49.522113+0000","last_clean":"2024-04-22T07:14:49.522113+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:14:49.522113+0000","last_undegraded":"2024-04-22T07:14:49.522113+0000","last_fullsized":"2024-04-22T07:14:49.522113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446077+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:14:49.446077+0000","last_peered":"2024-04-22T07:14:49.446077+0000","last_clean":"2024-04-22T07:14:49.446077+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:14:49.446077+0000","last_undegraded":"2024-04-22T07:14:49.446077+0000","last_fullsized":"2024-04-22T07:14:49.446077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454433+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:14:49.454433+0000","last_peered":"2024-04-22T07:14:49.454433+0000","last_clean":"2024-04-22T07:14:49.454433+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:14:49.454433+0000","last_undegraded":"2024-04-22T07:14:49.454433+0000","last_fullsized":"2024-04-22T07:14:49.454433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353645+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:14:49.353645+0000","last_peered":"2024-04-22T07:14:49.353645+0000","last_clean":"2024-04-22T07:14:49.353645+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:14:49.353645+0000","last_undegraded":"2024-04-22T07:14:49.353645+0000","last_fullsized":"2024-04-22T07:14:49.353645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446353+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:14:49.446353+0000","last_peered":"2024-04-22T07:14:49.446353+0000","last_clean":"2024-04-22T07:14:49.446353+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:14:49.446353+0000","last_undegraded":"2024-04-22T07:14:49.446353+0000","last_fullsized":"2024-04-22T07:14:49.446353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518246+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:14:49.518246+0000","last_peered":"2024-04-22T07:14:49.518246+0000","last_clean":"2024-04-22T07:14:49.518246+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:14:49.518246+0000","last_undegraded":"2024-04-22T07:14:49.518246+0000","last_fullsized":"2024-04-22T07:14:49.518246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454790+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:14:49.454790+0000","last_peered":"2024-04-22T07:14:49.454790+0000","last_clean":"2024-04-22T07:14:49.454790+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:14:49.454790+0000","last_undegraded":"2024-04-22T07:14:49.454790+0000","last_fullsized":"2024-04-22T07:14:49.454790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353879+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:14:49.353879+0000","last_peered":"2024-04-22T07:14:49.353879+0000","last_clean":"2024-04-22T07:14:49.353879+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:14:49.353879+0000","last_undegraded":"2024-04-22T07:14:49.353879+0000","last_fullsized":"2024-04-22T07:14:49.353879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521981+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:14:49.521981+0000","last_peered":"2024-04-22T07:14:49.521981+0000","last_clean":"2024-04-22T07:14:49.521981+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:14:49.521981+0000","last_undegraded":"2024-04-22T07:14:49.521981+0000","last_fullsized":"2024-04-22T07:14:49.521981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520446+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:14:49.520446+0000","last_peered":"2024-04-22T07:14:49.520446+0000","last_clean":"2024-04-22T07:14:49.520446+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:14:49.520446+0000","last_undegraded":"2024-04-22T07:14:49.520446+0000","last_fullsized":"2024-04-22T07:14:49.520446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567028+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:14:49.567028+0000","last_peered":"2024-04-22T07:14:49.567028+0000","last_clean":"2024-04-22T07:14:49.567028+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:14:49.567028+0000","last_undegraded":"2024-04-22T07:14:49.567028+0000","last_fullsized":"2024-04-22T07:14:49.567028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566423+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:14:49.566423+0000","last_peered":"2024-04-22T07:14:49.566423+0000","last_clean":"2024-04-22T07:14:49.566423+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:14:49.566423+0000","last_undegraded":"2024-04-22T07:14:49.566423+0000","last_fullsized":"2024-04-22T07:14:49.566423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.517925+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:14:49.517925+0000","last_peered":"2024-04-22T07:14:49.517925+0000","last_clean":"2024-04-22T07:14:49.517925+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:14:49.517925+0000","last_undegraded":"2024-04-22T07:14:49.517925+0000","last_fullsized":"2024-04-22T07:14:49.517925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679946+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:14:49.679946+0000","last_peered":"2024-04-22T07:14:49.679946+0000","last_clean":"2024-04-22T07:14:49.679946+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:14:49.679946+0000","last_undegraded":"2024-04-22T07:14:49.679946+0000","last_fullsized":"2024-04-22T07:14:49.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354137+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:14:49.354137+0000","last_peered":"2024-04-22T07:14:49.354137+0000","last_clean":"2024-04-22T07:14:49.354137+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:14:49.354137+0000","last_undegraded":"2024-04-22T07:14:49.354137+0000","last_fullsized":"2024-04-22T07:14:49.354137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356040+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:14:49.356040+0000","last_peered":"2024-04-22T07:14:49.356040+0000","last_clean":"2024-04-22T07:14:49.356040+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:14:49.356040+0000","last_undegraded":"2024-04-22T07:14:49.356040+0000","last_fullsized":"2024-04-22T07:14:49.356040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354290+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:14:49.354290+0000","last_peered":"2024-04-22T07:14:49.354290+0000","last_clean":"2024-04-22T07:14:49.354290+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:14:49.354290+0000","last_undegraded":"2024-04-22T07:14:49.354290+0000","last_fullsized":"2024-04-22T07:14:49.354290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447115+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:14:49.447115+0000","last_peered":"2024-04-22T07:14:49.447115+0000","last_clean":"2024-04-22T07:14:49.447115+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:14:49.447115+0000","last_undegraded":"2024-04-22T07:14:49.447115+0000","last_fullsized":"2024-04-22T07:14:49.447115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447311+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:14:49.447311+0000","last_peered":"2024-04-22T07:14:49.447311+0000","last_clean":"2024-04-22T07:14:49.447311+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:14:49.447311+0000","last_undegraded":"2024-04-22T07:14:49.447311+0000","last_fullsized":"2024-04-22T07:14:49.447311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567072+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:14:49.567072+0000","last_peered":"2024-04-22T07:14:49.567072+0000","last_clean":"2024-04-22T07:14:49.567072+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:14:49.567072+0000","last_undegraded":"2024-04-22T07:14:49.567072+0000","last_fullsized":"2024-04-22T07:14:49.567072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446441+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:14:49.446441+0000","last_peered":"2024-04-22T07:14:49.446441+0000","last_clean":"2024-04-22T07:14:49.446441+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:14:49.446441+0000","last_undegraded":"2024-04-22T07:14:49.446441+0000","last_fullsized":"2024-04-22T07:14:49.446441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518065+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:14:49.518065+0000","last_peered":"2024-04-22T07:14:49.518065+0000","last_clean":"2024-04-22T07:14:49.518065+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:14:49.518065+0000","last_undegraded":"2024-04-22T07:14:49.518065+0000","last_fullsized":"2024-04-22T07:14:49.518065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679550+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:14:49.679550+0000","last_peered":"2024-04-22T07:14:49.679550+0000","last_clean":"2024-04-22T07:14:49.679550+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:14:49.679550+0000","last_undegraded":"2024-04-22T07:14:49.679550+0000","last_fullsized":"2024-04-22T07:14:49.679550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451783+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:14:49.451783+0000","last_peered":"2024-04-22T07:14:49.451783+0000","last_clean":"2024-04-22T07:14:49.451783+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:14:49.451783+0000","last_undegraded":"2024-04-22T07:14:49.451783+0000","last_fullsized":"2024-04-22T07:14:49.451783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.505096+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:53.505096+0000","last_peered":"2024-04-22T07:14:53.505096+0000","last_clean":"2024-04-22T07:14:53.505096+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:53.505096+0000","last_undegraded":"2024-04-22T07:14:53.505096+0000","last_fullsized":"2024-04-22T07:14:53.505096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446021+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:14:49.446021+0000","last_peered":"2024-04-22T07:14:49.446021+0000","last_clean":"2024-04-22T07:14:49.446021+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:14:49.446021+0000","last_undegraded":"2024-04-22T07:14:49.446021+0000","last_fullsized":"2024-04-22T07:14:49.446021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454363+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:14:49.454363+0000","last_peered":"2024-04-22T07:14:49.454363+0000","last_clean":"2024-04-22T07:14:49.454363+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:14:49.454363+0000","last_undegraded":"2024-04-22T07:14:49.454363+0000","last_fullsized":"2024-04-22T07:14:49.454363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451662+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:14:49.451662+0000","last_peered":"2024-04-22T07:14:49.451662+0000","last_clean":"2024-04-22T07:14:49.451662+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:14:49.451662+0000","last_undegraded":"2024-04-22T07:14:49.451662+0000","last_fullsized":"2024-04-22T07:14:49.451662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520117+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:14:49.520117+0000","last_peered":"2024-04-22T07:14:49.520117+0000","last_clean":"2024-04-22T07:14:49.520117+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:14:49.520117+0000","last_undegraded":"2024-04-22T07:14:49.520117+0000","last_fullsized":"2024-04-22T07:14:49.520117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446512+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:14:49.446512+0000","last_peered":"2024-04-22T07:14:49.446512+0000","last_clean":"2024-04-22T07:14:49.446512+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:14:49.446512+0000","last_undegraded":"2024-04-22T07:14:49.446512+0000","last_fullsized":"2024-04-22T07:14:49.446512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451463+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:14:49.451463+0000","last_peered":"2024-04-22T07:14:49.451463+0000","last_clean":"2024-04-22T07:14:49.451463+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:14:49.451463+0000","last_undegraded":"2024-04-22T07:14:49.451463+0000","last_fullsized":"2024-04-22T07:14:49.451463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354021+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:14:49.354021+0000","last_peered":"2024-04-22T07:14:49.354021+0000","last_clean":"2024-04-22T07:14:49.354021+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:14:49.354021+0000","last_undegraded":"2024-04-22T07:14:49.354021+0000","last_fullsized":"2024-04-22T07:14:49.354021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.504677+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:53.504677+0000","last_peered":"2024-04-22T07:14:53.504677+0000","last_clean":"2024-04-22T07:14:53.504677+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:53.504677+0000","last_undegraded":"2024-04-22T07:14:53.504677+0000","last_fullsized":"2024-04-22T07:14:53.504677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518517+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:14:49.518517+0000","last_peered":"2024-04-22T07:14:49.518517+0000","last_clean":"2024-04-22T07:14:49.518517+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:14:49.518517+0000","last_undegraded":"2024-04-22T07:14:49.518517+0000","last_fullsized":"2024-04-22T07:14:49.518517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353921+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:14:49.353921+0000","last_peered":"2024-04-22T07:14:49.353921+0000","last_clean":"2024-04-22T07:14:49.353921+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:14:49.353921+0000","last_undegraded":"2024-04-22T07:14:49.353921+0000","last_fullsized":"2024-04-22T07:14:49.353921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567045+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:14:49.567045+0000","last_peered":"2024-04-22T07:14:49.567045+0000","last_clean":"2024-04-22T07:14:49.567045+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:14:49.567045+0000","last_undegraded":"2024-04-22T07:14:49.567045+0000","last_fullsized":"2024-04-22T07:14:49.567045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451908+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:14:49.451908+0000","last_peered":"2024-04-22T07:14:49.451908+0000","last_clean":"2024-04-22T07:14:49.451908+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:14:49.451908+0000","last_undegraded":"2024-04-22T07:14:49.451908+0000","last_fullsized":"2024-04-22T07:14:49.451908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518495+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:14:49.518495+0000","last_peered":"2024-04-22T07:14:49.518495+0000","last_clean":"2024-04-22T07:14:49.518495+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:14:49.518495+0000","last_undegraded":"2024-04-22T07:14:49.518495+0000","last_fullsized":"2024-04-22T07:14:49.518495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:54.505863+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:54.505863+0000","last_peered":"2024-04-22T07:14:54.505863+0000","last_clean":"2024-04-22T07:14:54.505863+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:54.505863+0000","last_undegraded":"2024-04-22T07:14:54.505863+0000","last_fullsized":"2024-04-22T07:14:54.505863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70669,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":189,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446273+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:14:49.446273+0000","last_peered":"2024-04-22T07:14:49.446273+0000","last_clean":"2024-04-22T07:14:49.446273+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:14:49.446273+0000","last_undegraded":"2024-04-22T07:14:49.446273+0000","last_fullsized":"2024-04-22T07:14:49.446273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679544+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:14:49.679544+0000","last_peered":"2024-04-22T07:14:49.679544+0000","last_clean":"2024-04-22T07:14:49.679544+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:14:49.679544+0000","last_undegraded":"2024-04-22T07:14:49.679544+0000","last_fullsized":"2024-04-22T07:14:49.679544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520133+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:14:49.520133+0000","last_peered":"2024-04-22T07:14:49.520133+0000","last_clean":"2024-04-22T07:14:49.520133+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:14:49.520133+0000","last_undegraded":"2024-04-22T07:14:49.520133+0000","last_fullsized":"2024-04-22T07:14:49.520133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451635+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:14:49.451635+0000","last_peered":"2024-04-22T07:14:49.451635+0000","last_clean":"2024-04-22T07:14:49.451635+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:14:49.451635+0000","last_undegraded":"2024-04-22T07:14:49.451635+0000","last_fullsized":"2024-04-22T07:14:49.451635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353878+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:14:49.353878+0000","last_peered":"2024-04-22T07:14:49.353878+0000","last_clean":"2024-04-22T07:14:49.353878+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:14:49.353878+0000","last_undegraded":"2024-04-22T07:14:49.353878+0000","last_fullsized":"2024-04-22T07:14:49.353878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.528806+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:53.528806+0000","last_peered":"2024-04-22T07:14:53.528806+0000","last_clean":"2024-04-22T07:14:53.528806+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:53.528806+0000","last_undegraded":"2024-04-22T07:14:53.528806+0000","last_fullsized":"2024-04-22T07:14:53.528806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353610+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:14:49.353610+0000","last_peered":"2024-04-22T07:14:49.353610+0000","last_clean":"2024-04-22T07:14:49.353610+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:14:49.353610+0000","last_undegraded":"2024-04-22T07:14:49.353610+0000","last_fullsized":"2024-04-22T07:14:49.353610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446009+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:14:49.446009+0000","last_peered":"2024-04-22T07:14:49.446009+0000","last_clean":"2024-04-22T07:14:49.446009+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:14:49.446009+0000","last_undegraded":"2024-04-22T07:14:49.446009+0000","last_fullsized":"2024-04-22T07:14:49.446009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451558+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:14:49.451558+0000","last_peered":"2024-04-22T07:14:49.451558+0000","last_clean":"2024-04-22T07:14:49.451558+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:14:49.451558+0000","last_undegraded":"2024-04-22T07:14:49.451558+0000","last_fullsized":"2024-04-22T07:14:49.451558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446380+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:14:49.446380+0000","last_peered":"2024-04-22T07:14:49.446380+0000","last_clean":"2024-04-22T07:14:49.446380+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:14:49.446380+0000","last_undegraded":"2024-04-22T07:14:49.446380+0000","last_fullsized":"2024-04-22T07:14:49.446380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679730+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:14:49.679730+0000","last_peered":"2024-04-22T07:14:49.679730+0000","last_clean":"2024-04-22T07:14:49.679730+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:14:49.679730+0000","last_undegraded":"2024-04-22T07:14:49.679730+0000","last_fullsized":"2024-04-22T07:14:49.679730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354097+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:14:49.354097+0000","last_peered":"2024-04-22T07:14:49.354097+0000","last_clean":"2024-04-22T07:14:49.354097+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:14:49.354097+0000","last_undegraded":"2024-04-22T07:14:49.354097+0000","last_fullsized":"2024-04-22T07:14:49.354097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454763+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:14:49.454763+0000","last_peered":"2024-04-22T07:14:49.454763+0000","last_clean":"2024-04-22T07:14:49.454763+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:14:49.454763+0000","last_undegraded":"2024-04-22T07:14:49.454763+0000","last_fullsized":"2024-04-22T07:14:49.454763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353758+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:14:49.353758+0000","last_peered":"2024-04-22T07:14:49.353758+0000","last_clean":"2024-04-22T07:14:49.353758+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:14:49.353758+0000","last_undegraded":"2024-04-22T07:14:49.353758+0000","last_fullsized":"2024-04-22T07:14:49.353758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454649+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:14:49.454649+0000","last_peered":"2024-04-22T07:14:49.454649+0000","last_clean":"2024-04-22T07:14:49.454649+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:14:49.454649+0000","last_undegraded":"2024-04-22T07:14:49.454649+0000","last_fullsized":"2024-04-22T07:14:49.454649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451592+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:14:49.451592+0000","last_peered":"2024-04-22T07:14:49.451592+0000","last_clean":"2024-04-22T07:14:49.451592+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:14:49.451592+0000","last_undegraded":"2024-04-22T07:14:49.451592+0000","last_fullsized":"2024-04-22T07:14:49.451592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:54.529574+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:54.529574+0000","last_peered":"2024-04-22T07:14:54.529574+0000","last_clean":"2024-04-22T07:14:54.529574+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:54.529574+0000","last_undegraded":"2024-04-22T07:14:54.529574+0000","last_fullsized":"2024-04-22T07:14:54.529574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446669+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:14:49.446669+0000","last_peered":"2024-04-22T07:14:49.446669+0000","last_clean":"2024-04-22T07:14:49.446669+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:14:49.446669+0000","last_undegraded":"2024-04-22T07:14:49.446669+0000","last_fullsized":"2024-04-22T07:14:49.446669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521956+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:14:49.521956+0000","last_peered":"2024-04-22T07:14:49.521956+0000","last_clean":"2024-04-22T07:14:49.521956+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:14:49.521956+0000","last_undegraded":"2024-04-22T07:14:49.521956+0000","last_fullsized":"2024-04-22T07:14:49.521956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":399,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:51.458109+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:14:51.458109+0000","last_peered":"2024-04-22T07:14:51.458109+0000","last_clean":"2024-04-22T07:14:51.458109+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:14:51.458109+0000","last_undegraded":"2024-04-22T07:14:51.458109+0000","last_fullsized":"2024-04-22T07:14:51.458109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.519969+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:14:49.519969+0000","last_peered":"2024-04-22T07:14:49.519969+0000","last_clean":"2024-04-22T07:14:49.519969+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:14:49.519969+0000","last_undegraded":"2024-04-22T07:14:49.519969+0000","last_fullsized":"2024-04-22T07:14:49.519969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353859+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:14:49.353859+0000","last_peered":"2024-04-22T07:14:49.353859+0000","last_clean":"2024-04-22T07:14:49.353859+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:14:49.353859+0000","last_undegraded":"2024-04-22T07:14:49.353859+0000","last_fullsized":"2024-04-22T07:14:49.353859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446310+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:14:49.446310+0000","last_peered":"2024-04-22T07:14:49.446310+0000","last_clean":"2024-04-22T07:14:49.446310+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:14:49.446310+0000","last_undegraded":"2024-04-22T07:14:49.446310+0000","last_fullsized":"2024-04-22T07:14:49.446310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520318+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:14:49.520318+0000","last_peered":"2024-04-22T07:14:49.520318+0000","last_clean":"2024-04-22T07:14:49.520318+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:14:49.520318+0000","last_undegraded":"2024-04-22T07:14:49.520318+0000","last_fullsized":"2024-04-22T07:14:49.520318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'46","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:50.178165+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:14:50.178165+0000","last_peered":"2024-04-22T07:14:50.178165+0000","last_clean":"2024-04-22T07:14:50.178165+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:14:50.178165+0000","last_undegraded":"2024-04-22T07:14:50.178165+0000","last_fullsized":"2024-04-22T07:14:50.178165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446988+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:14:49.446988+0000","last_peered":"2024-04-22T07:14:49.446988+0000","last_clean":"2024-04-22T07:14:49.446988+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:14:49.446988+0000","last_undegraded":"2024-04-22T07:14:49.446988+0000","last_fullsized":"2024-04-22T07:14:49.446988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522148+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:14:49.522148+0000","last_peered":"2024-04-22T07:14:49.522148+0000","last_clean":"2024-04-22T07:14:49.522148+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:14:49.522148+0000","last_undegraded":"2024-04-22T07:14:49.522148+0000","last_fullsized":"2024-04-22T07:14:49.522148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566877+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:14:49.566877+0000","last_peered":"2024-04-22T07:14:49.566877+0000","last_clean":"2024-04-22T07:14:49.566877+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:14:49.566877+0000","last_undegraded":"2024-04-22T07:14:49.566877+0000","last_fullsized":"2024-04-22T07:14:49.566877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518008+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:14:49.518008+0000","last_peered":"2024-04-22T07:14:49.518008+0000","last_clean":"2024-04-22T07:14:49.518008+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:14:49.518008+0000","last_undegraded":"2024-04-22T07:14:49.518008+0000","last_fullsized":"2024-04-22T07:14:49.518008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446005+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:14:49.446005+0000","last_peered":"2024-04-22T07:14:49.446005+0000","last_clean":"2024-04-22T07:14:49.446005+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:14:49.446005+0000","last_undegraded":"2024-04-22T07:14:49.446005+0000","last_fullsized":"2024-04-22T07:14:49.446005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446529+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:14:49.446529+0000","last_peered":"2024-04-22T07:14:49.446529+0000","last_clean":"2024-04-22T07:14:49.446529+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:14:49.446529+0000","last_undegraded":"2024-04-22T07:14:49.446529+0000","last_fullsized":"2024-04-22T07:14:49.446529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679532+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:14:49.679532+0000","last_peered":"2024-04-22T07:14:49.679532+0000","last_clean":"2024-04-22T07:14:49.679532+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:14:49.679532+0000","last_undegraded":"2024-04-22T07:14:49.679532+0000","last_fullsized":"2024-04-22T07:14:49.679532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355979+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:14:49.355979+0000","last_peered":"2024-04-22T07:14:49.355979+0000","last_clean":"2024-04-22T07:14:49.355979+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:14:49.355979+0000","last_undegraded":"2024-04-22T07:14:49.355979+0000","last_fullsized":"2024-04-22T07:14:49.355979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446724+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:14:49.446724+0000","last_peered":"2024-04-22T07:14:49.446724+0000","last_clean":"2024-04-22T07:14:49.446724+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:14:49.446724+0000","last_undegraded":"2024-04-22T07:14:49.446724+0000","last_fullsized":"2024-04-22T07:14:49.446724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446129+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:14:49.446129+0000","last_peered":"2024-04-22T07:14:49.446129+0000","last_clean":"2024-04-22T07:14:49.446129+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:14:49.446129+0000","last_undegraded":"2024-04-22T07:14:49.446129+0000","last_fullsized":"2024-04-22T07:14:49.446129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353868+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:14:49.353868+0000","last_peered":"2024-04-22T07:14:49.353868+0000","last_clean":"2024-04-22T07:14:49.353868+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:14:49.353868+0000","last_undegraded":"2024-04-22T07:14:49.353868+0000","last_fullsized":"2024-04-22T07:14:49.353868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356164+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:14:49.356164+0000","last_peered":"2024-04-22T07:14:49.356164+0000","last_clean":"2024-04-22T07:14:49.356164+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:14:49.356164+0000","last_undegraded":"2024-04-22T07:14:49.356164+0000","last_fullsized":"2024-04-22T07:14:49.356164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354047+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:14:49.354047+0000","last_peered":"2024-04-22T07:14:49.354047+0000","last_clean":"2024-04-22T07:14:49.354047+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:14:49.354047+0000","last_undegraded":"2024-04-22T07:14:49.354047+0000","last_fullsized":"2024-04-22T07:14:49.354047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.553901+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:14:49.553901+0000","last_peered":"2024-04-22T07:14:49.553901+0000","last_clean":"2024-04-22T07:14:49.553901+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:14:49.553901+0000","last_undegraded":"2024-04-22T07:14:49.553901+0000","last_fullsized":"2024-04-22T07:14:49.553901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353880+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:14:49.353880+0000","last_peered":"2024-04-22T07:14:49.353880+0000","last_clean":"2024-04-22T07:14:49.353880+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:14:49.353880+0000","last_undegraded":"2024-04-22T07:14:49.353880+0000","last_fullsized":"2024-04-22T07:14:49.353880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520280+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:14:49.520280+0000","last_peered":"2024-04-22T07:14:49.520280+0000","last_clean":"2024-04-22T07:14:49.520280+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:14:49.520280+0000","last_undegraded":"2024-04-22T07:14:49.520280+0000","last_fullsized":"2024-04-22T07:14:49.520280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354174+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:14:49.354174+0000","last_peered":"2024-04-22T07:14:49.354174+0000","last_clean":"2024-04-22T07:14:49.354174+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:14:49.354174+0000","last_undegraded":"2024-04-22T07:14:49.354174+0000","last_fullsized":"2024-04-22T07:14:49.354174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679633+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:14:49.679633+0000","last_peered":"2024-04-22T07:14:49.679633+0000","last_clean":"2024-04-22T07:14:49.679633+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:14:49.679633+0000","last_undegraded":"2024-04-22T07:14:49.679633+0000","last_fullsized":"2024-04-22T07:14:49.679633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355972+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:14:49.355972+0000","last_peered":"2024-04-22T07:14:49.355972+0000","last_clean":"2024-04-22T07:14:49.355972+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:14:49.355972+0000","last_undegraded":"2024-04-22T07:14:49.355972+0000","last_fullsized":"2024-04-22T07:14:49.355972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355819+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:14:49.355819+0000","last_peered":"2024-04-22T07:14:49.355819+0000","last_clean":"2024-04-22T07:14:49.355819+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:14:49.355819+0000","last_undegraded":"2024-04-22T07:14:49.355819+0000","last_fullsized":"2024-04-22T07:14:49.355819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566665+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:14:49.566665+0000","last_peered":"2024-04-22T07:14:49.566665+0000","last_clean":"2024-04-22T07:14:49.566665+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:14:49.566665+0000","last_undegraded":"2024-04-22T07:14:49.566665+0000","last_fullsized":"2024-04-22T07:14:49.566665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520295+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:14:49.520295+0000","last_peered":"2024-04-22T07:14:49.520295+0000","last_clean":"2024-04-22T07:14:49.520295+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:14:49.520295+0000","last_undegraded":"2024-04-22T07:14:49.520295+0000","last_fullsized":"2024-04-22T07:14:49.520295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679687+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:14:49.679687+0000","last_peered":"2024-04-22T07:14:49.679687+0000","last_clean":"2024-04-22T07:14:49.679687+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:14:49.679687+0000","last_undegraded":"2024-04-22T07:14:49.679687+0000","last_fullsized":"2024-04-22T07:14:49.679687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356248+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:14:49.356248+0000","last_peered":"2024-04-22T07:14:49.356248+0000","last_clean":"2024-04-22T07:14:49.356248+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:14:49.356248+0000","last_undegraded":"2024-04-22T07:14:49.356248+0000","last_fullsized":"2024-04-22T07:14:49.356248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354138+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:14:49.354138+0000","last_peered":"2024-04-22T07:14:49.354138+0000","last_clean":"2024-04-22T07:14:49.354138+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:14:49.354138+0000","last_undegraded":"2024-04-22T07:14:49.354138+0000","last_fullsized":"2024-04-22T07:14:49.354138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451527+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:14:49.451527+0000","last_peered":"2024-04-22T07:14:49.451527+0000","last_clean":"2024-04-22T07:14:49.451527+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:14:49.451527+0000","last_undegraded":"2024-04-22T07:14:49.451527+0000","last_fullsized":"2024-04-22T07:14:49.451527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446375+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:14:49.446375+0000","last_peered":"2024-04-22T07:14:49.446375+0000","last_clean":"2024-04-22T07:14:49.446375+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:14:49.446375+0000","last_undegraded":"2024-04-22T07:14:49.446375+0000","last_fullsized":"2024-04-22T07:14:49.446375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518310+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:14:49.518310+0000","last_peered":"2024-04-22T07:14:49.518310+0000","last_clean":"2024-04-22T07:14:49.518310+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:14:49.518310+0000","last_undegraded":"2024-04-22T07:14:49.518310+0000","last_fullsized":"2024-04-22T07:14:49.518310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520315+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:14:49.520315+0000","last_peered":"2024-04-22T07:14:49.520315+0000","last_clean":"2024-04-22T07:14:49.520315+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:14:49.520315+0000","last_undegraded":"2024-04-22T07:14:49.520315+0000","last_fullsized":"2024-04-22T07:14:49.520315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679771+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:14:49.679771+0000","last_peered":"2024-04-22T07:14:49.679771+0000","last_clean":"2024-04-22T07:14:49.679771+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:14:49.679771+0000","last_undegraded":"2024-04-22T07:14:49.679771+0000","last_fullsized":"2024-04-22T07:14:49.679771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600967,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":826,"num_read_kb":523,"num_write":290,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1802901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28788,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716364,"statfs":{"total":95995035648,"available":95965556736,"internally_reserved":0,"allocated":1814528,"data_stored":1265022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36684,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708468,"statfs":{"total":95995035648,"available":95957471232,"internally_reserved":0,"allocated":1380352,"data_stored":863942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1376256,"data_stored":863280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1064057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":541159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":471386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":542983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":995242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":471334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1728512,"data_stored":1194559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":204629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:15:00.688 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:15:00.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 -- 172.21.15.113:0/174171708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9441011a0 msgr2=0x7fb944101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/174171708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9441011a0 0x7fb944101620 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb934009930 tx=0x7fb93402bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:00.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 -- 172.21.15.113:0/174171708 shutdown_connections 2024-04-22T07:15:00.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/174171708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb944101b60 0x7fb94410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/174171708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9441011a0 0x7fb944101620 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/174171708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944105e30 0x7fb944106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 -- 172.21.15.113:0/174171708 >> 172.21.15.113:0/174171708 conn(0x7fb944077e30 msgr2=0x7fb944078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 -- 172.21.15.113:0/174171708 shutdown_connections 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 -- 172.21.15.113:0/174171708 wait complete. 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 Processor -- start 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.825+0000 7fb94a7fd640 1 -- start start 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9441011a0 0x7fb94419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 0x7fb94419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944105e30 0x7fb9441985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb944111240 con 0x7fb944101b60 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441110c0 con 0x7fb944105e30 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb9441113c0 con 0x7fb9441011a0 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 0x7fb94419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 0x7fb94419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56636/0 (socket says 172.21.15.113:56636) 2024-04-22T07:15:00.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 -- 172.21.15.113:0/1685761621 learned_addr learned my addr 172.21.15.113:0/1685761621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb943fff640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9441011a0 0x7fb94419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 -- 172.21.15.113:0/1685761621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9441011a0 msgr2=0x7fb94419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9441011a0 0x7fb94419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 -- 172.21.15.113:0/1685761621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944105e30 msgr2=0x7fb9441985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944105e30 0x7fb9441985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 -- 172.21.15.113:0/1685761621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb944198d70 con 0x7fb944101b60 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9437fe640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 0x7fb94419e930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb934009230 tx=0x7fb93402c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934043020 con 0x7fb944101b60 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb934004350 con 0x7fb944101b60 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb944199000 con 0x7fb944101b60 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934003ba0 con 0x7fb944101b60 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.826+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9441a52b0 con 0x7fb944101b60 2024-04-22T07:15:00.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.827+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb944068de0 con 0x7fb944101b60 2024-04-22T07:15:00.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.828+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fb934046020 con 0x7fb944101b60 2024-04-22T07:15:00.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.829+0000 7fb9417fa640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb9300a3e40 0x7fb9300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:00.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.829+0000 7fb943fff640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb9300a3e40 0x7fb9300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:00.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.829+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fb9340f2470 con 0x7fb944101b60 2024-04-22T07:15:00.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.830+0000 7fb943fff640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb9300a3e40 0x7fb9300a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00bbe0 tx=0x7fb92c00b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:00.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:00.834+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93404d070 con 0x7fb944101b60 2024-04-22T07:15:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.007+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb944062390 con 0x7fb9300a3e40 2024-04-22T07:15:01.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.014+0000 7fb9417fa640 1 -- 172.21.15.113:0/1685761621 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257749 (secure 0 0 0) 0x7fb944062390 con 0x7fb9300a3e40 2024-04-22T07:15:01.014 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:01.015 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:15:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb9300a3e40 msgr2=0x7fb9300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb9300a3e40 0x7fb9300a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00bbe0 tx=0x7fb92c00b390 comp rx=0 tx=0).stop 2024-04-22T07:15:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 msgr2=0x7fb94419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 0x7fb94419e930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb934009230 tx=0x7fb93402c6e0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 shutdown_connections 2024-04-22T07:15:01.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb9300a3e40 0x7fb9300a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944105e30 0x7fb9441985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb944101b60 0x7fb94419e930 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 --2- 172.21.15.113:0/1685761621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9441011a0 0x7fb94419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 >> 172.21.15.113:0/1685761621 conn(0x7fb944077e30 msgr2=0x7fb944069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 shutdown_connections 2024-04-22T07:15:01.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.017+0000 7fb94a7fd640 1 -- 172.21.15.113:0/1685761621 wait complete. 2024-04-22T07:15:01.026 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-22T07:14:59.367509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191569,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1063,"num_read_kb":853,"num_write":493,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368416,"kb_used_data":15084,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573408,"statfs":{"total":1151940427776,"available":1151563169792,"internally_reserved":0,"allocated":15446016,"data_stored":9213603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3693,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":235,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991889"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679807+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:14:49.679807+0000","last_peered":"2024-04-22T07:14:49.679807+0000","last_clean":"2024-04-22T07:14:49.679807+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:14:49.679807+0000","last_undegraded":"2024-04-22T07:14:49.679807+0000","last_fullsized":"2024-04-22T07:14:49.679807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566911+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:14:49.566911+0000","last_peered":"2024-04-22T07:14:49.566911+0000","last_clean":"2024-04-22T07:14:49.566911+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:14:49.566911+0000","last_undegraded":"2024-04-22T07:14:49.566911+0000","last_fullsized":"2024-04-22T07:14:49.566911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566794+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:14:49.566794+0000","last_peered":"2024-04-22T07:14:49.566794+0000","last_clean":"2024-04-22T07:14:49.566794+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:14:49.566794+0000","last_undegraded":"2024-04-22T07:14:49.566794+0000","last_fullsized":"2024-04-22T07:14:49.566794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522113+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:14:49.522113+0000","last_peered":"2024-04-22T07:14:49.522113+0000","last_clean":"2024-04-22T07:14:49.522113+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:14:49.522113+0000","last_undegraded":"2024-04-22T07:14:49.522113+0000","last_fullsized":"2024-04-22T07:14:49.522113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446077+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:14:49.446077+0000","last_peered":"2024-04-22T07:14:49.446077+0000","last_clean":"2024-04-22T07:14:49.446077+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:14:49.446077+0000","last_undegraded":"2024-04-22T07:14:49.446077+0000","last_fullsized":"2024-04-22T07:14:49.446077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454433+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:14:49.454433+0000","last_peered":"2024-04-22T07:14:49.454433+0000","last_clean":"2024-04-22T07:14:49.454433+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:14:49.454433+0000","last_undegraded":"2024-04-22T07:14:49.454433+0000","last_fullsized":"2024-04-22T07:14:49.454433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353645+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:14:49.353645+0000","last_peered":"2024-04-22T07:14:49.353645+0000","last_clean":"2024-04-22T07:14:49.353645+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:14:49.353645+0000","last_undegraded":"2024-04-22T07:14:49.353645+0000","last_fullsized":"2024-04-22T07:14:49.353645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446353+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:14:49.446353+0000","last_peered":"2024-04-22T07:14:49.446353+0000","last_clean":"2024-04-22T07:14:49.446353+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:14:49.446353+0000","last_undegraded":"2024-04-22T07:14:49.446353+0000","last_fullsized":"2024-04-22T07:14:49.446353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518246+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:14:49.518246+0000","last_peered":"2024-04-22T07:14:49.518246+0000","last_clean":"2024-04-22T07:14:49.518246+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:14:49.518246+0000","last_undegraded":"2024-04-22T07:14:49.518246+0000","last_fullsized":"2024-04-22T07:14:49.518246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454790+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:14:49.454790+0000","last_peered":"2024-04-22T07:14:49.454790+0000","last_clean":"2024-04-22T07:14:49.454790+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:14:49.454790+0000","last_undegraded":"2024-04-22T07:14:49.454790+0000","last_fullsized":"2024-04-22T07:14:49.454790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353879+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:14:49.353879+0000","last_peered":"2024-04-22T07:14:49.353879+0000","last_clean":"2024-04-22T07:14:49.353879+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:14:49.353879+0000","last_undegraded":"2024-04-22T07:14:49.353879+0000","last_fullsized":"2024-04-22T07:14:49.353879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521981+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:14:49.521981+0000","last_peered":"2024-04-22T07:14:49.521981+0000","last_clean":"2024-04-22T07:14:49.521981+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:14:49.521981+0000","last_undegraded":"2024-04-22T07:14:49.521981+0000","last_fullsized":"2024-04-22T07:14:49.521981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520446+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:14:49.520446+0000","last_peered":"2024-04-22T07:14:49.520446+0000","last_clean":"2024-04-22T07:14:49.520446+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:14:49.520446+0000","last_undegraded":"2024-04-22T07:14:49.520446+0000","last_fullsized":"2024-04-22T07:14:49.520446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567028+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:14:49.567028+0000","last_peered":"2024-04-22T07:14:49.567028+0000","last_clean":"2024-04-22T07:14:49.567028+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:14:49.567028+0000","last_undegraded":"2024-04-22T07:14:49.567028+0000","last_fullsized":"2024-04-22T07:14:49.567028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566423+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:14:49.566423+0000","last_peered":"2024-04-22T07:14:49.566423+0000","last_clean":"2024-04-22T07:14:49.566423+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:14:49.566423+0000","last_undegraded":"2024-04-22T07:14:49.566423+0000","last_fullsized":"2024-04-22T07:14:49.566423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.517925+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:14:49.517925+0000","last_peered":"2024-04-22T07:14:49.517925+0000","last_clean":"2024-04-22T07:14:49.517925+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:14:49.517925+0000","last_undegraded":"2024-04-22T07:14:49.517925+0000","last_fullsized":"2024-04-22T07:14:49.517925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679946+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:14:49.679946+0000","last_peered":"2024-04-22T07:14:49.679946+0000","last_clean":"2024-04-22T07:14:49.679946+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:14:49.679946+0000","last_undegraded":"2024-04-22T07:14:49.679946+0000","last_fullsized":"2024-04-22T07:14:49.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354137+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:14:49.354137+0000","last_peered":"2024-04-22T07:14:49.354137+0000","last_clean":"2024-04-22T07:14:49.354137+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:14:49.354137+0000","last_undegraded":"2024-04-22T07:14:49.354137+0000","last_fullsized":"2024-04-22T07:14:49.354137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356040+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:14:49.356040+0000","last_peered":"2024-04-22T07:14:49.356040+0000","last_clean":"2024-04-22T07:14:49.356040+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:14:49.356040+0000","last_undegraded":"2024-04-22T07:14:49.356040+0000","last_fullsized":"2024-04-22T07:14:49.356040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354290+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:14:49.354290+0000","last_peered":"2024-04-22T07:14:49.354290+0000","last_clean":"2024-04-22T07:14:49.354290+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:14:49.354290+0000","last_undegraded":"2024-04-22T07:14:49.354290+0000","last_fullsized":"2024-04-22T07:14:49.354290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447115+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:14:49.447115+0000","last_peered":"2024-04-22T07:14:49.447115+0000","last_clean":"2024-04-22T07:14:49.447115+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:14:49.447115+0000","last_undegraded":"2024-04-22T07:14:49.447115+0000","last_fullsized":"2024-04-22T07:14:49.447115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447311+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:14:49.447311+0000","last_peered":"2024-04-22T07:14:49.447311+0000","last_clean":"2024-04-22T07:14:49.447311+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:14:49.447311+0000","last_undegraded":"2024-04-22T07:14:49.447311+0000","last_fullsized":"2024-04-22T07:14:49.447311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567072+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:14:49.567072+0000","last_peered":"2024-04-22T07:14:49.567072+0000","last_clean":"2024-04-22T07:14:49.567072+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:14:49.567072+0000","last_undegraded":"2024-04-22T07:14:49.567072+0000","last_fullsized":"2024-04-22T07:14:49.567072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446441+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:14:49.446441+0000","last_peered":"2024-04-22T07:14:49.446441+0000","last_clean":"2024-04-22T07:14:49.446441+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:14:49.446441+0000","last_undegraded":"2024-04-22T07:14:49.446441+0000","last_fullsized":"2024-04-22T07:14:49.446441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518065+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:14:49.518065+0000","last_peered":"2024-04-22T07:14:49.518065+0000","last_clean":"2024-04-22T07:14:49.518065+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:14:49.518065+0000","last_undegraded":"2024-04-22T07:14:49.518065+0000","last_fullsized":"2024-04-22T07:14:49.518065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679550+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:14:49.679550+0000","last_peered":"2024-04-22T07:14:49.679550+0000","last_clean":"2024-04-22T07:14:49.679550+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:14:49.679550+0000","last_undegraded":"2024-04-22T07:14:49.679550+0000","last_fullsized":"2024-04-22T07:14:49.679550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451783+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:14:49.451783+0000","last_peered":"2024-04-22T07:14:49.451783+0000","last_clean":"2024-04-22T07:14:49.451783+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:14:49.451783+0000","last_undegraded":"2024-04-22T07:14:49.451783+0000","last_fullsized":"2024-04-22T07:14:49.451783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.505096+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:53.505096+0000","last_peered":"2024-04-22T07:14:53.505096+0000","last_clean":"2024-04-22T07:14:53.505096+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:53.505096+0000","last_undegraded":"2024-04-22T07:14:53.505096+0000","last_fullsized":"2024-04-22T07:14:53.505096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446021+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:14:49.446021+0000","last_peered":"2024-04-22T07:14:49.446021+0000","last_clean":"2024-04-22T07:14:49.446021+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:14:49.446021+0000","last_undegraded":"2024-04-22T07:14:49.446021+0000","last_fullsized":"2024-04-22T07:14:49.446021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454363+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:14:49.454363+0000","last_peered":"2024-04-22T07:14:49.454363+0000","last_clean":"2024-04-22T07:14:49.454363+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:14:49.454363+0000","last_undegraded":"2024-04-22T07:14:49.454363+0000","last_fullsized":"2024-04-22T07:14:49.454363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451662+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:14:49.451662+0000","last_peered":"2024-04-22T07:14:49.451662+0000","last_clean":"2024-04-22T07:14:49.451662+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:14:49.451662+0000","last_undegraded":"2024-04-22T07:14:49.451662+0000","last_fullsized":"2024-04-22T07:14:49.451662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520117+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:14:49.520117+0000","last_peered":"2024-04-22T07:14:49.520117+0000","last_clean":"2024-04-22T07:14:49.520117+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:14:49.520117+0000","last_undegraded":"2024-04-22T07:14:49.520117+0000","last_fullsized":"2024-04-22T07:14:49.520117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446512+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:14:49.446512+0000","last_peered":"2024-04-22T07:14:49.446512+0000","last_clean":"2024-04-22T07:14:49.446512+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:14:49.446512+0000","last_undegraded":"2024-04-22T07:14:49.446512+0000","last_fullsized":"2024-04-22T07:14:49.446512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451463+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:14:49.451463+0000","last_peered":"2024-04-22T07:14:49.451463+0000","last_clean":"2024-04-22T07:14:49.451463+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:14:49.451463+0000","last_undegraded":"2024-04-22T07:14:49.451463+0000","last_fullsized":"2024-04-22T07:14:49.451463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354021+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:14:49.354021+0000","last_peered":"2024-04-22T07:14:49.354021+0000","last_clean":"2024-04-22T07:14:49.354021+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:14:49.354021+0000","last_undegraded":"2024-04-22T07:14:49.354021+0000","last_fullsized":"2024-04-22T07:14:49.354021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.504677+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:53.504677+0000","last_peered":"2024-04-22T07:14:53.504677+0000","last_clean":"2024-04-22T07:14:53.504677+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:53.504677+0000","last_undegraded":"2024-04-22T07:14:53.504677+0000","last_fullsized":"2024-04-22T07:14:53.504677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518517+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:14:49.518517+0000","last_peered":"2024-04-22T07:14:49.518517+0000","last_clean":"2024-04-22T07:14:49.518517+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:14:49.518517+0000","last_undegraded":"2024-04-22T07:14:49.518517+0000","last_fullsized":"2024-04-22T07:14:49.518517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353921+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:14:49.353921+0000","last_peered":"2024-04-22T07:14:49.353921+0000","last_clean":"2024-04-22T07:14:49.353921+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:14:49.353921+0000","last_undegraded":"2024-04-22T07:14:49.353921+0000","last_fullsized":"2024-04-22T07:14:49.353921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567045+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:14:49.567045+0000","last_peered":"2024-04-22T07:14:49.567045+0000","last_clean":"2024-04-22T07:14:49.567045+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:14:49.567045+0000","last_undegraded":"2024-04-22T07:14:49.567045+0000","last_fullsized":"2024-04-22T07:14:49.567045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451908+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:14:49.451908+0000","last_peered":"2024-04-22T07:14:49.451908+0000","last_clean":"2024-04-22T07:14:49.451908+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:14:49.451908+0000","last_undegraded":"2024-04-22T07:14:49.451908+0000","last_fullsized":"2024-04-22T07:14:49.451908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518495+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:14:49.518495+0000","last_peered":"2024-04-22T07:14:49.518495+0000","last_clean":"2024-04-22T07:14:49.518495+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:14:49.518495+0000","last_undegraded":"2024-04-22T07:14:49.518495+0000","last_fullsized":"2024-04-22T07:14:49.518495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:54.505863+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:54.505863+0000","last_peered":"2024-04-22T07:14:54.505863+0000","last_clean":"2024-04-22T07:14:54.505863+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:54.505863+0000","last_undegraded":"2024-04-22T07:14:54.505863+0000","last_fullsized":"2024-04-22T07:14:54.505863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70669,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":189,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446273+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:14:49.446273+0000","last_peered":"2024-04-22T07:14:49.446273+0000","last_clean":"2024-04-22T07:14:49.446273+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:14:49.446273+0000","last_undegraded":"2024-04-22T07:14:49.446273+0000","last_fullsized":"2024-04-22T07:14:49.446273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679544+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:14:49.679544+0000","last_peered":"2024-04-22T07:14:49.679544+0000","last_clean":"2024-04-22T07:14:49.679544+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:14:49.679544+0000","last_undegraded":"2024-04-22T07:14:49.679544+0000","last_fullsized":"2024-04-22T07:14:49.679544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520133+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:14:49.520133+0000","last_peered":"2024-04-22T07:14:49.520133+0000","last_clean":"2024-04-22T07:14:49.520133+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:14:49.520133+0000","last_undegraded":"2024-04-22T07:14:49.520133+0000","last_fullsized":"2024-04-22T07:14:49.520133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451635+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:14:49.451635+0000","last_peered":"2024-04-22T07:14:49.451635+0000","last_clean":"2024-04-22T07:14:49.451635+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:14:49.451635+0000","last_undegraded":"2024-04-22T07:14:49.451635+0000","last_fullsized":"2024-04-22T07:14:49.451635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353878+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:14:49.353878+0000","last_peered":"2024-04-22T07:14:49.353878+0000","last_clean":"2024-04-22T07:14:49.353878+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:14:49.353878+0000","last_undegraded":"2024-04-22T07:14:49.353878+0000","last_fullsized":"2024-04-22T07:14:49.353878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.528806+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:53.528806+0000","last_peered":"2024-04-22T07:14:53.528806+0000","last_clean":"2024-04-22T07:14:53.528806+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:53.528806+0000","last_undegraded":"2024-04-22T07:14:53.528806+0000","last_fullsized":"2024-04-22T07:14:53.528806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353610+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:14:49.353610+0000","last_peered":"2024-04-22T07:14:49.353610+0000","last_clean":"2024-04-22T07:14:49.353610+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:14:49.353610+0000","last_undegraded":"2024-04-22T07:14:49.353610+0000","last_fullsized":"2024-04-22T07:14:49.353610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446009+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:14:49.446009+0000","last_peered":"2024-04-22T07:14:49.446009+0000","last_clean":"2024-04-22T07:14:49.446009+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:14:49.446009+0000","last_undegraded":"2024-04-22T07:14:49.446009+0000","last_fullsized":"2024-04-22T07:14:49.446009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451558+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:14:49.451558+0000","last_peered":"2024-04-22T07:14:49.451558+0000","last_clean":"2024-04-22T07:14:49.451558+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:14:49.451558+0000","last_undegraded":"2024-04-22T07:14:49.451558+0000","last_fullsized":"2024-04-22T07:14:49.451558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446380+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:14:49.446380+0000","last_peered":"2024-04-22T07:14:49.446380+0000","last_clean":"2024-04-22T07:14:49.446380+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:14:49.446380+0000","last_undegraded":"2024-04-22T07:14:49.446380+0000","last_fullsized":"2024-04-22T07:14:49.446380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679730+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:14:49.679730+0000","last_peered":"2024-04-22T07:14:49.679730+0000","last_clean":"2024-04-22T07:14:49.679730+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:14:49.679730+0000","last_undegraded":"2024-04-22T07:14:49.679730+0000","last_fullsized":"2024-04-22T07:14:49.679730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354097+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:14:49.354097+0000","last_peered":"2024-04-22T07:14:49.354097+0000","last_clean":"2024-04-22T07:14:49.354097+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:14:49.354097+0000","last_undegraded":"2024-04-22T07:14:49.354097+0000","last_fullsized":"2024-04-22T07:14:49.354097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454763+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:14:49.454763+0000","last_peered":"2024-04-22T07:14:49.454763+0000","last_clean":"2024-04-22T07:14:49.454763+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:14:49.454763+0000","last_undegraded":"2024-04-22T07:14:49.454763+0000","last_fullsized":"2024-04-22T07:14:49.454763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353758+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:14:49.353758+0000","last_peered":"2024-04-22T07:14:49.353758+0000","last_clean":"2024-04-22T07:14:49.353758+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:14:49.353758+0000","last_undegraded":"2024-04-22T07:14:49.353758+0000","last_fullsized":"2024-04-22T07:14:49.353758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454649+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:14:49.454649+0000","last_peered":"2024-04-22T07:14:49.454649+0000","last_clean":"2024-04-22T07:14:49.454649+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:14:49.454649+0000","last_undegraded":"2024-04-22T07:14:49.454649+0000","last_fullsized":"2024-04-22T07:14:49.454649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451592+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:14:49.451592+0000","last_peered":"2024-04-22T07:14:49.451592+0000","last_clean":"2024-04-22T07:14:49.451592+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:14:49.451592+0000","last_undegraded":"2024-04-22T07:14:49.451592+0000","last_fullsized":"2024-04-22T07:14:49.451592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:54.529574+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:54.529574+0000","last_peered":"2024-04-22T07:14:54.529574+0000","last_clean":"2024-04-22T07:14:54.529574+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:54.529574+0000","last_undegraded":"2024-04-22T07:14:54.529574+0000","last_fullsized":"2024-04-22T07:14:54.529574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446669+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:14:49.446669+0000","last_peered":"2024-04-22T07:14:49.446669+0000","last_clean":"2024-04-22T07:14:49.446669+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:14:49.446669+0000","last_undegraded":"2024-04-22T07:14:49.446669+0000","last_fullsized":"2024-04-22T07:14:49.446669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521956+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:14:49.521956+0000","last_peered":"2024-04-22T07:14:49.521956+0000","last_clean":"2024-04-22T07:14:49.521956+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:14:49.521956+0000","last_undegraded":"2024-04-22T07:14:49.521956+0000","last_fullsized":"2024-04-22T07:14:49.521956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":399,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:51.458109+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:14:51.458109+0000","last_peered":"2024-04-22T07:14:51.458109+0000","last_clean":"2024-04-22T07:14:51.458109+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:14:51.458109+0000","last_undegraded":"2024-04-22T07:14:51.458109+0000","last_fullsized":"2024-04-22T07:14:51.458109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.519969+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:14:49.519969+0000","last_peered":"2024-04-22T07:14:49.519969+0000","last_clean":"2024-04-22T07:14:49.519969+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:14:49.519969+0000","last_undegraded":"2024-04-22T07:14:49.519969+0000","last_fullsized":"2024-04-22T07:14:49.519969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353859+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:14:49.353859+0000","last_peered":"2024-04-22T07:14:49.353859+0000","last_clean":"2024-04-22T07:14:49.353859+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:14:49.353859+0000","last_undegraded":"2024-04-22T07:14:49.353859+0000","last_fullsized":"2024-04-22T07:14:49.353859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446310+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:14:49.446310+0000","last_peered":"2024-04-22T07:14:49.446310+0000","last_clean":"2024-04-22T07:14:49.446310+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:14:49.446310+0000","last_undegraded":"2024-04-22T07:14:49.446310+0000","last_fullsized":"2024-04-22T07:14:49.446310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520318+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:14:49.520318+0000","last_peered":"2024-04-22T07:14:49.520318+0000","last_clean":"2024-04-22T07:14:49.520318+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:14:49.520318+0000","last_undegraded":"2024-04-22T07:14:49.520318+0000","last_fullsized":"2024-04-22T07:14:49.520318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'46","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:50.178165+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:14:50.178165+0000","last_peered":"2024-04-22T07:14:50.178165+0000","last_clean":"2024-04-22T07:14:50.178165+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:14:50.178165+0000","last_undegraded":"2024-04-22T07:14:50.178165+0000","last_fullsized":"2024-04-22T07:14:50.178165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446988+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:14:49.446988+0000","last_peered":"2024-04-22T07:14:49.446988+0000","last_clean":"2024-04-22T07:14:49.446988+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:14:49.446988+0000","last_undegraded":"2024-04-22T07:14:49.446988+0000","last_fullsized":"2024-04-22T07:14:49.446988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522148+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:14:49.522148+0000","last_peered":"2024-04-22T07:14:49.522148+0000","last_clean":"2024-04-22T07:14:49.522148+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:14:49.522148+0000","last_undegraded":"2024-04-22T07:14:49.522148+0000","last_fullsized":"2024-04-22T07:14:49.522148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566877+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:14:49.566877+0000","last_peered":"2024-04-22T07:14:49.566877+0000","last_clean":"2024-04-22T07:14:49.566877+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:14:49.566877+0000","last_undegraded":"2024-04-22T07:14:49.566877+0000","last_fullsized":"2024-04-22T07:14:49.566877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518008+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:14:49.518008+0000","last_peered":"2024-04-22T07:14:49.518008+0000","last_clean":"2024-04-22T07:14:49.518008+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:14:49.518008+0000","last_undegraded":"2024-04-22T07:14:49.518008+0000","last_fullsized":"2024-04-22T07:14:49.518008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446005+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:14:49.446005+0000","last_peered":"2024-04-22T07:14:49.446005+0000","last_clean":"2024-04-22T07:14:49.446005+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:14:49.446005+0000","last_undegraded":"2024-04-22T07:14:49.446005+0000","last_fullsized":"2024-04-22T07:14:49.446005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446529+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:14:49.446529+0000","last_peered":"2024-04-22T07:14:49.446529+0000","last_clean":"2024-04-22T07:14:49.446529+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:14:49.446529+0000","last_undegraded":"2024-04-22T07:14:49.446529+0000","last_fullsized":"2024-04-22T07:14:49.446529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679532+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:14:49.679532+0000","last_peered":"2024-04-22T07:14:49.679532+0000","last_clean":"2024-04-22T07:14:49.679532+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:14:49.679532+0000","last_undegraded":"2024-04-22T07:14:49.679532+0000","last_fullsized":"2024-04-22T07:14:49.679532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355979+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:14:49.355979+0000","last_peered":"2024-04-22T07:14:49.355979+0000","last_clean":"2024-04-22T07:14:49.355979+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:14:49.355979+0000","last_undegraded":"2024-04-22T07:14:49.355979+0000","last_fullsized":"2024-04-22T07:14:49.355979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446724+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:14:49.446724+0000","last_peered":"2024-04-22T07:14:49.446724+0000","last_clean":"2024-04-22T07:14:49.446724+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:14:49.446724+0000","last_undegraded":"2024-04-22T07:14:49.446724+0000","last_fullsized":"2024-04-22T07:14:49.446724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446129+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:14:49.446129+0000","last_peered":"2024-04-22T07:14:49.446129+0000","last_clean":"2024-04-22T07:14:49.446129+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:14:49.446129+0000","last_undegraded":"2024-04-22T07:14:49.446129+0000","last_fullsized":"2024-04-22T07:14:49.446129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353868+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:14:49.353868+0000","last_peered":"2024-04-22T07:14:49.353868+0000","last_clean":"2024-04-22T07:14:49.353868+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:14:49.353868+0000","last_undegraded":"2024-04-22T07:14:49.353868+0000","last_fullsized":"2024-04-22T07:14:49.353868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356164+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:14:49.356164+0000","last_peered":"2024-04-22T07:14:49.356164+0000","last_clean":"2024-04-22T07:14:49.356164+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:14:49.356164+0000","last_undegraded":"2024-04-22T07:14:49.356164+0000","last_fullsized":"2024-04-22T07:14:49.356164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354047+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:14:49.354047+0000","last_peered":"2024-04-22T07:14:49.354047+0000","last_clean":"2024-04-22T07:14:49.354047+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:14:49.354047+0000","last_undegraded":"2024-04-22T07:14:49.354047+0000","last_fullsized":"2024-04-22T07:14:49.354047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.553901+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:14:49.553901+0000","last_peered":"2024-04-22T07:14:49.553901+0000","last_clean":"2024-04-22T07:14:49.553901+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:14:49.553901+0000","last_undegraded":"2024-04-22T07:14:49.553901+0000","last_fullsized":"2024-04-22T07:14:49.553901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353880+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:14:49.353880+0000","last_peered":"2024-04-22T07:14:49.353880+0000","last_clean":"2024-04-22T07:14:49.353880+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:14:49.353880+0000","last_undegraded":"2024-04-22T07:14:49.353880+0000","last_fullsized":"2024-04-22T07:14:49.353880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520280+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:14:49.520280+0000","last_peered":"2024-04-22T07:14:49.520280+0000","last_clean":"2024-04-22T07:14:49.520280+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:14:49.520280+0000","last_undegraded":"2024-04-22T07:14:49.520280+0000","last_fullsized":"2024-04-22T07:14:49.520280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354174+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:14:49.354174+0000","last_peered":"2024-04-22T07:14:49.354174+0000","last_clean":"2024-04-22T07:14:49.354174+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:14:49.354174+0000","last_undegraded":"2024-04-22T07:14:49.354174+0000","last_fullsized":"2024-04-22T07:14:49.354174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679633+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:14:49.679633+0000","last_peered":"2024-04-22T07:14:49.679633+0000","last_clean":"2024-04-22T07:14:49.679633+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:14:49.679633+0000","last_undegraded":"2024-04-22T07:14:49.679633+0000","last_fullsized":"2024-04-22T07:14:49.679633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355972+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:14:49.355972+0000","last_peered":"2024-04-22T07:14:49.355972+0000","last_clean":"2024-04-22T07:14:49.355972+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:14:49.355972+0000","last_undegraded":"2024-04-22T07:14:49.355972+0000","last_fullsized":"2024-04-22T07:14:49.355972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355819+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:14:49.355819+0000","last_peered":"2024-04-22T07:14:49.355819+0000","last_clean":"2024-04-22T07:14:49.355819+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:14:49.355819+0000","last_undegraded":"2024-04-22T07:14:49.355819+0000","last_fullsized":"2024-04-22T07:14:49.355819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566665+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:14:49.566665+0000","last_peered":"2024-04-22T07:14:49.566665+0000","last_clean":"2024-04-22T07:14:49.566665+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:14:49.566665+0000","last_undegraded":"2024-04-22T07:14:49.566665+0000","last_fullsized":"2024-04-22T07:14:49.566665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520295+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:14:49.520295+0000","last_peered":"2024-04-22T07:14:49.520295+0000","last_clean":"2024-04-22T07:14:49.520295+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:14:49.520295+0000","last_undegraded":"2024-04-22T07:14:49.520295+0000","last_fullsized":"2024-04-22T07:14:49.520295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679687+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:14:49.679687+0000","last_peered":"2024-04-22T07:14:49.679687+0000","last_clean":"2024-04-22T07:14:49.679687+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:14:49.679687+0000","last_undegraded":"2024-04-22T07:14:49.679687+0000","last_fullsized":"2024-04-22T07:14:49.679687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356248+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:14:49.356248+0000","last_peered":"2024-04-22T07:14:49.356248+0000","last_clean":"2024-04-22T07:14:49.356248+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:14:49.356248+0000","last_undegraded":"2024-04-22T07:14:49.356248+0000","last_fullsized":"2024-04-22T07:14:49.356248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354138+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:14:49.354138+0000","last_peered":"2024-04-22T07:14:49.354138+0000","last_clean":"2024-04-22T07:14:49.354138+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:14:49.354138+0000","last_undegraded":"2024-04-22T07:14:49.354138+0000","last_fullsized":"2024-04-22T07:14:49.354138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451527+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:14:49.451527+0000","last_peered":"2024-04-22T07:14:49.451527+0000","last_clean":"2024-04-22T07:14:49.451527+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:14:49.451527+0000","last_undegraded":"2024-04-22T07:14:49.451527+0000","last_fullsized":"2024-04-22T07:14:49.451527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446375+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:14:49.446375+0000","last_peered":"2024-04-22T07:14:49.446375+0000","last_clean":"2024-04-22T07:14:49.446375+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:14:49.446375+0000","last_undegraded":"2024-04-22T07:14:49.446375+0000","last_fullsized":"2024-04-22T07:14:49.446375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518310+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:14:49.518310+0000","last_peered":"2024-04-22T07:14:49.518310+0000","last_clean":"2024-04-22T07:14:49.518310+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:14:49.518310+0000","last_undegraded":"2024-04-22T07:14:49.518310+0000","last_fullsized":"2024-04-22T07:14:49.518310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520315+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:14:49.520315+0000","last_peered":"2024-04-22T07:14:49.520315+0000","last_clean":"2024-04-22T07:14:49.520315+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:14:49.520315+0000","last_undegraded":"2024-04-22T07:14:49.520315+0000","last_fullsized":"2024-04-22T07:14:49.520315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679771+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:14:49.679771+0000","last_peered":"2024-04-22T07:14:49.679771+0000","last_clean":"2024-04-22T07:14:49.679771+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:14:49.679771+0000","last_undegraded":"2024-04-22T07:14:49.679771+0000","last_fullsized":"2024-04-22T07:14:49.679771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600967,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":826,"num_read_kb":523,"num_write":290,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1802901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28788,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716364,"statfs":{"total":95995035648,"available":95965556736,"internally_reserved":0,"allocated":1814528,"data_stored":1265022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36684,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708468,"statfs":{"total":95995035648,"available":95957471232,"internally_reserved":0,"allocated":1380352,"data_stored":863942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1376256,"data_stored":863280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1064057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":541159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":471386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":542983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":995242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":471334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1728512,"data_stored":1194559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":204629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:15:01.030 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-22T07:15:01.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 -- 172.21.15.113:0/1708635046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101a40 msgr2=0x7f04e0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1708635046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101a40 0x7f04e0105a80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009900 tx=0x7f04d002bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 -- 172.21.15.113:0/1708635046 shutdown_connections 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1708635046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e0106160 0x7f04e010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1708635046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101a40 0x7f04e0105a80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1708635046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0101090 0x7f04e0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 -- 172.21.15.113:0/1708635046 >> 172.21.15.113:0/1708635046 conn(0x7f04e00fc910 msgr2=0x7f04e00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 -- 172.21.15.113:0/1708635046 shutdown_connections 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.167+0000 7f04e75b6640 1 -- 172.21.15.113:0/1708635046 wait complete. 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 Processor -- start 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 -- start start 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 0x7f04e0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0106160 0x7f04e0198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e019cff0 0x7f04e019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0112300 con 0x7f04e0101090 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0112180 con 0x7f04e0106160 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e75b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0112480 con 0x7f04e019cff0 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 0x7f04e0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 0x7f04e0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56654/0 (socket says 172.21.15.113:56654) 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 -- 172.21.15.113:0/1912338913 learned_addr learned my addr 172.21.15.113:0/1912338913 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e4b51640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0106160 0x7f04e0198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 -- 172.21.15.113:0/1912338913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e019cff0 msgr2=0x7f04e019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e019cff0 0x7f04e019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 -- 172.21.15.113:0/1912338913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0106160 msgr2=0x7f04e0198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0106160 0x7f04e0198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 -- 172.21.15.113:0/1912338913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e019dae0 con 0x7f04e0101090 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e4b51640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0106160 0x7f04e0198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04e5352640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 0x7f04e0198780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f04d800ebc0 tx=0x7f04d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8010630 con 0x7f04e0101090 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.168+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f04d8010ca0 con 0x7f04e0101090 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.169+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e019ddd0 con 0x7f04e0101090 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.169+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8018ba0 con 0x7f04e0101090 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.169+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a55c0 con 0x7f04e0101090 2024-04-22T07:15:01.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.169+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0069230 con 0x7f04e0101090 2024-04-22T07:15:01.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.171+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f04d80107d0 con 0x7f04e0101090 2024-04-22T07:15:01.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.172+0000 7f04d67fc640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04cc0a3e40 0x7f04cc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.172+0000 7f04e4b51640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04cc0a3e40 0x7f04cc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.172+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f04d80d15f0 con 0x7f04e0101090 2024-04-22T07:15:01.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.172+0000 7f04e4b51640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04cc0a3e40 0x7f04cc0a6300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f04d0036040 tx=0x7f04d0000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:01.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.175+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04d8010ac0 con 0x7f04e0101090 2024-04-22T07:15:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:01 smithi113 ceph-mon[27933]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-22T07:15:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:01 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/484608611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:01.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.351+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04e0101a40 con 0x7f04cc0a3e40 2024-04-22T07:15:01.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.358+0000 7f04d67fc640 1 -- 172.21.15.113:0/1912338913 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257749 (secure 0 0 0) 0x7f04e0101a40 con 0x7f04cc0a3e40 2024-04-22T07:15:01.358 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:01.359 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-04-22T07:15:01.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04cc0a3e40 msgr2=0x7f04cc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04cc0a3e40 0x7f04cc0a6300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f04d0036040 tx=0x7f04d0000bc0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 msgr2=0x7f04e0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 0x7f04e0198780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f04d800ebc0 tx=0x7f04d800c6a0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 shutdown_connections 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f04cc0a3e40 0x7f04cc0a6300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04e019cff0 0x7f04e019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04e0106160 0x7f04e0198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 --2- 172.21.15.113:0/1912338913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04e0101090 0x7f04e0198780 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 >> 172.21.15.113:0/1912338913 conn(0x7f04e00fc910 msgr2=0x7f04e00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 shutdown_connections 2024-04-22T07:15:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.361+0000 7f04e75b6640 1 -- 172.21.15.113:0/1912338913 wait complete. 2024-04-22T07:15:01.369 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-22T07:14:59.367509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191569,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1063,"num_read_kb":853,"num_write":493,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181352,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368416,"kb_used_data":15084,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573408,"statfs":{"total":1151940427776,"available":1151563169792,"internally_reserved":0,"allocated":15446016,"data_stored":9213603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3693,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":235,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991889"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679807+0000","last_change":"2024-04-22T07:12:55.175655+0000","last_active":"2024-04-22T07:14:49.679807+0000","last_peered":"2024-04-22T07:14:49.679807+0000","last_clean":"2024-04-22T07:14:49.679807+0000","last_became_active":"2024-04-22T07:12:55.175298+0000","last_became_peered":"2024-04-22T07:12:55.175298+0000","last_unstale":"2024-04-22T07:14:49.679807+0000","last_undegraded":"2024-04-22T07:14:49.679807+0000","last_fullsized":"2024-04-22T07:14:49.679807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:37:51.874946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566911+0000","last_change":"2024-04-22T07:12:55.170463+0000","last_active":"2024-04-22T07:14:49.566911+0000","last_peered":"2024-04-22T07:14:49.566911+0000","last_clean":"2024-04-22T07:14:49.566911+0000","last_became_active":"2024-04-22T07:12:55.170076+0000","last_became_peered":"2024-04-22T07:12:55.170076+0000","last_unstale":"2024-04-22T07:14:49.566911+0000","last_undegraded":"2024-04-22T07:14:49.566911+0000","last_fullsized":"2024-04-22T07:14:49.566911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:48:57.994397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566794+0000","last_change":"2024-04-22T07:12:55.174428+0000","last_active":"2024-04-22T07:14:49.566794+0000","last_peered":"2024-04-22T07:14:49.566794+0000","last_clean":"2024-04-22T07:14:49.566794+0000","last_became_active":"2024-04-22T07:12:55.174114+0000","last_became_peered":"2024-04-22T07:12:55.174114+0000","last_unstale":"2024-04-22T07:14:49.566794+0000","last_undegraded":"2024-04-22T07:14:49.566794+0000","last_fullsized":"2024-04-22T07:14:49.566794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:36:22.222175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522113+0000","last_change":"2024-04-22T07:12:55.175695+0000","last_active":"2024-04-22T07:14:49.522113+0000","last_peered":"2024-04-22T07:14:49.522113+0000","last_clean":"2024-04-22T07:14:49.522113+0000","last_became_active":"2024-04-22T07:12:55.175364+0000","last_became_peered":"2024-04-22T07:12:55.175364+0000","last_unstale":"2024-04-22T07:14:49.522113+0000","last_undegraded":"2024-04-22T07:14:49.522113+0000","last_fullsized":"2024-04-22T07:14:49.522113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:20:14.062848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446077+0000","last_change":"2024-04-22T07:12:55.168406+0000","last_active":"2024-04-22T07:14:49.446077+0000","last_peered":"2024-04-22T07:14:49.446077+0000","last_clean":"2024-04-22T07:14:49.446077+0000","last_became_active":"2024-04-22T07:12:55.167923+0000","last_became_peered":"2024-04-22T07:12:55.167923+0000","last_unstale":"2024-04-22T07:14:49.446077+0000","last_undegraded":"2024-04-22T07:14:49.446077+0000","last_fullsized":"2024-04-22T07:14:49.446077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:00:25.296495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454433+0000","last_change":"2024-04-22T07:12:55.177397+0000","last_active":"2024-04-22T07:14:49.454433+0000","last_peered":"2024-04-22T07:14:49.454433+0000","last_clean":"2024-04-22T07:14:49.454433+0000","last_became_active":"2024-04-22T07:12:55.177062+0000","last_became_peered":"2024-04-22T07:12:55.177062+0000","last_unstale":"2024-04-22T07:14:49.454433+0000","last_undegraded":"2024-04-22T07:14:49.454433+0000","last_fullsized":"2024-04-22T07:14:49.454433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:14.411359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353645+0000","last_change":"2024-04-22T07:12:55.171113+0000","last_active":"2024-04-22T07:14:49.353645+0000","last_peered":"2024-04-22T07:14:49.353645+0000","last_clean":"2024-04-22T07:14:49.353645+0000","last_became_active":"2024-04-22T07:12:55.170763+0000","last_became_peered":"2024-04-22T07:12:55.170763+0000","last_unstale":"2024-04-22T07:14:49.353645+0000","last_undegraded":"2024-04-22T07:14:49.353645+0000","last_fullsized":"2024-04-22T07:14:49.353645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.556246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446353+0000","last_change":"2024-04-22T07:12:55.164763+0000","last_active":"2024-04-22T07:14:49.446353+0000","last_peered":"2024-04-22T07:14:49.446353+0000","last_clean":"2024-04-22T07:14:49.446353+0000","last_became_active":"2024-04-22T07:12:55.164417+0000","last_became_peered":"2024-04-22T07:12:55.164417+0000","last_unstale":"2024-04-22T07:14:49.446353+0000","last_undegraded":"2024-04-22T07:14:49.446353+0000","last_fullsized":"2024-04-22T07:14:49.446353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:09:23.079421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518246+0000","last_change":"2024-04-22T07:12:55.177656+0000","last_active":"2024-04-22T07:14:49.518246+0000","last_peered":"2024-04-22T07:14:49.518246+0000","last_clean":"2024-04-22T07:14:49.518246+0000","last_became_active":"2024-04-22T07:12:55.177352+0000","last_became_peered":"2024-04-22T07:12:55.177352+0000","last_unstale":"2024-04-22T07:14:49.518246+0000","last_undegraded":"2024-04-22T07:14:49.518246+0000","last_fullsized":"2024-04-22T07:14:49.518246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:23:05.894438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454790+0000","last_change":"2024-04-22T07:12:55.180863+0000","last_active":"2024-04-22T07:14:49.454790+0000","last_peered":"2024-04-22T07:14:49.454790+0000","last_clean":"2024-04-22T07:14:49.454790+0000","last_became_active":"2024-04-22T07:12:55.180343+0000","last_became_peered":"2024-04-22T07:12:55.180343+0000","last_unstale":"2024-04-22T07:14:49.454790+0000","last_undegraded":"2024-04-22T07:14:49.454790+0000","last_fullsized":"2024-04-22T07:14:49.454790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:19:23.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353879+0000","last_change":"2024-04-22T07:12:55.165883+0000","last_active":"2024-04-22T07:14:49.353879+0000","last_peered":"2024-04-22T07:14:49.353879+0000","last_clean":"2024-04-22T07:14:49.353879+0000","last_became_active":"2024-04-22T07:12:55.165488+0000","last_became_peered":"2024-04-22T07:12:55.165488+0000","last_unstale":"2024-04-22T07:14:49.353879+0000","last_undegraded":"2024-04-22T07:14:49.353879+0000","last_fullsized":"2024-04-22T07:14:49.353879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:26:07.295474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521981+0000","last_change":"2024-04-22T07:12:55.174081+0000","last_active":"2024-04-22T07:14:49.521981+0000","last_peered":"2024-04-22T07:14:49.521981+0000","last_clean":"2024-04-22T07:14:49.521981+0000","last_became_active":"2024-04-22T07:12:55.173742+0000","last_became_peered":"2024-04-22T07:12:55.173742+0000","last_unstale":"2024-04-22T07:14:49.521981+0000","last_undegraded":"2024-04-22T07:14:49.521981+0000","last_fullsized":"2024-04-22T07:14:49.521981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:51:02.677450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520446+0000","last_change":"2024-04-22T07:12:55.174390+0000","last_active":"2024-04-22T07:14:49.520446+0000","last_peered":"2024-04-22T07:14:49.520446+0000","last_clean":"2024-04-22T07:14:49.520446+0000","last_became_active":"2024-04-22T07:12:55.174103+0000","last_became_peered":"2024-04-22T07:12:55.174103+0000","last_unstale":"2024-04-22T07:14:49.520446+0000","last_undegraded":"2024-04-22T07:14:49.520446+0000","last_fullsized":"2024-04-22T07:14:49.520446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:43:38.796106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567028+0000","last_change":"2024-04-22T07:12:55.169405+0000","last_active":"2024-04-22T07:14:49.567028+0000","last_peered":"2024-04-22T07:14:49.567028+0000","last_clean":"2024-04-22T07:14:49.567028+0000","last_became_active":"2024-04-22T07:12:55.169076+0000","last_became_peered":"2024-04-22T07:12:55.169076+0000","last_unstale":"2024-04-22T07:14:49.567028+0000","last_undegraded":"2024-04-22T07:14:49.567028+0000","last_fullsized":"2024-04-22T07:14:49.567028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:15:00.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566423+0000","last_change":"2024-04-22T07:12:55.168668+0000","last_active":"2024-04-22T07:14:49.566423+0000","last_peered":"2024-04-22T07:14:49.566423+0000","last_clean":"2024-04-22T07:14:49.566423+0000","last_became_active":"2024-04-22T07:12:55.168301+0000","last_became_peered":"2024-04-22T07:12:55.168301+0000","last_unstale":"2024-04-22T07:14:49.566423+0000","last_undegraded":"2024-04-22T07:14:49.566423+0000","last_fullsized":"2024-04-22T07:14:49.566423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:30:31.135259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.517925+0000","last_change":"2024-04-22T07:12:55.170226+0000","last_active":"2024-04-22T07:14:49.517925+0000","last_peered":"2024-04-22T07:14:49.517925+0000","last_clean":"2024-04-22T07:14:49.517925+0000","last_became_active":"2024-04-22T07:12:55.169748+0000","last_became_peered":"2024-04-22T07:12:55.169748+0000","last_unstale":"2024-04-22T07:14:49.517925+0000","last_undegraded":"2024-04-22T07:14:49.517925+0000","last_fullsized":"2024-04-22T07:14:49.517925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:40:29.003932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679946+0000","last_change":"2024-04-22T07:12:55.176263+0000","last_active":"2024-04-22T07:14:49.679946+0000","last_peered":"2024-04-22T07:14:49.679946+0000","last_clean":"2024-04-22T07:14:49.679946+0000","last_became_active":"2024-04-22T07:12:55.175866+0000","last_became_peered":"2024-04-22T07:12:55.175866+0000","last_unstale":"2024-04-22T07:14:49.679946+0000","last_undegraded":"2024-04-22T07:14:49.679946+0000","last_fullsized":"2024-04-22T07:14:49.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:44:54.791282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354137+0000","last_change":"2024-04-22T07:12:55.169938+0000","last_active":"2024-04-22T07:14:49.354137+0000","last_peered":"2024-04-22T07:14:49.354137+0000","last_clean":"2024-04-22T07:14:49.354137+0000","last_became_active":"2024-04-22T07:12:55.169597+0000","last_became_peered":"2024-04-22T07:12:55.169597+0000","last_unstale":"2024-04-22T07:14:49.354137+0000","last_undegraded":"2024-04-22T07:14:49.354137+0000","last_fullsized":"2024-04-22T07:14:49.354137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:53.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356040+0000","last_change":"2024-04-22T07:12:55.165741+0000","last_active":"2024-04-22T07:14:49.356040+0000","last_peered":"2024-04-22T07:14:49.356040+0000","last_clean":"2024-04-22T07:14:49.356040+0000","last_became_active":"2024-04-22T07:12:55.165193+0000","last_became_peered":"2024-04-22T07:12:55.165193+0000","last_unstale":"2024-04-22T07:14:49.356040+0000","last_undegraded":"2024-04-22T07:14:49.356040+0000","last_fullsized":"2024-04-22T07:14:49.356040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:13:23.990069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354290+0000","last_change":"2024-04-22T07:13:56.196930+0000","last_active":"2024-04-22T07:14:49.354290+0000","last_peered":"2024-04-22T07:14:49.354290+0000","last_clean":"2024-04-22T07:14:49.354290+0000","last_became_active":"2024-04-22T07:13:56.190842+0000","last_became_peered":"2024-04-22T07:13:56.190842+0000","last_unstale":"2024-04-22T07:14:49.354290+0000","last_undegraded":"2024-04-22T07:14:49.354290+0000","last_fullsized":"2024-04-22T07:14:49.354290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:21.593926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447115+0000","last_change":"2024-04-22T07:12:55.172258+0000","last_active":"2024-04-22T07:14:49.447115+0000","last_peered":"2024-04-22T07:14:49.447115+0000","last_clean":"2024-04-22T07:14:49.447115+0000","last_became_active":"2024-04-22T07:12:55.171961+0000","last_became_peered":"2024-04-22T07:12:55.171961+0000","last_unstale":"2024-04-22T07:14:49.447115+0000","last_undegraded":"2024-04-22T07:14:49.447115+0000","last_fullsized":"2024-04-22T07:14:49.447115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:28:31.290573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.447311+0000","last_change":"2024-04-22T07:12:54.149101+0000","last_active":"2024-04-22T07:14:49.447311+0000","last_peered":"2024-04-22T07:14:49.447311+0000","last_clean":"2024-04-22T07:14:49.447311+0000","last_became_active":"2024-04-22T07:12:54.148745+0000","last_became_peered":"2024-04-22T07:12:54.148745+0000","last_unstale":"2024-04-22T07:14:49.447311+0000","last_undegraded":"2024-04-22T07:14:49.447311+0000","last_fullsized":"2024-04-22T07:14:49.447311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:40:48.330856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567072+0000","last_change":"2024-04-22T07:12:55.174128+0000","last_active":"2024-04-22T07:14:49.567072+0000","last_peered":"2024-04-22T07:14:49.567072+0000","last_clean":"2024-04-22T07:14:49.567072+0000","last_became_active":"2024-04-22T07:12:55.173737+0000","last_became_peered":"2024-04-22T07:12:55.173737+0000","last_unstale":"2024-04-22T07:14:49.567072+0000","last_undegraded":"2024-04-22T07:14:49.567072+0000","last_fullsized":"2024-04-22T07:14:49.567072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:14:45.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446441+0000","last_change":"2024-04-22T07:12:54.147019+0000","last_active":"2024-04-22T07:14:49.446441+0000","last_peered":"2024-04-22T07:14:49.446441+0000","last_clean":"2024-04-22T07:14:49.446441+0000","last_became_active":"2024-04-22T07:12:54.146696+0000","last_became_peered":"2024-04-22T07:12:54.146696+0000","last_unstale":"2024-04-22T07:14:49.446441+0000","last_undegraded":"2024-04-22T07:14:49.446441+0000","last_fullsized":"2024-04-22T07:14:49.446441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:31:53.400053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518065+0000","last_change":"2024-04-22T07:12:55.171854+0000","last_active":"2024-04-22T07:14:49.518065+0000","last_peered":"2024-04-22T07:14:49.518065+0000","last_clean":"2024-04-22T07:14:49.518065+0000","last_became_active":"2024-04-22T07:12:55.170346+0000","last_became_peered":"2024-04-22T07:12:55.170346+0000","last_unstale":"2024-04-22T07:14:49.518065+0000","last_undegraded":"2024-04-22T07:14:49.518065+0000","last_fullsized":"2024-04-22T07:14:49.518065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:41.458895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679550+0000","last_change":"2024-04-22T07:13:56.200838+0000","last_active":"2024-04-22T07:14:49.679550+0000","last_peered":"2024-04-22T07:14:49.679550+0000","last_clean":"2024-04-22T07:14:49.679550+0000","last_became_active":"2024-04-22T07:13:56.192656+0000","last_became_peered":"2024-04-22T07:13:56.192656+0000","last_unstale":"2024-04-22T07:14:49.679550+0000","last_undegraded":"2024-04-22T07:14:49.679550+0000","last_fullsized":"2024-04-22T07:14:49.679550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:17:24.470926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451783+0000","last_change":"2024-04-22T07:12:55.173988+0000","last_active":"2024-04-22T07:14:49.451783+0000","last_peered":"2024-04-22T07:14:49.451783+0000","last_clean":"2024-04-22T07:14:49.451783+0000","last_became_active":"2024-04-22T07:12:55.173660+0000","last_became_peered":"2024-04-22T07:12:55.173660+0000","last_unstale":"2024-04-22T07:14:49.451783+0000","last_undegraded":"2024-04-22T07:14:49.451783+0000","last_fullsized":"2024-04-22T07:14:49.451783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:02.957610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.505096+0000","last_change":"2024-04-22T07:12:54.143822+0000","last_active":"2024-04-22T07:14:53.505096+0000","last_peered":"2024-04-22T07:14:53.505096+0000","last_clean":"2024-04-22T07:14:53.505096+0000","last_became_active":"2024-04-22T07:12:54.143501+0000","last_became_peered":"2024-04-22T07:12:54.143501+0000","last_unstale":"2024-04-22T07:14:53.505096+0000","last_undegraded":"2024-04-22T07:14:53.505096+0000","last_fullsized":"2024-04-22T07:14:53.505096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:29.707458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446021+0000","last_change":"2024-04-22T07:12:55.171406+0000","last_active":"2024-04-22T07:14:49.446021+0000","last_peered":"2024-04-22T07:14:49.446021+0000","last_clean":"2024-04-22T07:14:49.446021+0000","last_became_active":"2024-04-22T07:12:55.171064+0000","last_became_peered":"2024-04-22T07:12:55.171064+0000","last_unstale":"2024-04-22T07:14:49.446021+0000","last_undegraded":"2024-04-22T07:14:49.446021+0000","last_fullsized":"2024-04-22T07:14:49.446021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:06:05.609460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454363+0000","last_change":"2024-04-22T07:12:54.137288+0000","last_active":"2024-04-22T07:14:49.454363+0000","last_peered":"2024-04-22T07:14:49.454363+0000","last_clean":"2024-04-22T07:14:49.454363+0000","last_became_active":"2024-04-22T07:12:54.136914+0000","last_became_peered":"2024-04-22T07:12:54.136914+0000","last_unstale":"2024-04-22T07:14:49.454363+0000","last_undegraded":"2024-04-22T07:14:49.454363+0000","last_fullsized":"2024-04-22T07:14:49.454363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:40:44.989714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451662+0000","last_change":"2024-04-22T07:12:55.167298+0000","last_active":"2024-04-22T07:14:49.451662+0000","last_peered":"2024-04-22T07:14:49.451662+0000","last_clean":"2024-04-22T07:14:49.451662+0000","last_became_active":"2024-04-22T07:12:55.166634+0000","last_became_peered":"2024-04-22T07:12:55.166634+0000","last_unstale":"2024-04-22T07:14:49.451662+0000","last_undegraded":"2024-04-22T07:14:49.451662+0000","last_fullsized":"2024-04-22T07:14:49.451662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:37:12.259287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520117+0000","last_change":"2024-04-22T07:12:54.140597+0000","last_active":"2024-04-22T07:14:49.520117+0000","last_peered":"2024-04-22T07:14:49.520117+0000","last_clean":"2024-04-22T07:14:49.520117+0000","last_became_active":"2024-04-22T07:12:54.140260+0000","last_became_peered":"2024-04-22T07:12:54.140260+0000","last_unstale":"2024-04-22T07:14:49.520117+0000","last_undegraded":"2024-04-22T07:14:49.520117+0000","last_fullsized":"2024-04-22T07:14:49.520117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:57:43.549889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446512+0000","last_change":"2024-04-22T07:12:55.165712+0000","last_active":"2024-04-22T07:14:49.446512+0000","last_peered":"2024-04-22T07:14:49.446512+0000","last_clean":"2024-04-22T07:14:49.446512+0000","last_became_active":"2024-04-22T07:12:55.165400+0000","last_became_peered":"2024-04-22T07:12:55.165400+0000","last_unstale":"2024-04-22T07:14:49.446512+0000","last_undegraded":"2024-04-22T07:14:49.446512+0000","last_fullsized":"2024-04-22T07:14:49.446512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:03.493412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451463+0000","last_change":"2024-04-22T07:12:54.144199+0000","last_active":"2024-04-22T07:14:49.451463+0000","last_peered":"2024-04-22T07:14:49.451463+0000","last_clean":"2024-04-22T07:14:49.451463+0000","last_became_active":"2024-04-22T07:12:54.143649+0000","last_became_peered":"2024-04-22T07:12:54.143649+0000","last_unstale":"2024-04-22T07:14:49.451463+0000","last_undegraded":"2024-04-22T07:14:49.451463+0000","last_fullsized":"2024-04-22T07:14:49.451463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:05:48.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354021+0000","last_change":"2024-04-22T07:12:55.169672+0000","last_active":"2024-04-22T07:14:49.354021+0000","last_peered":"2024-04-22T07:14:49.354021+0000","last_clean":"2024-04-22T07:14:49.354021+0000","last_became_active":"2024-04-22T07:12:55.169330+0000","last_became_peered":"2024-04-22T07:12:55.169330+0000","last_unstale":"2024-04-22T07:14:49.354021+0000","last_undegraded":"2024-04-22T07:14:49.354021+0000","last_fullsized":"2024-04-22T07:14:49.354021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:59.589439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.504677+0000","last_change":"2024-04-22T07:12:54.140489+0000","last_active":"2024-04-22T07:14:53.504677+0000","last_peered":"2024-04-22T07:14:53.504677+0000","last_clean":"2024-04-22T07:14:53.504677+0000","last_became_active":"2024-04-22T07:12:54.140145+0000","last_became_peered":"2024-04-22T07:12:54.140145+0000","last_unstale":"2024-04-22T07:14:53.504677+0000","last_undegraded":"2024-04-22T07:14:53.504677+0000","last_fullsized":"2024-04-22T07:14:53.504677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:46:40.003394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518517+0000","last_change":"2024-04-22T07:12:55.170152+0000","last_active":"2024-04-22T07:14:49.518517+0000","last_peered":"2024-04-22T07:14:49.518517+0000","last_clean":"2024-04-22T07:14:49.518517+0000","last_became_active":"2024-04-22T07:12:55.169811+0000","last_became_peered":"2024-04-22T07:12:55.169811+0000","last_unstale":"2024-04-22T07:14:49.518517+0000","last_undegraded":"2024-04-22T07:14:49.518517+0000","last_fullsized":"2024-04-22T07:14:49.518517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:06:38.509746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353921+0000","last_change":"2024-04-22T07:12:54.145914+0000","last_active":"2024-04-22T07:14:49.353921+0000","last_peered":"2024-04-22T07:14:49.353921+0000","last_clean":"2024-04-22T07:14:49.353921+0000","last_became_active":"2024-04-22T07:12:54.145608+0000","last_became_peered":"2024-04-22T07:12:54.145608+0000","last_unstale":"2024-04-22T07:14:49.353921+0000","last_undegraded":"2024-04-22T07:14:49.353921+0000","last_fullsized":"2024-04-22T07:14:49.353921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:59:16.845547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.567045+0000","last_change":"2024-04-22T07:12:55.171483+0000","last_active":"2024-04-22T07:14:49.567045+0000","last_peered":"2024-04-22T07:14:49.567045+0000","last_clean":"2024-04-22T07:14:49.567045+0000","last_became_active":"2024-04-22T07:12:55.171172+0000","last_became_peered":"2024-04-22T07:12:55.171172+0000","last_unstale":"2024-04-22T07:14:49.567045+0000","last_undegraded":"2024-04-22T07:14:49.567045+0000","last_fullsized":"2024-04-22T07:14:49.567045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:38:09.811697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451908+0000","last_change":"2024-04-22T07:12:54.137528+0000","last_active":"2024-04-22T07:14:49.451908+0000","last_peered":"2024-04-22T07:14:49.451908+0000","last_clean":"2024-04-22T07:14:49.451908+0000","last_became_active":"2024-04-22T07:12:54.137239+0000","last_became_peered":"2024-04-22T07:12:54.137239+0000","last_unstale":"2024-04-22T07:14:49.451908+0000","last_undegraded":"2024-04-22T07:14:49.451908+0000","last_fullsized":"2024-04-22T07:14:49.451908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:01:13.599284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518495+0000","last_change":"2024-04-22T07:12:55.169736+0000","last_active":"2024-04-22T07:14:49.518495+0000","last_peered":"2024-04-22T07:14:49.518495+0000","last_clean":"2024-04-22T07:14:49.518495+0000","last_became_active":"2024-04-22T07:12:55.168343+0000","last_became_peered":"2024-04-22T07:12:55.168343+0000","last_unstale":"2024-04-22T07:14:49.518495+0000","last_undegraded":"2024-04-22T07:14:49.518495+0000","last_fullsized":"2024-04-22T07:14:49.518495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:41:17.745337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:54.505863+0000","last_change":"2024-04-22T07:12:54.141473+0000","last_active":"2024-04-22T07:14:54.505863+0000","last_peered":"2024-04-22T07:14:54.505863+0000","last_clean":"2024-04-22T07:14:54.505863+0000","last_became_active":"2024-04-22T07:12:54.141130+0000","last_became_peered":"2024-04-22T07:12:54.141130+0000","last_unstale":"2024-04-22T07:14:54.505863+0000","last_undegraded":"2024-04-22T07:14:54.505863+0000","last_fullsized":"2024-04-22T07:14:54.505863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:52:18.764408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70669,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":189,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446273+0000","last_change":"2024-04-22T07:12:55.169558+0000","last_active":"2024-04-22T07:14:49.446273+0000","last_peered":"2024-04-22T07:14:49.446273+0000","last_clean":"2024-04-22T07:14:49.446273+0000","last_became_active":"2024-04-22T07:12:55.169230+0000","last_became_peered":"2024-04-22T07:12:55.169230+0000","last_unstale":"2024-04-22T07:14:49.446273+0000","last_undegraded":"2024-04-22T07:14:49.446273+0000","last_fullsized":"2024-04-22T07:14:49.446273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:43:15.464440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679544+0000","last_change":"2024-04-22T07:13:56.198776+0000","last_active":"2024-04-22T07:14:49.679544+0000","last_peered":"2024-04-22T07:14:49.679544+0000","last_clean":"2024-04-22T07:14:49.679544+0000","last_became_active":"2024-04-22T07:13:56.191481+0000","last_became_peered":"2024-04-22T07:13:56.191481+0000","last_unstale":"2024-04-22T07:14:49.679544+0000","last_undegraded":"2024-04-22T07:14:49.679544+0000","last_fullsized":"2024-04-22T07:14:49.679544+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:16:06.710812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520133+0000","last_change":"2024-04-22T07:12:55.176583+0000","last_active":"2024-04-22T07:14:49.520133+0000","last_peered":"2024-04-22T07:14:49.520133+0000","last_clean":"2024-04-22T07:14:49.520133+0000","last_became_active":"2024-04-22T07:12:55.176211+0000","last_became_peered":"2024-04-22T07:12:55.176211+0000","last_unstale":"2024-04-22T07:14:49.520133+0000","last_undegraded":"2024-04-22T07:14:49.520133+0000","last_fullsized":"2024-04-22T07:14:49.520133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:27:02.346738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451635+0000","last_change":"2024-04-22T07:12:54.143013+0000","last_active":"2024-04-22T07:14:49.451635+0000","last_peered":"2024-04-22T07:14:49.451635+0000","last_clean":"2024-04-22T07:14:49.451635+0000","last_became_active":"2024-04-22T07:12:54.142706+0000","last_became_peered":"2024-04-22T07:12:54.142706+0000","last_unstale":"2024-04-22T07:14:49.451635+0000","last_undegraded":"2024-04-22T07:14:49.451635+0000","last_fullsized":"2024-04-22T07:14:49.451635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:18:36.664074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353878+0000","last_change":"2024-04-22T07:12:55.164476+0000","last_active":"2024-04-22T07:14:49.353878+0000","last_peered":"2024-04-22T07:14:49.353878+0000","last_clean":"2024-04-22T07:14:49.353878+0000","last_became_active":"2024-04-22T07:12:55.164133+0000","last_became_peered":"2024-04-22T07:12:55.164133+0000","last_unstale":"2024-04-22T07:14:49.353878+0000","last_undegraded":"2024-04-22T07:14:49.353878+0000","last_fullsized":"2024-04-22T07:14:49.353878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:53:00.522911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:53.528806+0000","last_change":"2024-04-22T07:12:54.143446+0000","last_active":"2024-04-22T07:14:53.528806+0000","last_peered":"2024-04-22T07:14:53.528806+0000","last_clean":"2024-04-22T07:14:53.528806+0000","last_became_active":"2024-04-22T07:12:54.143086+0000","last_became_peered":"2024-04-22T07:12:54.143086+0000","last_unstale":"2024-04-22T07:14:53.528806+0000","last_undegraded":"2024-04-22T07:14:53.528806+0000","last_fullsized":"2024-04-22T07:14:53.528806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:00:59.586464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353610+0000","last_change":"2024-04-22T07:12:55.171659+0000","last_active":"2024-04-22T07:14:49.353610+0000","last_peered":"2024-04-22T07:14:49.353610+0000","last_clean":"2024-04-22T07:14:49.353610+0000","last_became_active":"2024-04-22T07:12:55.170813+0000","last_became_peered":"2024-04-22T07:12:55.170813+0000","last_unstale":"2024-04-22T07:14:49.353610+0000","last_undegraded":"2024-04-22T07:14:49.353610+0000","last_fullsized":"2024-04-22T07:14:49.353610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:44:44.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446009+0000","last_change":"2024-04-22T07:12:54.149036+0000","last_active":"2024-04-22T07:14:49.446009+0000","last_peered":"2024-04-22T07:14:49.446009+0000","last_clean":"2024-04-22T07:14:49.446009+0000","last_became_active":"2024-04-22T07:12:54.148733+0000","last_became_peered":"2024-04-22T07:12:54.148733+0000","last_unstale":"2024-04-22T07:14:49.446009+0000","last_undegraded":"2024-04-22T07:14:49.446009+0000","last_fullsized":"2024-04-22T07:14:49.446009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:01:52.680228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451558+0000","last_change":"2024-04-22T07:12:55.168146+0000","last_active":"2024-04-22T07:14:49.451558+0000","last_peered":"2024-04-22T07:14:49.451558+0000","last_clean":"2024-04-22T07:14:49.451558+0000","last_became_active":"2024-04-22T07:12:55.167708+0000","last_became_peered":"2024-04-22T07:12:55.167708+0000","last_unstale":"2024-04-22T07:14:49.451558+0000","last_undegraded":"2024-04-22T07:14:49.451558+0000","last_fullsized":"2024-04-22T07:14:49.451558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:50:48.364736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446380+0000","last_change":"2024-04-22T07:12:54.140649+0000","last_active":"2024-04-22T07:14:49.446380+0000","last_peered":"2024-04-22T07:14:49.446380+0000","last_clean":"2024-04-22T07:14:49.446380+0000","last_became_active":"2024-04-22T07:12:54.140300+0000","last_became_peered":"2024-04-22T07:12:54.140300+0000","last_unstale":"2024-04-22T07:14:49.446380+0000","last_undegraded":"2024-04-22T07:14:49.446380+0000","last_fullsized":"2024-04-22T07:14:49.446380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:13:07.413959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679730+0000","last_change":"2024-04-22T07:12:55.174316+0000","last_active":"2024-04-22T07:14:49.679730+0000","last_peered":"2024-04-22T07:14:49.679730+0000","last_clean":"2024-04-22T07:14:49.679730+0000","last_became_active":"2024-04-22T07:12:55.173709+0000","last_became_peered":"2024-04-22T07:12:55.173709+0000","last_unstale":"2024-04-22T07:14:49.679730+0000","last_undegraded":"2024-04-22T07:14:49.679730+0000","last_fullsized":"2024-04-22T07:14:49.679730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:56:26.825039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354097+0000","last_change":"2024-04-22T07:12:54.145483+0000","last_active":"2024-04-22T07:14:49.354097+0000","last_peered":"2024-04-22T07:14:49.354097+0000","last_clean":"2024-04-22T07:14:49.354097+0000","last_became_active":"2024-04-22T07:12:54.145131+0000","last_became_peered":"2024-04-22T07:12:54.145131+0000","last_unstale":"2024-04-22T07:14:49.354097+0000","last_undegraded":"2024-04-22T07:14:49.354097+0000","last_fullsized":"2024-04-22T07:14:49.354097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.994939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454763+0000","last_change":"2024-04-22T07:12:55.173058+0000","last_active":"2024-04-22T07:14:49.454763+0000","last_peered":"2024-04-22T07:14:49.454763+0000","last_clean":"2024-04-22T07:14:49.454763+0000","last_became_active":"2024-04-22T07:12:55.172664+0000","last_became_peered":"2024-04-22T07:12:55.172664+0000","last_unstale":"2024-04-22T07:14:49.454763+0000","last_undegraded":"2024-04-22T07:14:49.454763+0000","last_fullsized":"2024-04-22T07:14:49.454763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:20:52.014897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353758+0000","last_change":"2024-04-22T07:12:54.146558+0000","last_active":"2024-04-22T07:14:49.353758+0000","last_peered":"2024-04-22T07:14:49.353758+0000","last_clean":"2024-04-22T07:14:49.353758+0000","last_became_active":"2024-04-22T07:12:54.146260+0000","last_became_peered":"2024-04-22T07:12:54.146260+0000","last_unstale":"2024-04-22T07:14:49.353758+0000","last_undegraded":"2024-04-22T07:14:49.353758+0000","last_fullsized":"2024-04-22T07:14:49.353758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:54:44.893626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.454649+0000","last_change":"2024-04-22T07:13:56.428324+0000","last_active":"2024-04-22T07:14:49.454649+0000","last_peered":"2024-04-22T07:14:49.454649+0000","last_clean":"2024-04-22T07:14:49.454649+0000","last_became_active":"2024-04-22T07:13:56.427950+0000","last_became_peered":"2024-04-22T07:13:56.427950+0000","last_unstale":"2024-04-22T07:14:49.454649+0000","last_undegraded":"2024-04-22T07:14:49.454649+0000","last_fullsized":"2024-04-22T07:14:49.454649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:09:58.425653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451592+0000","last_change":"2024-04-22T07:13:56.194115+0000","last_active":"2024-04-22T07:14:49.451592+0000","last_peered":"2024-04-22T07:14:49.451592+0000","last_clean":"2024-04-22T07:14:49.451592+0000","last_became_active":"2024-04-22T07:13:56.193728+0000","last_became_peered":"2024-04-22T07:13:56.193728+0000","last_unstale":"2024-04-22T07:14:49.451592+0000","last_undegraded":"2024-04-22T07:14:49.451592+0000","last_fullsized":"2024-04-22T07:14:49.451592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:01:11.371312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:54.529574+0000","last_change":"2024-04-22T07:12:54.137376+0000","last_active":"2024-04-22T07:14:54.529574+0000","last_peered":"2024-04-22T07:14:54.529574+0000","last_clean":"2024-04-22T07:14:54.529574+0000","last_became_active":"2024-04-22T07:12:54.137068+0000","last_became_peered":"2024-04-22T07:12:54.137068+0000","last_unstale":"2024-04-22T07:14:54.529574+0000","last_undegraded":"2024-04-22T07:14:54.529574+0000","last_fullsized":"2024-04-22T07:14:54.529574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.297644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446669+0000","last_change":"2024-04-22T07:12:55.172899+0000","last_active":"2024-04-22T07:14:49.446669+0000","last_peered":"2024-04-22T07:14:49.446669+0000","last_clean":"2024-04-22T07:14:49.446669+0000","last_became_active":"2024-04-22T07:12:55.172610+0000","last_became_peered":"2024-04-22T07:12:55.172610+0000","last_unstale":"2024-04-22T07:14:49.446669+0000","last_undegraded":"2024-04-22T07:14:49.446669+0000","last_fullsized":"2024-04-22T07:14:49.446669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:24:25.589813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.521956+0000","last_change":"2024-04-22T07:12:54.139217+0000","last_active":"2024-04-22T07:14:49.521956+0000","last_peered":"2024-04-22T07:14:49.521956+0000","last_clean":"2024-04-22T07:14:49.521956+0000","last_became_active":"2024-04-22T07:12:54.138895+0000","last_became_peered":"2024-04-22T07:12:54.138895+0000","last_unstale":"2024-04-22T07:14:49.521956+0000","last_undegraded":"2024-04-22T07:14:49.521956+0000","last_fullsized":"2024-04-22T07:14:49.521956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:24:22.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":399,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:51.458109+0000","last_change":"2024-04-22T07:11:41.326663+0000","last_active":"2024-04-22T07:14:51.458109+0000","last_peered":"2024-04-22T07:14:51.458109+0000","last_clean":"2024-04-22T07:14:51.458109+0000","last_became_active":"2024-04-22T07:11:41.326077+0000","last_became_peered":"2024-04-22T07:11:41.326077+0000","last_unstale":"2024-04-22T07:14:51.458109+0000","last_undegraded":"2024-04-22T07:14:51.458109+0000","last_fullsized":"2024-04-22T07:14:51.458109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:08:19.403887+0000","last_clean_scrub_stamp":"2024-04-22T07:08:19.403887+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:45:23.866797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.519969+0000","last_change":"2024-04-22T07:12:55.171454+0000","last_active":"2024-04-22T07:14:49.519969+0000","last_peered":"2024-04-22T07:14:49.519969+0000","last_clean":"2024-04-22T07:14:49.519969+0000","last_became_active":"2024-04-22T07:12:55.171000+0000","last_became_peered":"2024-04-22T07:12:55.171000+0000","last_unstale":"2024-04-22T07:14:49.519969+0000","last_undegraded":"2024-04-22T07:14:49.519969+0000","last_fullsized":"2024-04-22T07:14:49.519969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:56:43.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353859+0000","last_change":"2024-04-22T07:12:54.141675+0000","last_active":"2024-04-22T07:14:49.353859+0000","last_peered":"2024-04-22T07:14:49.353859+0000","last_clean":"2024-04-22T07:14:49.353859+0000","last_became_active":"2024-04-22T07:12:54.140812+0000","last_became_peered":"2024-04-22T07:12:54.140812+0000","last_unstale":"2024-04-22T07:14:49.353859+0000","last_undegraded":"2024-04-22T07:14:49.353859+0000","last_fullsized":"2024-04-22T07:14:49.353859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:44:53.965640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446310+0000","last_change":"2024-04-22T07:12:54.149038+0000","last_active":"2024-04-22T07:14:49.446310+0000","last_peered":"2024-04-22T07:14:49.446310+0000","last_clean":"2024-04-22T07:14:49.446310+0000","last_became_active":"2024-04-22T07:12:54.148712+0000","last_became_peered":"2024-04-22T07:12:54.148712+0000","last_unstale":"2024-04-22T07:14:49.446310+0000","last_undegraded":"2024-04-22T07:14:49.446310+0000","last_fullsized":"2024-04-22T07:14:49.446310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:51:01.556389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520318+0000","last_change":"2024-04-22T07:12:55.162044+0000","last_active":"2024-04-22T07:14:49.520318+0000","last_peered":"2024-04-22T07:14:49.520318+0000","last_clean":"2024-04-22T07:14:49.520318+0000","last_became_active":"2024-04-22T07:12:55.160958+0000","last_became_peered":"2024-04-22T07:12:55.160958+0000","last_unstale":"2024-04-22T07:14:49.520318+0000","last_undegraded":"2024-04-22T07:14:49.520318+0000","last_fullsized":"2024-04-22T07:14:49.520318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:17:12.476754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'46","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:50.178165+0000","last_change":"2024-04-22T07:12:54.138698+0000","last_active":"2024-04-22T07:14:50.178165+0000","last_peered":"2024-04-22T07:14:50.178165+0000","last_clean":"2024-04-22T07:14:50.178165+0000","last_became_active":"2024-04-22T07:12:54.138388+0000","last_became_peered":"2024-04-22T07:12:54.138388+0000","last_unstale":"2024-04-22T07:14:50.178165+0000","last_undegraded":"2024-04-22T07:14:50.178165+0000","last_fullsized":"2024-04-22T07:14:50.178165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:45.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446988+0000","last_change":"2024-04-22T07:12:55.167571+0000","last_active":"2024-04-22T07:14:49.446988+0000","last_peered":"2024-04-22T07:14:49.446988+0000","last_clean":"2024-04-22T07:14:49.446988+0000","last_became_active":"2024-04-22T07:12:55.167265+0000","last_became_peered":"2024-04-22T07:12:55.167265+0000","last_unstale":"2024-04-22T07:14:49.446988+0000","last_undegraded":"2024-04-22T07:14:49.446988+0000","last_fullsized":"2024-04-22T07:14:49.446988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:20:47.385796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.522148+0000","last_change":"2024-04-22T07:12:54.142356+0000","last_active":"2024-04-22T07:14:49.522148+0000","last_peered":"2024-04-22T07:14:49.522148+0000","last_clean":"2024-04-22T07:14:49.522148+0000","last_became_active":"2024-04-22T07:12:54.142024+0000","last_became_peered":"2024-04-22T07:12:54.142024+0000","last_unstale":"2024-04-22T07:14:49.522148+0000","last_undegraded":"2024-04-22T07:14:49.522148+0000","last_fullsized":"2024-04-22T07:14:49.522148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:43:48.095837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566877+0000","last_change":"2024-04-22T07:12:55.172786+0000","last_active":"2024-04-22T07:14:49.566877+0000","last_peered":"2024-04-22T07:14:49.566877+0000","last_clean":"2024-04-22T07:14:49.566877+0000","last_became_active":"2024-04-22T07:12:55.172445+0000","last_became_peered":"2024-04-22T07:12:55.172445+0000","last_unstale":"2024-04-22T07:14:49.566877+0000","last_undegraded":"2024-04-22T07:14:49.566877+0000","last_fullsized":"2024-04-22T07:14:49.566877+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:03:04.795331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518008+0000","last_change":"2024-04-22T07:12:54.145498+0000","last_active":"2024-04-22T07:14:49.518008+0000","last_peered":"2024-04-22T07:14:49.518008+0000","last_clean":"2024-04-22T07:14:49.518008+0000","last_became_active":"2024-04-22T07:12:54.145207+0000","last_became_peered":"2024-04-22T07:12:54.145207+0000","last_unstale":"2024-04-22T07:14:49.518008+0000","last_undegraded":"2024-04-22T07:14:49.518008+0000","last_fullsized":"2024-04-22T07:14:49.518008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:21:30.243866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446005+0000","last_change":"2024-04-22T07:12:55.167556+0000","last_active":"2024-04-22T07:14:49.446005+0000","last_peered":"2024-04-22T07:14:49.446005+0000","last_clean":"2024-04-22T07:14:49.446005+0000","last_became_active":"2024-04-22T07:12:55.167032+0000","last_became_peered":"2024-04-22T07:12:55.167032+0000","last_unstale":"2024-04-22T07:14:49.446005+0000","last_undegraded":"2024-04-22T07:14:49.446005+0000","last_fullsized":"2024-04-22T07:14:49.446005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:50:21.422820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446529+0000","last_change":"2024-04-22T07:12:54.143739+0000","last_active":"2024-04-22T07:14:49.446529+0000","last_peered":"2024-04-22T07:14:49.446529+0000","last_clean":"2024-04-22T07:14:49.446529+0000","last_became_active":"2024-04-22T07:12:54.143055+0000","last_became_peered":"2024-04-22T07:12:54.143055+0000","last_unstale":"2024-04-22T07:14:49.446529+0000","last_undegraded":"2024-04-22T07:14:49.446529+0000","last_fullsized":"2024-04-22T07:14:49.446529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:42:35.168859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679532+0000","last_change":"2024-04-22T07:12:55.169202+0000","last_active":"2024-04-22T07:14:49.679532+0000","last_peered":"2024-04-22T07:14:49.679532+0000","last_clean":"2024-04-22T07:14:49.679532+0000","last_became_active":"2024-04-22T07:12:55.168838+0000","last_became_peered":"2024-04-22T07:12:55.168838+0000","last_unstale":"2024-04-22T07:14:49.679532+0000","last_undegraded":"2024-04-22T07:14:49.679532+0000","last_fullsized":"2024-04-22T07:14:49.679532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:02:54.650997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355979+0000","last_change":"2024-04-22T07:12:54.142861+0000","last_active":"2024-04-22T07:14:49.355979+0000","last_peered":"2024-04-22T07:14:49.355979+0000","last_clean":"2024-04-22T07:14:49.355979+0000","last_became_active":"2024-04-22T07:12:54.142469+0000","last_became_peered":"2024-04-22T07:12:54.142469+0000","last_unstale":"2024-04-22T07:14:49.355979+0000","last_undegraded":"2024-04-22T07:14:49.355979+0000","last_fullsized":"2024-04-22T07:14:49.355979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:48:18.307261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446724+0000","last_change":"2024-04-22T07:12:55.166043+0000","last_active":"2024-04-22T07:14:49.446724+0000","last_peered":"2024-04-22T07:14:49.446724+0000","last_clean":"2024-04-22T07:14:49.446724+0000","last_became_active":"2024-04-22T07:12:55.165737+0000","last_became_peered":"2024-04-22T07:12:55.165737+0000","last_unstale":"2024-04-22T07:14:49.446724+0000","last_undegraded":"2024-04-22T07:14:49.446724+0000","last_fullsized":"2024-04-22T07:14:49.446724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:25:53.241507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446129+0000","last_change":"2024-04-22T07:13:56.431722+0000","last_active":"2024-04-22T07:14:49.446129+0000","last_peered":"2024-04-22T07:14:49.446129+0000","last_clean":"2024-04-22T07:14:49.446129+0000","last_became_active":"2024-04-22T07:13:56.428273+0000","last_became_peered":"2024-04-22T07:13:56.428273+0000","last_unstale":"2024-04-22T07:14:49.446129+0000","last_undegraded":"2024-04-22T07:14:49.446129+0000","last_fullsized":"2024-04-22T07:14:49.446129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:53.830673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353868+0000","last_change":"2024-04-22T07:12:55.163110+0000","last_active":"2024-04-22T07:14:49.353868+0000","last_peered":"2024-04-22T07:14:49.353868+0000","last_clean":"2024-04-22T07:14:49.353868+0000","last_became_active":"2024-04-22T07:12:55.162795+0000","last_became_peered":"2024-04-22T07:12:55.162795+0000","last_unstale":"2024-04-22T07:14:49.353868+0000","last_undegraded":"2024-04-22T07:14:49.353868+0000","last_fullsized":"2024-04-22T07:14:49.353868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:38:05.549107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356164+0000","last_change":"2024-04-22T07:12:54.144884+0000","last_active":"2024-04-22T07:14:49.356164+0000","last_peered":"2024-04-22T07:14:49.356164+0000","last_clean":"2024-04-22T07:14:49.356164+0000","last_became_active":"2024-04-22T07:12:54.144417+0000","last_became_peered":"2024-04-22T07:12:54.144417+0000","last_unstale":"2024-04-22T07:14:49.356164+0000","last_undegraded":"2024-04-22T07:14:49.356164+0000","last_fullsized":"2024-04-22T07:14:49.356164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:22:46.806637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354047+0000","last_change":"2024-04-22T07:12:55.167683+0000","last_active":"2024-04-22T07:14:49.354047+0000","last_peered":"2024-04-22T07:14:49.354047+0000","last_clean":"2024-04-22T07:14:49.354047+0000","last_became_active":"2024-04-22T07:12:55.167287+0000","last_became_peered":"2024-04-22T07:12:55.167287+0000","last_unstale":"2024-04-22T07:14:49.354047+0000","last_undegraded":"2024-04-22T07:14:49.354047+0000","last_fullsized":"2024-04-22T07:14:49.354047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:47:48.194371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":68,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.553901+0000","last_change":"2024-04-22T07:12:54.135829+0000","last_active":"2024-04-22T07:14:49.553901+0000","last_peered":"2024-04-22T07:14:49.553901+0000","last_clean":"2024-04-22T07:14:49.553901+0000","last_became_active":"2024-04-22T07:12:54.135343+0000","last_became_peered":"2024-04-22T07:12:54.135343+0000","last_unstale":"2024-04-22T07:14:49.553901+0000","last_undegraded":"2024-04-22T07:14:49.553901+0000","last_fullsized":"2024-04-22T07:14:49.553901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:31:09.684566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.353880+0000","last_change":"2024-04-22T07:12:55.171254+0000","last_active":"2024-04-22T07:14:49.353880+0000","last_peered":"2024-04-22T07:14:49.353880+0000","last_clean":"2024-04-22T07:14:49.353880+0000","last_became_active":"2024-04-22T07:12:55.170747+0000","last_became_peered":"2024-04-22T07:12:55.170747+0000","last_unstale":"2024-04-22T07:14:49.353880+0000","last_undegraded":"2024-04-22T07:14:49.353880+0000","last_fullsized":"2024-04-22T07:14:49.353880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:02:25.977306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520280+0000","last_change":"2024-04-22T07:12:54.134324+0000","last_active":"2024-04-22T07:14:49.520280+0000","last_peered":"2024-04-22T07:14:49.520280+0000","last_clean":"2024-04-22T07:14:49.520280+0000","last_became_active":"2024-04-22T07:12:54.133995+0000","last_became_peered":"2024-04-22T07:12:54.133995+0000","last_unstale":"2024-04-22T07:14:49.520280+0000","last_undegraded":"2024-04-22T07:14:49.520280+0000","last_fullsized":"2024-04-22T07:14:49.520280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:53.122675+0000","last_clean_scrub_stamp":"2024-04-22T07:12:53.122675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:58:04.214061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354174+0000","last_change":"2024-04-22T07:12:55.166754+0000","last_active":"2024-04-22T07:14:49.354174+0000","last_peered":"2024-04-22T07:14:49.354174+0000","last_clean":"2024-04-22T07:14:49.354174+0000","last_became_active":"2024-04-22T07:12:55.166298+0000","last_became_peered":"2024-04-22T07:12:55.166298+0000","last_unstale":"2024-04-22T07:14:49.354174+0000","last_undegraded":"2024-04-22T07:14:49.354174+0000","last_fullsized":"2024-04-22T07:14:49.354174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:52:30.808334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679633+0000","last_change":"2024-04-22T07:12:55.174410+0000","last_active":"2024-04-22T07:14:49.679633+0000","last_peered":"2024-04-22T07:14:49.679633+0000","last_clean":"2024-04-22T07:14:49.679633+0000","last_became_active":"2024-04-22T07:12:55.173966+0000","last_became_peered":"2024-04-22T07:12:55.173966+0000","last_unstale":"2024-04-22T07:14:49.679633+0000","last_undegraded":"2024-04-22T07:14:49.679633+0000","last_fullsized":"2024-04-22T07:14:49.679633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:04:51.957251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355972+0000","last_change":"2024-04-22T07:13:56.428079+0000","last_active":"2024-04-22T07:14:49.355972+0000","last_peered":"2024-04-22T07:14:49.355972+0000","last_clean":"2024-04-22T07:14:49.355972+0000","last_became_active":"2024-04-22T07:13:56.427721+0000","last_became_peered":"2024-04-22T07:13:56.427721+0000","last_unstale":"2024-04-22T07:14:49.355972+0000","last_undegraded":"2024-04-22T07:14:49.355972+0000","last_fullsized":"2024-04-22T07:14:49.355972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:02:04.664292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.355819+0000","last_change":"2024-04-22T07:12:55.165986+0000","last_active":"2024-04-22T07:14:49.355819+0000","last_peered":"2024-04-22T07:14:49.355819+0000","last_clean":"2024-04-22T07:14:49.355819+0000","last_became_active":"2024-04-22T07:12:55.165580+0000","last_became_peered":"2024-04-22T07:12:55.165580+0000","last_unstale":"2024-04-22T07:14:49.355819+0000","last_undegraded":"2024-04-22T07:14:49.355819+0000","last_fullsized":"2024-04-22T07:14:49.355819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:57:20.597451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.566665+0000","last_change":"2024-04-22T07:12:55.171852+0000","last_active":"2024-04-22T07:14:49.566665+0000","last_peered":"2024-04-22T07:14:49.566665+0000","last_clean":"2024-04-22T07:14:49.566665+0000","last_became_active":"2024-04-22T07:12:55.171538+0000","last_became_peered":"2024-04-22T07:12:55.171538+0000","last_unstale":"2024-04-22T07:14:49.566665+0000","last_undegraded":"2024-04-22T07:14:49.566665+0000","last_fullsized":"2024-04-22T07:14:49.566665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:50:10.703799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520295+0000","last_change":"2024-04-22T07:12:55.173307+0000","last_active":"2024-04-22T07:14:49.520295+0000","last_peered":"2024-04-22T07:14:49.520295+0000","last_clean":"2024-04-22T07:14:49.520295+0000","last_became_active":"2024-04-22T07:12:55.172948+0000","last_became_peered":"2024-04-22T07:12:55.172948+0000","last_unstale":"2024-04-22T07:14:49.520295+0000","last_undegraded":"2024-04-22T07:14:49.520295+0000","last_fullsized":"2024-04-22T07:14:49.520295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:02:20.043666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679687+0000","last_change":"2024-04-22T07:12:55.170568+0000","last_active":"2024-04-22T07:14:49.679687+0000","last_peered":"2024-04-22T07:14:49.679687+0000","last_clean":"2024-04-22T07:14:49.679687+0000","last_became_active":"2024-04-22T07:12:55.170172+0000","last_became_peered":"2024-04-22T07:12:55.170172+0000","last_unstale":"2024-04-22T07:14:49.679687+0000","last_undegraded":"2024-04-22T07:14:49.679687+0000","last_fullsized":"2024-04-22T07:14:49.679687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:20:28.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.356248+0000","last_change":"2024-04-22T07:12:55.160734+0000","last_active":"2024-04-22T07:14:49.356248+0000","last_peered":"2024-04-22T07:14:49.356248+0000","last_clean":"2024-04-22T07:14:49.356248+0000","last_became_active":"2024-04-22T07:12:55.160390+0000","last_became_peered":"2024-04-22T07:12:55.160390+0000","last_unstale":"2024-04-22T07:14:49.356248+0000","last_undegraded":"2024-04-22T07:14:49.356248+0000","last_fullsized":"2024-04-22T07:14:49.356248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:22:20.290954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.354138+0000","last_change":"2024-04-22T07:13:56.428258+0000","last_active":"2024-04-22T07:14:49.354138+0000","last_peered":"2024-04-22T07:14:49.354138+0000","last_clean":"2024-04-22T07:14:49.354138+0000","last_became_active":"2024-04-22T07:13:56.427880+0000","last_became_peered":"2024-04-22T07:13:56.427880+0000","last_unstale":"2024-04-22T07:14:49.354138+0000","last_undegraded":"2024-04-22T07:14:49.354138+0000","last_fullsized":"2024-04-22T07:14:49.354138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:38:08.985756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.451527+0000","last_change":"2024-04-22T07:12:55.167823+0000","last_active":"2024-04-22T07:14:49.451527+0000","last_peered":"2024-04-22T07:14:49.451527+0000","last_clean":"2024-04-22T07:14:49.451527+0000","last_became_active":"2024-04-22T07:12:55.167271+0000","last_became_peered":"2024-04-22T07:12:55.167271+0000","last_unstale":"2024-04-22T07:14:49.451527+0000","last_undegraded":"2024-04-22T07:14:49.451527+0000","last_fullsized":"2024-04-22T07:14:49.451527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:18:38.579028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.446375+0000","last_change":"2024-04-22T07:12:55.169973+0000","last_active":"2024-04-22T07:14:49.446375+0000","last_peered":"2024-04-22T07:14:49.446375+0000","last_clean":"2024-04-22T07:14:49.446375+0000","last_became_active":"2024-04-22T07:12:55.169170+0000","last_became_peered":"2024-04-22T07:12:55.169170+0000","last_unstale":"2024-04-22T07:14:49.446375+0000","last_undegraded":"2024-04-22T07:14:49.446375+0000","last_fullsized":"2024-04-22T07:14:49.446375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:25.596314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.518310+0000","last_change":"2024-04-22T07:12:55.172963+0000","last_active":"2024-04-22T07:14:49.518310+0000","last_peered":"2024-04-22T07:14:49.518310+0000","last_clean":"2024-04-22T07:14:49.518310+0000","last_became_active":"2024-04-22T07:12:55.172657+0000","last_became_peered":"2024-04-22T07:12:55.172657+0000","last_unstale":"2024-04-22T07:14:49.518310+0000","last_undegraded":"2024-04-22T07:14:49.518310+0000","last_fullsized":"2024-04-22T07:14:49.518310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:42:59.749728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.520315+0000","last_change":"2024-04-22T07:12:55.175769+0000","last_active":"2024-04-22T07:14:49.520315+0000","last_peered":"2024-04-22T07:14:49.520315+0000","last_clean":"2024-04-22T07:14:49.520315+0000","last_became_active":"2024-04-22T07:12:55.175461+0000","last_became_peered":"2024-04-22T07:12:55.175461+0000","last_unstale":"2024-04-22T07:14:49.520315+0000","last_undegraded":"2024-04-22T07:14:49.520315+0000","last_fullsized":"2024-04-22T07:14:49.520315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:58:53.836064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T07:14:49.679771+0000","last_change":"2024-04-22T07:12:55.174130+0000","last_active":"2024-04-22T07:14:49.679771+0000","last_peered":"2024-04-22T07:14:49.679771+0000","last_clean":"2024-04-22T07:14:49.679771+0000","last_became_active":"2024-04-22T07:12:55.173747+0000","last_became_peered":"2024-04-22T07:12:55.173747+0000","last_unstale":"2024-04-22T07:14:49.679771+0000","last_undegraded":"2024-04-22T07:14:49.679771+0000","last_fullsized":"2024-04-22T07:14:49.679771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:12:54.124380+0000","last_clean_scrub_stamp":"2024-04-22T07:12:54.124380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:54:09.894330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600967,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":826,"num_read_kb":523,"num_write":290,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1802901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28788,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716364,"statfs":{"total":95995035648,"available":95965556736,"internally_reserved":0,"allocated":1814528,"data_stored":1265022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36684,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708468,"statfs":{"total":95995035648,"available":95957471232,"internally_reserved":0,"allocated":1380352,"data_stored":863942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037812,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1376256,"data_stored":863280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1064057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":470115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":541159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":471386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":542983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":995242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":471334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1728512,"data_stored":1194559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":204629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:15:01.373 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 -- 172.21.15.113:0/1852874206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 msgr2=0x7f07b406ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1852874206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b406ab00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009900 tx=0x7f07b002bcd0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 -- 172.21.15.113:0/1852874206 shutdown_connections 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1852874206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b406b040 0x7f07b410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1852874206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b406ab00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1852874206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07b4108030 0x7f07b4108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 -- 172.21.15.113:0/1852874206 >> 172.21.15.113:0/1852874206 conn(0x7f07b4077e30 msgr2=0x7f07b4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 -- 172.21.15.113:0/1852874206 shutdown_connections 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.510+0000 7f07bbec7640 1 -- 172.21.15.113:0/1852874206 wait complete. 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 Processor -- start 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 -- start start 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b406b040 0x7f07b419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07b4108030 0x7f07b4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41122d0 con 0x7f07b406a680 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4112150 con 0x7f07b4108030 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07bbec7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4112450 con 0x7f07b406b040 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56672/0 (socket says 172.21.15.113:56672) 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 -- 172.21.15.113:0/1885199520 learned_addr learned my addr 172.21.15.113:0/1885199520 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 -- 172.21.15.113:0/1885199520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b406b040 msgr2=0x7f07b419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b406b040 0x7f07b419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07ba464640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07b4108030 0x7f07b4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 -- 172.21.15.113:0/1885199520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07b4108030 msgr2=0x7f07b4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07b4108030 0x7f07b4198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 -- 172.21.15.113:0/1885199520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b4198d90 con 0x7f07b406a680 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07b9c63640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b419e410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e9f0 tx=0x7f07a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.511+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400cd80 con 0x7f07b406a680 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.512+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b4199080 con 0x7f07b406a680 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.512+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f07a4010900 con 0x7f07b406a680 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.512+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40188d0 con 0x7f07b406a680 2024-04-22T07:15:01.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.512+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41a52d0 con 0x7f07b406a680 2024-04-22T07:15:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.513+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4069230 con 0x7f07b406a680 2024-04-22T07:15:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.514+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7f07a4010ab0 con 0x7f07b406a680 2024-04-22T07:15:01.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.514+0000 7f07aaffd640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f07a00a3e40 0x7f07a00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.514+0000 7f07b9462640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f07a00a3e40 0x7f07a00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.514+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f07a40d0f30 con 0x7f07b406a680 2024-04-22T07:15:01.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.515+0000 7f07b9462640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f07a00a3e40 0x7f07a00a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07b00096c0 tx=0x7f07b0000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:01.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.519+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07a409d3b0 con 0x7f07b406a680 2024-04-22T07:15:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:01 smithi133 ceph-mon[31198]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-22T07:15:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:01 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/484608611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:01 smithi154 ceph-mon[31862]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-22T07:15:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:01 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/484608611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.735+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b4199af0 con 0x7f07b406a680 2024-04-22T07:15:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.736+0000 7f07aaffd640 1 -- 172.21.15.113:0/1885199520 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f07a409ccb0 con 0x7f07b406a680 2024-04-22T07:15:01.734 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:01.734 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f07a00a3e40 msgr2=0x7f07a00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f07a00a3e40 0x7f07a00a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07b00096c0 tx=0x7f07b0000bc0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 msgr2=0x7f07b419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b419e410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e9f0 tx=0x7f07a400eeb0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 shutdown_connections 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f07a00a3e40 0x7f07a00a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07b4108030 0x7f07b4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b406b040 0x7f07b419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 --2- 172.21.15.113:0/1885199520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b406a680 0x7f07b419e410 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 >> 172.21.15.113:0/1885199520 conn(0x7f07b4077e30 msgr2=0x7f07b40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 shutdown_connections 2024-04-22T07:15:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.738+0000 7f07bbec7640 1 -- 172.21.15.113:0/1885199520 wait complete. 2024-04-22T07:15:01.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:15:01.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 -- 172.21.15.113:0/893237554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa348106160 msgr2=0x7fa34810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 --2- 172.21.15.113:0/893237554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa348106160 0x7fa34810dd30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa340009ca0 tx=0x7fa34002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 -- 172.21.15.113:0/893237554 shutdown_connections 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 --2- 172.21.15.113:0/893237554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa348106160 0x7fa34810dd30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 --2- 172.21.15.113:0/893237554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101a40 0x7fa348105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 --2- 172.21.15.113:0/893237554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa348101090 0x7fa348101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 -- 172.21.15.113:0/893237554 >> 172.21.15.113:0/893237554 conn(0x7fa3480fc910 msgr2=0x7fa3480fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 -- 172.21.15.113:0/893237554 shutdown_connections 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.847+0000 7fa34f575640 1 -- 172.21.15.113:0/893237554 wait complete. 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 Processor -- start 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 -- start start 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101090 0x7fa3481986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa348106160 0x7fa348198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 0x7fa34819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3481111e0 con 0x7fa34819cf40 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa348111060 con 0x7fa348106160 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34f575640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa348111360 con 0x7fa348101090 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 0x7fa34819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 0x7fa34819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56682/0 (socket says 172.21.15.113:56682) 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 -- 172.21.15.113:0/2118332613 learned_addr learned my addr 172.21.15.113:0/2118332613 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34cb10640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa348106160 0x7fa348198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34d311640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101090 0x7fa3481986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 -- 172.21.15.113:0/2118332613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101090 msgr2=0x7fa3481986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101090 0x7fa3481986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 -- 172.21.15.113:0/2118332613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa348106160 msgr2=0x7fa348198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa348106160 0x7fa348198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 -- 172.21.15.113:0/2118332613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34819da30 con 0x7fa34819cf40 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34d311640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101090 0x7fa3481986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.848+0000 7fa34db12640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 0x7fa34819d3f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa34002c700 tx=0x7fa340035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.849+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340039040 con 0x7fa34819cf40 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.849+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34819dcc0 con 0x7fa34819cf40 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.849+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa340044070 con 0x7fa34819cf40 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.849+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340004540 con 0x7fa34819cf40 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.849+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3481a5490 con 0x7fa34819cf40 2024-04-22T07:15:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.850+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa348069230 con 0x7fa34819cf40 2024-04-22T07:15:01.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.851+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 24) ==== 138874+0+0 (secure 0 0 0) 0x7fa3400046e0 con 0x7fa34819cf40 2024-04-22T07:15:01.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.851+0000 7fa33e7fc640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3340a3e40 0x7fa3340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:01.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.851+0000 7fa34d311640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3340a3e40 0x7fa3340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:01.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.851+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa3400f3810 con 0x7fa34819cf40 2024-04-22T07:15:01.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.852+0000 7fa34d311640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3340a3e40 0x7fa3340a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa34400baa0 tx=0x7fa344005d90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:01.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:01.855+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3400bfec0 con 0x7fa34819cf40 2024-04-22T07:15:02.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.034+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3481999d0 con 0x7fa34819cf40 2024-04-22T07:15:02.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.036+0000 7fa33e7fc640 1 -- 172.21.15.113:0/2118332613 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7fa3481999d0 con 0x7fa34819cf40 2024-04-22T07:15:02.034 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:02.034 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3340a3e40 msgr2=0x7fa3340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3340a3e40 0x7fa3340a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa34400baa0 tx=0x7fa344005d90 comp rx=0 tx=0).stop 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 msgr2=0x7fa34819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 0x7fa34819d3f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa34002c700 tx=0x7fa340035d80 comp rx=0 tx=0).stop 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 shutdown_connections 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3340a3e40 0x7fa3340a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa34819cf40 0x7fa34819d3f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa348106160 0x7fa348198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 --2- 172.21.15.113:0/2118332613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa348101090 0x7fa3481986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 >> 172.21.15.113:0/2118332613 conn(0x7fa3480fc910 msgr2=0x7fa34806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 shutdown_connections 2024-04-22T07:15:02.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.038+0000 7fa34f575640 1 -- 172.21.15.113:0/2118332613 wait complete. 2024-04-22T07:15:02.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:15:02.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:15:02.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2097677905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 msgr2=0x7fba58102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2097677905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58102570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fba48009900 tx=0x7fba4802bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2097677905 shutdown_connections 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2097677905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58107260 0x7fba5810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2097677905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58102b40 0x7fba58106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2097677905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58102570 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2097677905 >> 172.21.15.113:0/2097677905 conn(0x7fba580fd990 msgr2=0x7fba580ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2097677905 shutdown_connections 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.156+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2097677905 wait complete. 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 Processor -- start 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 -- start start 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58107260 0x7fba58074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58074cd0 0x7fba581a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba58112300 con 0x7fba58102190 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba58112180 con 0x7fba58074cd0 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5cd4b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba58112480 con 0x7fba58107260 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56706/0 (socket says 172.21.15.113:56706) 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba5659c640 1 -- 172.21.15.113:0/2103868367 learned_addr learned my addr 172.21.15.113:0/2103868367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba55d9b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58107260 0x7fba58074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.157+0000 7fba56d9d640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58074cd0 0x7fba581a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5659c640 1 -- 172.21.15.113:0/2103868367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58107260 msgr2=0x7fba58074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5659c640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58107260 0x7fba58074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5659c640 1 -- 172.21.15.113:0/2103868367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58074cd0 msgr2=0x7fba581a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5659c640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58074cd0 0x7fba581a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5659c640 1 -- 172.21.15.113:0/2103868367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba581a7130 con 0x7fba58102190 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5659c640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58074250 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fba48006010 tx=0x7fba4802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48041070 con 0x7fba58102190 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fba480052f0 con 0x7fba58102190 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48003db0 con 0x7fba58102190 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba581a73c0 con 0x7fba58102190 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.158+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba581a78c0 con 0x7fba58102190 2024-04-22T07:15:02.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.159+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba580691c0 con 0x7fba58102190 2024-04-22T07:15:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.160+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fba48044070 con 0x7fba58102190 2024-04-22T07:15:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.161+0000 7fba477fe640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fba400a3ed0 0x7fba400a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.161+0000 7fba55d9b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fba400a3ed0 0x7fba400a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.161+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fba480f28d0 con 0x7fba58102190 2024-04-22T07:15:02.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.161+0000 7fba55d9b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fba400a3ed0 0x7fba400a6390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba58075c90 tx=0x7fba4c00b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:02.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.165+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba480bee70 con 0x7fba58102190 2024-04-22T07:15:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:02 smithi113 ceph-mon[27933]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:02 smithi113 ceph-mon[27933]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1885199520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2118332613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:02.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.343+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba5810a490 con 0x7fba58102190 2024-04-22T07:15:02.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.343+0000 7fba477fe640 1 -- 172.21.15.113:0/2103868367 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fba480be770 con 0x7fba58102190 2024-04-22T07:15:02.341 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:15:02.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fba400a3ed0 msgr2=0x7fba400a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fba400a3ed0 0x7fba400a6390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fba58075c90 tx=0x7fba4c00b390 comp rx=0 tx=0).stop 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 msgr2=0x7fba58074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58074250 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fba48006010 tx=0x7fba4802c710 comp rx=0 tx=0).stop 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 shutdown_connections 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fba400a3ed0 0x7fba400a6390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba58074cd0 0x7fba581a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba58107260 0x7fba58074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 --2- 172.21.15.113:0/2103868367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba58102190 0x7fba58074250 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 >> 172.21.15.113:0/2103868367 conn(0x7fba580fd990 msgr2=0x7fba580ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 shutdown_connections 2024-04-22T07:15:02.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.346+0000 7fba5cd4b640 1 -- 172.21.15.113:0/2103868367 wait complete. 2024-04-22T07:15:02.354 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:15:02.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- 172.21.15.113:0/567538999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700101130 msgr2=0x7f9700101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- 172.21.15.113:0/567538999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700101130 0x7f9700101590 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009900 tx=0x7f96f402bc80 comp rx=0 tx=0).stop 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- 172.21.15.113:0/567538999 shutdown_connections 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- 172.21.15.113:0/567538999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9700101ad0 0x7f970010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- 172.21.15.113:0/567538999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700101130 0x7f9700101590 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- 172.21.15.113:0/567538999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700105ea0 0x7f9700106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- 172.21.15.113:0/567538999 >> 172.21.15.113:0/567538999 conn(0x7f9700077e30 msgr2=0x7f9700078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- 172.21.15.113:0/567538999 shutdown_connections 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- 172.21.15.113:0/567538999 wait complete. 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 Processor -- start 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- start start 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700101130 0x7f9700199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9700101ad0 0x7f9700199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 0x7f97001a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9700111350 con 0x7f9700105ea0 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97001111d0 con 0x7f9700101ad0 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.459+0000 7f9708552640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97001114d0 con 0x7f9700101130 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 0x7f97001a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 0x7f97001a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56710/0 (socket says 172.21.15.113:56710) 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 -- 172.21.15.113:0/2952301369 learned_addr learned my addr 172.21.15.113:0/2952301369 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9705aed640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9700101ad0 0x7f9700199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f97062ee640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700101130 0x7f9700199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 -- 172.21.15.113:0/2952301369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700101130 msgr2=0x7f9700199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700101130 0x7f9700199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 -- 172.21.15.113:0/2952301369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9700101ad0 msgr2=0x7f9700199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9700101ad0 0x7f9700199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 -- 172.21.15.113:0/2952301369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97001a4d30 con 0x7f9700105ea0 2024-04-22T07:15:02.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9706aef640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 0x7f97001a47f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f96f000d950 tx=0x7f96f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0017030 con 0x7f9700105ea0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f96f0004020 con 0x7f9700105ea0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001a4f00 con 0x7f9700105ea0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0003bc0 con 0x7f9700105ea0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.460+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a53d0 con 0x7f9700105ea0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.461+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97000691c0 con 0x7f9700105ea0 2024-04-22T07:15:02.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.462+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f96f001a070 con 0x7f9700105ea0 2024-04-22T07:15:02.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.463+0000 7f96ff7fe640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96e80a3ed0 0x7f96e80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.463+0000 7f97062ee640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96e80a3ed0 0x7f96e80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.463+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f96f00d0220 con 0x7f9700105ea0 2024-04-22T07:15:02.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.463+0000 7f97062ee640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96e80a3ed0 0x7f96e80a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005fd0 tx=0x7f96f8009200 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:02.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.467+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96f0021070 con 0x7f9700105ea0 2024-04-22T07:15:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:02 smithi133 ceph-mon[31198]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:02 smithi133 ceph-mon[31198]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1885199520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2118332613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:02 smithi154 ceph-mon[31862]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:02 smithi154 ceph-mon[31862]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1885199520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2118332613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:02.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.646+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97001093b0 con 0x7f9700105ea0 2024-04-22T07:15:02.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.646+0000 7f96ff7fe640 1 -- 172.21.15.113:0/2952301369 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f96f00bc020 con 0x7f9700105ea0 2024-04-22T07:15:02.645 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:15:02.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96e80a3ed0 msgr2=0x7f96e80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96e80a3ed0 0x7f96e80a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005fd0 tx=0x7f96f8009200 comp rx=0 tx=0).stop 2024-04-22T07:15:02.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 msgr2=0x7f97001a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 0x7f97001a47f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f96f000d950 tx=0x7f96f000de10 comp rx=0 tx=0).stop 2024-04-22T07:15:02.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 shutdown_connections 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f96e80a3ed0 0x7f96e80a6390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9700105ea0 0x7f97001a47f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9700101ad0 0x7f9700199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 --2- 172.21.15.113:0/2952301369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9700101130 0x7f9700199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 >> 172.21.15.113:0/2952301369 conn(0x7f9700077e30 msgr2=0x7f97000fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 shutdown_connections 2024-04-22T07:15:02.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.648+0000 7f9708552640 1 -- 172.21.15.113:0/2952301369 wait complete. 2024-04-22T07:15:02.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 -- 172.21.15.113:0/462991441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 msgr2=0x7fa9dc110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/462991441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc110c20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009ca0 tx=0x7fa9d402c180 comp rx=0 tx=0).stop 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 -- 172.21.15.113:0/462991441 shutdown_connections 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/462991441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc110c20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/462991441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc074b40 0x7fa9dc10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/462991441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9dc074220 0x7fa9dc074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 -- 172.21.15.113:0/462991441 >> 172.21.15.113:0/462991441 conn(0x7fa9dc0793a0 msgr2=0x7fa9dc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 -- 172.21.15.113:0/462991441 shutdown_connections 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.769+0000 7fa9e1d96640 1 -- 172.21.15.113:0/462991441 wait complete. 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 Processor -- start 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 -- start start 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9dc074220 0x7fa9dc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc074b40 0x7fa9dc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc114500 con 0x7fa9dc10e830 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc114380 con 0x7fa9dc074b40 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9e1d96640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc114680 con 0x7fa9dc074220 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56726/0 (socket says 172.21.15.113:56726) 2024-04-22T07:15:02.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 -- 172.21.15.113:0/2376426303 learned_addr learned my addr 172.21.15.113:0/2376426303 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9db7fe640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9dc074220 0x7fa9dc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9daffd640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc074b40 0x7fa9dc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 -- 172.21.15.113:0/2376426303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9dc074220 msgr2=0x7fa9dc1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9dc074220 0x7fa9dc1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 -- 172.21.15.113:0/2376426303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc074b40 msgr2=0x7fa9dc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc074b40 0x7fa9dc1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.770+0000 7fa9dbfff640 1 -- 172.21.15.113:0/2376426303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc19afb0 con 0x7fa9dc10e830 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.771+0000 7fa9dbfff640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc19a7e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402c790 tx=0x7fa9d4035e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.771+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4039040 con 0x7fa9dc10e830 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.771+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa9d4044070 con 0x7fa9dc10e830 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.771+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc19b240 con 0x7fa9dc10e830 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.771+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4032940 con 0x7fa9dc10e830 2024-04-22T07:15:02.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.771+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc1a74c0 con 0x7fa9dc10e830 2024-04-22T07:15:02.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.772+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc075960 con 0x7fa9dc10e830 2024-04-22T07:15:02.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.773+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fa9d4032ae0 con 0x7fa9dc10e830 2024-04-22T07:15:02.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.774+0000 7fa9d8ff9640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa9c80a3ed0 0x7fa9c80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:02.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.774+0000 7fa9db7fe640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa9c80a3ed0 0x7fa9c80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:02.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.774+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa9d40f2f10 con 0x7fa9dc10e830 2024-04-22T07:15:02.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.774+0000 7fa9db7fe640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa9c80a3ed0 0x7fa9c80a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005fd0 tx=0x7fa9cc00b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:02.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.778+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9d40bf4b0 con 0x7fa9dc10e830 2024-04-22T07:15:02.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.954+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9dc1099b0 con 0x7fa9dc10e830 2024-04-22T07:15:02.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.954+0000 7fa9d8ff9640 1 -- 172.21.15.113:0/2376426303 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fa9d40bedb0 con 0x7fa9dc10e830 2024-04-22T07:15:02.952 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:15:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa9c80a3ed0 msgr2=0x7fa9c80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa9c80a3ed0 0x7fa9c80a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc005fd0 tx=0x7fa9cc00b390 comp rx=0 tx=0).stop 2024-04-22T07:15:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 msgr2=0x7fa9dc19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:02.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc19a7e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9d402c790 tx=0x7fa9d4035e40 comp rx=0 tx=0).stop 2024-04-22T07:15:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 shutdown_connections 2024-04-22T07:15:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa9c80a3ed0 0x7fa9c80a6390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9dc10e830 0x7fa9dc19a7e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc074b40 0x7fa9dc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 --2- 172.21.15.113:0/2376426303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9dc074220 0x7fa9dc1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:02.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 >> 172.21.15.113:0/2376426303 conn(0x7fa9dc0793a0 msgr2=0x7fa9dc0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:02.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 shutdown_connections 2024-04-22T07:15:02.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:02.956+0000 7fa9e1d96640 1 -- 172.21.15.113:0/2376426303 wait complete. 2024-04-22T07:15:02.965 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:03.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 -- 172.21.15.113:0/605927358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 msgr2=0x7ff284069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/605927358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff284069560 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:03.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 -- 172.21.15.113:0/605927358 shutdown_connections 2024-04-22T07:15:03.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/605927358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff284069aa0 0x7ff28410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/605927358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff284069560 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/605927358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff284106f50 0x7ff284107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 -- 172.21.15.113:0/605927358 >> 172.21.15.113:0/605927358 conn(0x7ff284077e30 msgr2=0x7ff284078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 -- 172.21.15.113:0/605927358 shutdown_connections 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 -- 172.21.15.113:0/605927358 wait complete. 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.071+0000 7ff28a3b2640 1 Processor -- start 2024-04-22T07:15:03.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 -- start start 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff28419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff284069aa0 0x7ff28419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff284106f50 0x7ff2841985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841122d0 con 0x7ff2840690e0 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff284112150 con 0x7ff284106f50 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28a3b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff284112450 con 0x7ff284069aa0 2024-04-22T07:15:03.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff28419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff28419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56744/0 (socket says 172.21.15.113:56744) 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 -- 172.21.15.113:0/122760049 learned_addr learned my addr 172.21.15.113:0/122760049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff28894f640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff284106f50 0x7ff2841985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 -- 172.21.15.113:0/122760049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff284069aa0 msgr2=0x7ff28419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff284069aa0 0x7ff28419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 -- 172.21.15.113:0/122760049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff284106f50 msgr2=0x7ff2841985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff284106f50 0x7ff2841985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 -- 172.21.15.113:0/122760049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff284198d70 con 0x7ff2840690e0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.072+0000 7ff283fff640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff28419e3f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff27400d950 tx=0x7ff27400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.073+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274017070 con 0x7ff2840690e0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.073+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff274009c30 con 0x7ff2840690e0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.073+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27400f6b0 con 0x7ff2840690e0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.073+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff284199060 con 0x7ff2840690e0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.073+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841a52b0 con 0x7ff2840690e0 2024-04-22T07:15:03.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.074+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff284068de0 con 0x7ff2840690e0 2024-04-22T07:15:03.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.075+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7ff2740109c0 con 0x7ff2840690e0 2024-04-22T07:15:03.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.075+0000 7ff2817fa640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff26c0a3ed0 0x7ff26c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.075+0000 7ff2837fe640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff26c0a3ed0 0x7ff26c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.075+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff2740d0640 con 0x7ff2840690e0 2024-04-22T07:15:03.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.076+0000 7ff2837fe640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff26c0a3ed0 0x7ff26c0a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2780096c0 tx=0x7ff27802cf90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:03.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.080+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff274009de0 con 0x7ff2840690e0 2024-04-22T07:15:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.301+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff28402d5c0 con 0x7ff2840690e0 2024-04-22T07:15:03.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.302+0000 7ff2817fa640 1 -- 172.21.15.113:0/122760049 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7ff27409cad0 con 0x7ff2840690e0 2024-04-22T07:15:03.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:03.300 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:03.301 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff26c0a3ed0 msgr2=0x7ff26c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff26c0a3ed0 0x7ff26c0a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2780096c0 tx=0x7ff27802cf90 comp rx=0 tx=0).stop 2024-04-22T07:15:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 msgr2=0x7ff28419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff28419e3f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff27400d950 tx=0x7ff27400de10 comp rx=0 tx=0).stop 2024-04-22T07:15:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 shutdown_connections 2024-04-22T07:15:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff26c0a3ed0 0x7ff26c0a6390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff284106f50 0x7ff2841985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff284069aa0 0x7ff28419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 --2- 172.21.15.113:0/122760049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2840690e0 0x7ff28419e3f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 >> 172.21.15.113:0/122760049 conn(0x7ff284077e30 msgr2=0x7ff2840fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 shutdown_connections 2024-04-22T07:15:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.304+0000 7ff28a3b2640 1 -- 172.21.15.113:0/122760049 wait complete. 2024-04-22T07:15:03.312 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:03 smithi113 ceph-mon[27933]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:03.312 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:03 smithi113 ceph-mon[27933]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-04-22T07:15:03.313 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:03 smithi113 ceph-mon[27933]: mgrmap e25: x(active, since 12s), standbys: z, y 2024-04-22T07:15:03.313 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:03 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2103868367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.313 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:03 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2952301369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.313 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:03 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2376426303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 -- 172.21.15.113:0/3011157802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60101b60 msgr2=0x7fed6010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3011157802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60101b60 0x7fed6010dc90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fed58009cd0 tx=0x7fed5802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 -- 172.21.15.113:0/3011157802 shutdown_connections 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3011157802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60101b60 0x7fed6010dc90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3011157802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed601011a0 0x7fed60101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3011157802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed60105e30 0x7fed60106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 -- 172.21.15.113:0/3011157802 >> 172.21.15.113:0/3011157802 conn(0x7fed60077e30 msgr2=0x7fed60078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 -- 172.21.15.113:0/3011157802 shutdown_connections 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 -- 172.21.15.113:0/3011157802 wait complete. 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.415+0000 7fed65e9a640 1 Processor -- start 2024-04-22T07:15:03.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 -- start start 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed601011a0 0x7fed60199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed60101b60 0x7fed60199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 0x7fed601a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fed60111240 con 0x7fed60105e30 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed601110c0 con 0x7fed60101b60 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed65e9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed601113c0 con 0x7fed601011a0 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 0x7fed601a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 0x7fed601a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56762/0 (socket says 172.21.15.113:56762) 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 -- 172.21.15.113:0/3969923150 learned_addr learned my addr 172.21.15.113:0/3969923150 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5f7fe640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed601011a0 0x7fed60199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5effd640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed60101b60 0x7fed60199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 -- 172.21.15.113:0/3969923150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed601011a0 msgr2=0x7fed60199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed601011a0 0x7fed60199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 -- 172.21.15.113:0/3969923150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed60101b60 msgr2=0x7fed60199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed60101b60 0x7fed60199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 -- 172.21.15.113:0/3969923150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed601a4bc0 con 0x7fed60105e30 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5ffff640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 0x7fed601a4680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed58009ca0 tx=0x7fed58035d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.416+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58039040 con 0x7fed60105e30 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.417+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed601a4d90 con 0x7fed60105e30 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.417+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fed58044070 con 0x7fed60105e30 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.417+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58004440 con 0x7fed60105e30 2024-04-22T07:15:03.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.417+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed601a51f0 con 0x7fed60105e30 2024-04-22T07:15:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.418+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed60068de0 con 0x7fed60105e30 2024-04-22T07:15:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.418+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fed580045e0 con 0x7fed60105e30 2024-04-22T07:15:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.419+0000 7fed5cff9640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fed4c0a3ed0 0x7fed4c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.419+0000 7fed5f7fe640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fed4c0a3ed0 0x7fed4c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.419+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fed580f30c0 con 0x7fed60105e30 2024-04-22T07:15:03.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.419+0000 7fed5f7fe640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fed4c0a3ed0 0x7fed4c0a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed5000bb60 tx=0x7fed5000b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.424+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed580bf660 con 0x7fed60105e30 2024-04-22T07:15:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:03 smithi133 ceph-mon[31198]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:03 smithi133 ceph-mon[31198]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-04-22T07:15:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:03 smithi133 ceph-mon[31198]: mgrmap e25: x(active, since 12s), standbys: z, y 2024-04-22T07:15:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:03 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2103868367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:03 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2952301369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:03 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2376426303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:03 smithi154 ceph-mon[31862]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T07:15:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:03 smithi154 ceph-mon[31862]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-04-22T07:15:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:03 smithi154 ceph-mon[31862]: mgrmap e25: x(active, since 12s), standbys: z, y 2024-04-22T07:15:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:03 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2103868367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:03 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2952301369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:03 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2376426303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:15:03.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.603+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed60109330 con 0x7fed60105e30 2024-04-22T07:15:03.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.605+0000 7fed5cff9640 1 -- 172.21.15.113:0/3969923150 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23082 (secure 0 0 0) 0x7fed580bef60 con 0x7fed60105e30 2024-04-22T07:15:03.603 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:03.603 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:14:49.346745+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fed4c0a3ed0 msgr2=0x7fed4c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fed4c0a3ed0 0x7fed4c0a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed5000bb60 tx=0x7fed5000b390 comp rx=0 tx=0).stop 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 msgr2=0x7fed601a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 0x7fed601a4680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed58009ca0 tx=0x7fed58035d00 comp rx=0 tx=0).stop 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 shutdown_connections 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fed4c0a3ed0 0x7fed4c0a6390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed60105e30 0x7fed601a4680 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed60101b60 0x7fed60199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 --2- 172.21.15.113:0/3969923150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed601011a0 0x7fed60199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 >> 172.21.15.113:0/3969923150 conn(0x7fed60077e30 msgr2=0x7fed600feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:03.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 shutdown_connections 2024-04-22T07:15:03.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.607+0000 7fed65e9a640 1 -- 172.21.15.113:0/3969923150 wait complete. 2024-04-22T07:15:03.616 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 623871 2024-04-22T07:15:03.616 DEBUG:tasks.quiescer:Yielding 2024-04-22T07:15:03.616 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T07:15:03.632 INFO:tasks.workunit:Pulling workunits from ref 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:15:03.632 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T07:15:03.632 INFO:tasks.workunit:timeout=3h 2024-04-22T07:15:03.632 INFO:tasks.workunit:cleanup=True 2024-04-22T07:15:03.633 DEBUG:teuthology.orchestra.run.smithi113:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T07:15:03.633 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-22 07:13:00.725827744 +0000 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-22 07:13:00.725827744 +0000 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-22 07:14:47.287093995 +0000 2024-04-22T07:15:03.648 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-04-22 07:13:00.725827744 +0000 2024-04-22T07:15:03.649 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T07:15:03.649 DEBUG:teuthology.orchestra.run.smithi113:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T07:15:03.720 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 -- 172.21.15.113:0/2924246125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c100e80 msgr2=0x7f231c10ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 --2- 172.21.15.113:0/2924246125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c100e80 0x7f231c10ede0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f230c009930 tx=0x7f230c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 -- 172.21.15.113:0/2924246125 shutdown_connections 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 --2- 172.21.15.113:0/2924246125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c100e80 0x7f231c10ede0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 --2- 172.21.15.113:0/2924246125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c1004c0 0x7f231c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 --2- 172.21.15.113:0/2924246125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f231c106f50 0x7f231c107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 -- 172.21.15.113:0/2924246125 >> 172.21.15.113:0/2924246125 conn(0x7f231c077450 msgr2=0x7f231c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 -- 172.21.15.113:0/2924246125 shutdown_connections 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 -- 172.21.15.113:0/2924246125 wait complete. 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.737+0000 7f23218eb640 1 Processor -- start 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 -- start start 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 0x7f231c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f231c100e80 0x7f231c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f50 0x7f231c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1123e0 con 0x7f231c1004c0 2024-04-22T07:15:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f231c112260 con 0x7f231c106f50 2024-04-22T07:15:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f23218eb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f231c112560 con 0x7f231c100e80 2024-04-22T07:15:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 0x7f231c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 0x7f231c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56780/0 (socket says 172.21.15.113:56780) 2024-04-22T07:15:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 -- 172.21.15.113:0/1937350640 learned_addr learned my addr 172.21.15.113:0/1937350640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231b7fe640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f50 0x7f231c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231a7fc640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f231c100e80 0x7f231c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 -- 172.21.15.113:0/1937350640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f231c100e80 msgr2=0x7f231c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f231c100e80 0x7f231c19eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 -- 172.21.15.113:0/1937350640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f50 msgr2=0x7f231c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f50 0x7f231c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 -- 172.21.15.113:0/1937350640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c198f20 con 0x7f231c1004c0 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231b7fe640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f50 0x7f231c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.738+0000 7f231affd640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 0x7f231c19e5a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f231400cce0 tx=0x7f231400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:03.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.739+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231400f650 con 0x7f231c1004c0 2024-04-22T07:15:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.739+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c199210 con 0x7f231c1004c0 2024-04-22T07:15:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.739+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f231400fcc0 con 0x7f231c1004c0 2024-04-22T07:15:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.739+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314018c40 con 0x7f231c1004c0 2024-04-22T07:15:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.739+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1a5520 con 0x7f231c1004c0 2024-04-22T07:15:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.740+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231c1037e0 con 0x7f231c1004c0 2024-04-22T07:15:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.741+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f231400f7f0 con 0x7f231c1004c0 2024-04-22T07:15:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.742+0000 7f23208e9640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23040a3ed0 0x7f23040a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.742+0000 7f231a7fc640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23040a3ed0 0x7f23040a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.742+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f23140d1330 con 0x7f231c1004c0 2024-04-22T07:15:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.742+0000 7f231a7fc640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23040a3ed0 0x7f23040a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f231c199eb0 tx=0x7f2310005c60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:03.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.745+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2314014030 con 0x7f231c1004c0 2024-04-22T07:15:03.749 INFO:tasks.workunit.client.0.smithi113.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T07:15:03.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.969+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f231c10a4a0 con 0x7f231c1004c0 2024-04-22T07:15:03.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.969+0000 7f23208e9640 1 -- 172.21.15.113:0/1937350640 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f231400fae0 con 0x7f231c1004c0 2024-04-22T07:15:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:03.967 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:03.968 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:03.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23040a3ed0 msgr2=0x7f23040a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23040a3ed0 0x7f23040a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f231c199eb0 tx=0x7f2310005c60 comp rx=0 tx=0).stop 2024-04-22T07:15:03.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 msgr2=0x7f231c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:03.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 0x7f231c19e5a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f231400cce0 tx=0x7f231400a620 comp rx=0 tx=0).stop 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 shutdown_connections 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23040a3ed0 0x7f23040a6390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f50 0x7f231c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f231c100e80 0x7f231c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 --2- 172.21.15.113:0/1937350640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f231c1004c0 0x7f231c19e5a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 >> 172.21.15.113:0/1937350640 conn(0x7f231c077450 msgr2=0x7f231c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 shutdown_connections 2024-04-22T07:15:03.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:03.971+0000 7f23218eb640 1 -- 172.21.15.113:0/1937350640 wait complete. 2024-04-22T07:15:03.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T07:12:54.514841+0000', 'modified': '2024-04-22T07:14:47.974590+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24473, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6835/217490889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 217490889}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 217490889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/1615298521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1615298521}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1615298521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/3935683574', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3935683574}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3935683574}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.154:6839/3404626603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 3404626603}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 3404626603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6837/65907218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 65907218}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 65907218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6839/982201245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 982201245}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 982201245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24506, 24469, 24473]} 2024-04-22T07:15:03.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-22T07:15:03.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T07:15:04.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/3783293442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 msgr2=0x7ff1d810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/3783293442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d810eeb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009ca0 tx=0x7ff1d002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/3783293442 shutdown_connections 2024-04-22T07:15:04.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/3783293442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d810eeb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/3783293442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d8102bc0 0x7ff1d8106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/3783293442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d8102210 0x7ff1d81025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/3783293442 >> 172.21.15.113:0/3783293442 conn(0x7ff1d80fdab0 msgr2=0x7ff1d80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/3783293442 shutdown_connections 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.126+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/3783293442 wait complete. 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 Processor -- start 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 -- start start 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d8102210 0x7ff1d819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d81986e0 0x7ff1d8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8112380 con 0x7ff1d81072e0 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8112200 con 0x7ff1d81986e0 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1dd1b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8112500 con 0x7ff1d8102210 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56790/0 (socket says 172.21.15.113:56790) 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1d659c640 1 -- 172.21.15.113:0/1336361344 learned_addr learned my addr 172.21.15.113:0/1336361344 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1d759e640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d81986e0 0x7ff1d8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.127+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d8102210 0x7ff1d819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1d659c640 1 -- 172.21.15.113:0/1336361344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d8102210 msgr2=0x7ff1d819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d8102210 0x7ff1d819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1d659c640 1 -- 172.21.15.113:0/1336361344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d81986e0 msgr2=0x7ff1d8198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d81986e0 0x7ff1d8198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1d659c640 1 -- 172.21.15.113:0/1336361344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d81992e0 con 0x7ff1d81072e0 2024-04-22T07:15:04.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d819eb60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c970 tx=0x7ff1cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1bffff640 1 -- 172.21.15.113:0/1336361344 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc007bb0 con 0x7ff1d81072e0 2024-04-22T07:15:04.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d81a53c0 con 0x7ff1d81072e0 2024-04-22T07:15:04.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1bffff640 1 -- 172.21.15.113:0/1336361344 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff1cc007d50 con 0x7ff1d81072e0 2024-04-22T07:15:04.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1bffff640 1 -- 172.21.15.113:0/1336361344 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc018780 con 0x7ff1d81072e0 2024-04-22T07:15:04.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.128+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d81a57c0 con 0x7ff1d81072e0 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1bffff640 1 -- 172.21.15.113:0/1336361344 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7ff1cc018980 con 0x7ff1d81072e0 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 Processor -- start 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 -- start start 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb197b0 0x560b1bb19b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb22710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb25130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b1ba7db10 con 0x560b1bb22ce0 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b1ba1b240 con 0x560b1bb197b0 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.130+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b1ba79ad0 con 0x560b1bb1a160 2024-04-22T07:15:04.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb25130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1bffff640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1c00a3ed0 0x7ff1c00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb25130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56806/0 (socket says 172.21.15.113:56806) 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 -- 172.21.15.113:0/1546337736 learned_addr learned my addr 172.21.15.113:0/1546337736 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb22710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d759e640 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb197b0 0x560b1bb19b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d759e640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1c00a3ed0 0x7ff1c00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1bffff640 1 -- 172.21.15.113:0/1336361344 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff1cc0d09f0 con 0x7ff1d81072e0 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 -- 172.21.15.113:0/1546337736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 msgr2=0x560b1bb22710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb22710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 -- 172.21.15.113:0/1546337736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb197b0 msgr2=0x560b1bb19b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb197b0 0x560b1bb19b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 -- 172.21.15.113:0/1546337736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1b7902d0 con 0x560b1bb22ce0 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb25130 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0ce070 tx=0x7ff1cc00fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345214d2a0627550 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1d759e640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1c00a3ed0 0x7ff1c00a6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005e30 tx=0x7ff1d0005d80 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1bd7fa640 1 -- 172.21.15.113:0/1546337736 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc09c780 con 0x560b1bb22ce0 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1bd7fa640 1 -- 172.21.15.113:0/1546337736 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff1cc09ce80 con 0x560b1bb22ce0 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.131+0000 7ff1bd7fa640 1 -- 172.21.15.113:0/1546337736 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc00fa40 con 0x560b1bb22ce0 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1546337736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 msgr2=0x560b1bb25130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb25130 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0ce070 tx=0x7ff1cc00fde0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1546337736 shutdown_connections 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb25130 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb22710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1546337736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb197b0 0x560b1bb19b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1546337736 >> 172.21.15.113:0/1546337736 conn(0x560b1bb13a60 msgr2=0x560b1bb13e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1546337736 shutdown_connections 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1546337736 wait complete. 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.132+0000 7ff1dfd9e740 1 -- start start 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 Processor -- start 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 0x560b1bb1fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb25bf0 con 0x560b1bb197b0 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b1ba06190 con 0x560b1bb22ce0 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb25d70 con 0x560b1bb1a160 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1dfd9e740 10 client.0 _ll_register_callbacks cb 0x560b1ba018b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 0x560b1bb1fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 0x560b1bb1fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56808/0 (socket says 172.21.15.113:56808) 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 -- 172.21.15.113:0/4173378877 learned_addr learned my addr 172.21.15.113:0/4173378877 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d659c640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d759e640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 -- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 msgr2=0x560b1bb201c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb201c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 -- 172.21.15.113:0/4173378877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 msgr2=0x560b1bb20700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 -- 172.21.15.113:0/4173378877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1b85a6b0 con 0x560b1bb197b0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d659c640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 0x560b1bb1fc80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8012730 tx=0x7ff1c8012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1643a28f0be3bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1baff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1baff5640 1 -- 172.21.15.113:0/4173378877 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800f920 con 0x560b1bb197b0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.133+0000 7ff1baff5640 1 -- 172.21.15.113:0/4173378877 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff1c800fac0 con 0x560b1bb197b0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc09ce80 con 0x560b1bb197b0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1baff5640 1 -- 172.21.15.113:0/4173378877 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff1c800fcc0 con 0x560b1bb197b0 2024-04-22T07:15:04.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 10 client.15657 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560b1bb31f50 con 0x560b1bb197b0 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1baff5640 1 -- 172.21.15.113:0/4173378877 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff1cc09ce80 con 0x560b1bb197b0 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1baff5640 1 -- 172.21.15.113:0/4173378877 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff1c8010430 con 0x560b1bb197b0 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 10 client.15657 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 10 client.15657 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 20 client.15657 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 4 client.15657 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b1ba8b530 con 0x560b1bb345a0 2024-04-22T07:15:04.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.134+0000 7ff1d659c640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.135+0000 7ff1baff5640 10 client.15657 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.135+0000 7ff1d659c640 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb35a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a4ae4e66cc87ac0 server_cookie=d3eca1a55921d436 in_seq=0 out_seq=0 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1baff5640 1 -- 172.21.15.113:0/4173378877 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b1ba8b530 con 0x560b1bb345a0 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1baff5640 10 client.15657 handle_command_reply: tid=0 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 client.15657 shutdown 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 msgr2=0x560b1bb1fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 0x560b1bb1fc80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8012730 tx=0x7ff1c8012bf0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 shutdown_connections 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb35a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb1a160 0x560b1bb201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1baff5640 0 client.15657 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:04.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/4173378877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb197b0 0x560b1bb1fc80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 >> 172.21.15.113:0/4173378877 conn(0x560b1bb13a60 msgr2=0x560b1bb15bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 shutdown_connections 2024-04-22T07:15:04.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/4173378877 wait complete. 2024-04-22T07:15:04.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.136+0000 7ff1dfd9e740 20 client.15657 trim_cache size 0 max 16384 2024-04-22T07:15:04.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 Processor -- start 2024-04-22T07:15:04.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 -- start start 2024-04-22T07:15:04.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 0x560b1bb0e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb33d50 con 0x560b1bb1a160 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb34420 con 0x560b1bb22ce0 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb34af0 con 0x560b1bb197b0 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 0x560b1bb0e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1d659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 0x560b1bb0e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56824/0 (socket says 172.21.15.113:56824) 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1d659c640 1 -- 172.21.15.113:0/1246365848 learned_addr learned my addr 172.21.15.113:0/1246365848 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1d759e640 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.154+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d659c640 1 -- 172.21.15.113:0/1246365848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 msgr2=0x560b1bb223b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb223b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d659c640 1 -- 172.21.15.113:0/1246365848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 msgr2=0x560b1bb20520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d659c640 1 -- 172.21.15.113:0/1246365848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b1bb1b8a0 con 0x560b1bb1a160 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d759e640 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1d659c640 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 0x560b1bb0e800 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00afd0 tx=0x7ff1cc09c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89b54f8596483e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1bcff9640 1 -- 172.21.15.113:0/1246365848 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc09c780 con 0x560b1bb1a160 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1bcff9640 1 -- 172.21.15.113:0/1246365848 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff1cc0a53f0 con 0x560b1bb1a160 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.155+0000 7ff1bcff9640 1 -- 172.21.15.113:0/1246365848 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc018c70 con 0x560b1bb1a160 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1246365848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 msgr2=0x560b1bb0e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 0x560b1bb0e800 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00afd0 tx=0x7ff1cc09c600 comp rx=0 tx=0).stop 2024-04-22T07:15:04.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1246365848 shutdown_connections 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb22ce0 0x560b1bb20520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb1a160 0x560b1bb0e800 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/1246365848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1246365848 >> 172.21.15.113:0/1246365848 conn(0x560b1bb13a60 msgr2=0x560b1bb13e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1246365848 shutdown_connections 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/1246365848 wait complete. 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- start start 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 Processor -- start 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb3db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb1a160 0x560b1bb3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb36fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb35890 con 0x560b1bb22ce0 2024-04-22T07:15:04.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb351c0 con 0x560b1bb1a160 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560b1bb35f60 con 0x560b1bb197b0 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.156+0000 7ff1dfd9e740 10 client.0 _ll_register_callbacks cb 0x7ff1c8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb36fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb36fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56830/0 (socket says 172.21.15.113:56830) 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 -- 172.21.15.113:0/3376813460 learned_addr learned my addr 172.21.15.113:0/3376813460 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb3db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d659c640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb1a160 0x560b1bb3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 -- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 msgr2=0x560b1bb3db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb3db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 -- 172.21.15.113:0/3376813460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb1a160 msgr2=0x560b1bb3e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb1a160 0x560b1bb3e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 -- 172.21.15.113:0/3376813460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b1bb377e0 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d659c640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb1a160 0x560b1bb3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1d759e640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb36fb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009ca0 tx=0x7ff1d002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428939b774349ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1baff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1baff5640 1 -- 172.21.15.113:0/3376813460 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0056060 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc0a53f0 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1baff5640 1 -- 172.21.15.113:0/3376813460 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff1d0004510 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1baff5640 1 -- 172.21.15.113:0/3376813460 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff1d0039040 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1dfd9e740 10 client.15663 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.157+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560b1bb3c120 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1baff5640 1 -- 172.21.15.113:0/3376813460 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff1cc0a53f0 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1baff5640 1 -- 172.21.15.113:0/3376813460 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff1d0042420 con 0x560b1bb22ce0 2024-04-22T07:15:04.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1dfd9e740 10 client.15663 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1dfd9e740 10 client.15663 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1dfd9e740 20 client.15663 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1dfd9e740 4 client.15663 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "status"}) -- 0x560b1ba853a0 con 0x560b1bb345a0 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1d6d9d640 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb9d980 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e38e9073bb44e73 server_cookie=136ab43d3714ee8 in_seq=0 out_seq=0 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.158+0000 7ff1baff5640 10 client.15663 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1baff5640 1 -- 172.21.15.113:0/3376813460 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560b1ba853a0 con 0x560b1bb345a0 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1baff5640 10 client.15663 handle_command_reply: tid=0 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 client.15663 shutdown 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 msgr2=0x560b1bb36fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb36fb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009ca0 tx=0x7ff1d002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 shutdown_connections 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x560b1bb345a0 0x560b1bb9d980 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560b1bb22ce0 0x560b1bb36fb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560b1bb1a160 0x560b1bb3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1baff5640 0 client.15663 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 --2- 172.21.15.113:0/3376813460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560b1bb197b0 0x560b1bb3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 >> 172.21.15.113:0/3376813460 conn(0x560b1bb13a60 msgr2=0x560b1bb0e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 shutdown_connections 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 1 -- 172.21.15.113:0/3376813460 wait complete. 2024-04-22T07:15:04.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dfd9e740 20 client.15663 trim_cache size 0 max 16384 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 2, 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24469, 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 125.98481418999999, 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 22, 2024-04-22T07:15:04.167 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 89, 2024-04-22T07:15:04.168 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 88, 2024-04-22T07:15:04.168 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 142.75335136000001 2024-04-22T07:15:04.168 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:04.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1c00a3ed0 msgr2=0x7ff1c00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1c00a3ed0 0x7ff1c00a6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005e30 tx=0x7ff1d0005d80 comp rx=0 tx=0).stop 2024-04-22T07:15:04.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 msgr2=0x7ff1d819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.159+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d819eb60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c970 tx=0x7ff1cc00ce30 comp rx=0 tx=0).stop 2024-04-22T07:15:04.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 shutdown_connections 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1c00a3ed0 0x7ff1c00a6390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1d81986e0 0x7ff1d8198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d81072e0 0x7ff1d819eb60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 --2- 172.21.15.113:0/1336361344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1d8102210 0x7ff1d819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 >> 172.21.15.113:0/1336361344 conn(0x7ff1d80fdab0 msgr2=0x7ff1d80ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 shutdown_connections 2024-04-22T07:15:04.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.160+0000 7ff1dd1b2640 1 -- 172.21.15.113:0/1336361344 wait complete. 2024-04-22T07:15:04.170 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-22T07:15:04.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- 172.21.15.113:0/961275695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806af10 msgr2=0x7f5f0810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- 172.21.15.113:0/961275695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806af10 0x7f5f0810edf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009cd0 tx=0x7f5f0002c180 comp rx=0 tx=0).stop 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- 172.21.15.113:0/961275695 shutdown_connections 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- 172.21.15.113:0/961275695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806af10 0x7f5f0810edf0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- 172.21.15.113:0/961275695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806a550 0x7f5f0806a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- 172.21.15.113:0/961275695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f08108150 0x7f5f08108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- 172.21.15.113:0/961275695 >> 172.21.15.113:0/961275695 conn(0x7f5f08076c50 msgr2=0x7f5f08077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- 172.21.15.113:0/961275695 shutdown_connections 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- 172.21.15.113:0/961275695 wait complete. 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 Processor -- start 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- start start 2024-04-22T07:15:04.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 0x7f5f08198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806af10 0x7f5f08198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f08108150 0x7f5f0819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08112300 con 0x7f5f0806a550 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08112180 con 0x7f5f08108150 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.275+0000 7f5f0db15640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08112480 con 0x7f5f0806af10 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 0x7f5f08198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 0x7f5f08198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56850/0 (socket says 172.21.15.113:56850) 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 -- 172.21.15.113:0/3661669126 learned_addr learned my addr 172.21.15.113:0/3661669126 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5efffff640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806af10 0x7f5f08198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f07fff640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f08108150 0x7f5f0819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 -- 172.21.15.113:0/3661669126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806af10 msgr2=0x7f5f08198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806af10 0x7f5f08198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 -- 172.21.15.113:0/3661669126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f08108150 msgr2=0x7f5f0819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f08108150 0x7f5f0819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 -- 172.21.15.113:0/3661669126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0819d6c0 con 0x7f5f0806a550 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5efffff640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806af10 0x7f5f08198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 0x7f5f08198720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c970 tx=0x7f5ef800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f057fa640 1 -- 172.21.15.113:0/3661669126 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8007c10 con 0x7f5f0806a550 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f057fa640 1 -- 172.21.15.113:0/3661669126 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5ef8007db0 con 0x7f5f0806a550 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0819d9b0 con 0x7f5f0806a550 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f057fa640 1 -- 172.21.15.113:0/3661669126 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800f910 con 0x7f5f0806a550 2024-04-22T07:15:04.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.276+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f081a5240 con 0x7f5f0806a550 2024-04-22T07:15:04.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.278+0000 7f5f057fa640 1 -- 172.21.15.113:0/3661669126 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f5ef800fab0 con 0x7f5f0806a550 2024-04-22T07:15:04.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.278+0000 7f5f107e5740 1 Processor -- start 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.278+0000 7f5f107e5740 1 -- start start 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90610 0x564f61a909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a99b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f107e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564f619f4b10 con 0x564f61a90fc0 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f107e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564f61992240 con 0x564f61a9a120 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f057fa640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ef40a3ed0 0x7f5ef40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f107e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f619f0ad0 con 0x564f61a90610 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a99b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a99b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56852/0 (socket says 172.21.15.113:56852) 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f07fff640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ef40a3ed0 0x7f5ef40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 -- 172.21.15.113:0/3529432583 learned_addr learned my addr 172.21.15.113:0/3529432583 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f07fff640 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90610 0x564f61a909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f057fa640 1 -- 172.21.15.113:0/3661669126 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5ef80d0ee0 con 0x7f5f0806a550 2024-04-22T07:15:04.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5efffff640 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 -- 172.21.15.113:0/3529432583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90610 msgr2=0x564f61a909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90610 0x564f61a909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 -- 172.21.15.113:0/3529432583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 msgr2=0x564f61a9a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 -- 172.21.15.113:0/3529432583 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f617072d0 con 0x564f61a90fc0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f07fff640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ef40a3ed0 0x7f5ef40a6390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009a60 tx=0x7f5f00032570 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a99b00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80a53b0 tx=0x7f5ef8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c457e721bcb24e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.279+0000 7f5efdffb640 1 -- 172.21.15.113:0/3529432583 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80a58e0 con 0x564f61a90fc0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5efdffb640 1 -- 172.21.15.113:0/3529432583 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5ef8018b90 con 0x564f61a90fc0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5efdffb640 1 -- 172.21.15.113:0/3529432583 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80a56b0 con 0x564f61a90fc0 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 -- 172.21.15.113:0/3529432583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 msgr2=0x564f61a99b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a99b00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80a53b0 tx=0x7f5ef8000bc0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 -- 172.21.15.113:0/3529432583 shutdown_connections 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a99b00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3529432583 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90610 0x564f61a909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 -- 172.21.15.113:0/3529432583 >> 172.21.15.113:0/3529432583 conn(0x564f61a8aa60 msgr2=0x564f61a8ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 -- 172.21.15.113:0/3529432583 shutdown_connections 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.280+0000 7f5f107e5740 1 -- 172.21.15.113:0/3529432583 wait complete. 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 -- start start 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 Processor -- start 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61a9dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564f61aa10a0 con 0x564f61a90610 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564f6197d190 con 0x564f61a9a120 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f61aa1220 con 0x564f61a90fc0 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f107e5740 10 client.0 _ll_register_callbacks cb 0x564f619788b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61a9dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61a9dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56854/0 (socket says 172.21.15.113:56854) 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 -- 172.21.15.113:0/3290272901 learned_addr learned my addr 172.21.15.113:0/3290272901 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5efffff640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f07fff640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 -- 172.21.15.113:0/3290272901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 msgr2=0x564f61a9e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a9e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 -- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 msgr2=0x564f61a9e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f077fe640 1 -- 172.21.15.113:0/3290272901 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f617d16b0 con 0x564f61a90610 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.281+0000 7f5f07fff640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f077fe640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61a9dc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8099160 tx=0x7f5ef8018d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0129694486ff42a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5ee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/3290272901 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8065020 con 0x564f61a90610 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/3290272901 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5ef80a32e0 con 0x564f61a90610 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef8018b90 con 0x564f61a90610 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/3290272901 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5ef80bd2f0 con 0x564f61a90610 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 10 client.15675 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:04.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564f61aad430 con 0x564f61a90610 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/3290272901 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5ef8018b90 con 0x564f61a90610 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/3290272901 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5ef809bbc0 con 0x564f61a90610 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 10 client.15675 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 10 client.15675 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 20 client.15675 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61aae780 0x564f61ab0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 4 client.15675 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f61a02530 con 0x564f61aae780 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.282+0000 7f5efffff640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61aae780 0x564f61ab0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.283+0000 7f5ee6ffd640 10 client.15675 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.283+0000 7f5efffff640 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61aae780 0x564f61ab0930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bafcc999683e9fdd server_cookie=c1c44da7f67e8029 in_seq=0 out_seq=0 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/3290272901 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f61a02530 con 0x564f61aae780 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5ee6ffd640 10 client.15675 handle_command_reply: tid=0 2024-04-22T07:15:04.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 client.15675 shutdown 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 msgr2=0x564f61a9dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61a9dc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8099160 tx=0x7f5ef8018d40 comp rx=0 tx=0).stop 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 shutdown_connections 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61aae780 0x564f61ab0930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a9e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a9e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5ee6ffd640 0 client.15675 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 --2- 172.21.15.113:0/3290272901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61a9dc00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 >> 172.21.15.113:0/3290272901 conn(0x564f61a8aa60 msgr2=0x564f61a8cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 shutdown_connections 2024-04-22T07:15:04.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 1 -- 172.21.15.113:0/3290272901 wait complete. 2024-04-22T07:15:04.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.284+0000 7f5f107e5740 20 client.15675 trim_cache size 0 max 16384 2024-04-22T07:15:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 Processor -- start 2024-04-22T07:15:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 -- start start 2024-04-22T07:15:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61aa0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a89a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a97f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564f61aaec70 con 0x564f61a90610 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564f61aaf340 con 0x564f61a9a120 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f107e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f61aafa10 con 0x564f61a90fc0 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61aa0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61aa0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56868/0 (socket says 172.21.15.113:56868) 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 -- 172.21.15.113:0/2438707839 learned_addr learned my addr 172.21.15.113:0/2438707839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 -- 172.21.15.113:0/2438707839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 msgr2=0x564f61a89a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a89a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 -- 172.21.15.113:0/2438707839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 msgr2=0x564f61a97f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a97f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 -- 172.21.15.113:0/2438707839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f61a8f840 con 0x564f61a90610 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.302+0000 7f5f077fe640 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61aa0330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c940 tx=0x7f5ef8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5fba2519ef7cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5efd7fa640 1 -- 172.21.15.113:0/2438707839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8065050 con 0x564f61a90610 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5efd7fa640 1 -- 172.21.15.113:0/2438707839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5ef80a58e0 con 0x564f61a90610 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5efd7fa640 1 -- 172.21.15.113:0/2438707839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80ce7d0 con 0x564f61a90610 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 -- 172.21.15.113:0/2438707839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 msgr2=0x564f61aa0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61aa0330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c940 tx=0x7f5ef8000bc0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 -- 172.21.15.113:0/2438707839 shutdown_connections 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a9a120 0x564f61a97f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a90fc0 0x564f61a89a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 --2- 172.21.15.113:0/2438707839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90610 0x564f61aa0330 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 -- 172.21.15.113:0/2438707839 >> 172.21.15.113:0/2438707839 conn(0x564f61a8aa60 msgr2=0x564f61a8ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 -- 172.21.15.113:0/2438707839 shutdown_connections 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.303+0000 7f5f107e5740 1 -- 172.21.15.113:0/2438707839 wait complete. 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 -- start start 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 Processor -- start 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a90610 0x564f61a9a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a9a120 0x564f61a9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564f61ab07b0 con 0x564f61a90fc0 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564f61ab00e0 con 0x564f61a90610 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f61ab0e80 con 0x564f61a9a120 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f107e5740 10 client.0 _ll_register_callbacks cb 0x7f5ef800a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a9ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56876/0 (socket says 172.21.15.113:56876) 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 -- 172.21.15.113:0/1966960963 learned_addr learned my addr 172.21.15.113:0/1966960963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f077fe640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a90610 0x564f61a9a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f07fff640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a9a120 0x564f61a9b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 -- 172.21.15.113:0/1966960963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a9a120 msgr2=0x564f61a9b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a9a120 0x564f61a9b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 -- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a90610 msgr2=0x564f61a9a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a90610 0x564f61a9a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5efffff640 1 -- 172.21.15.113:0/1966960963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f61ab5bd0 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.304+0000 7f5f07fff640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a9a120 0x564f61a9b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5efffff640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a9ae10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x564f61a90300 tx=0x7f5ef0014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebba820e147a85fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5ee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/1966960963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0005220 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/1966960963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f5ef00053c0 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef80a58e0 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/1966960963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5ef002d050 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 10 client.15681 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564f61b15710 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/1966960963 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5ef80a58e0 con 0x564f61a90fc0 2024-04-22T07:15:04.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/1966960963 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5ef001e230 con 0x564f61a90fc0 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 10 client.15681 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 10 client.15681 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 20 client.15681 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61b1a350 0x564f61b19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 4 client.15681 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.305+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "status"}) -- 0x564f619fc3a0 con 0x564f61b1a350 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f077fe640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61b1a350 0x564f61b19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5ee6ffd640 10 client.15681 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f077fe640 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61b1a350 0x564f61b19060 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b63c938e2d075ad0 server_cookie=ec4e83c2fc22a3e5 in_seq=0 out_seq=0 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5ee6ffd640 1 -- 172.21.15.113:0/1966960963 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564f619fc3a0 con 0x564f61b1a350 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5ee6ffd640 10 client.15681 handle_command_reply: tid=0 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 client.15681 shutdown 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 msgr2=0x564f61a9ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a9ae10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x564f61a90300 tx=0x7f5ef0014900 comp rx=0 tx=0).stop 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 shutdown_connections 2024-04-22T07:15:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x564f61b1a350 0x564f61b19060 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f61a9a120 0x564f61a9b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564f61a90fc0 0x564f61a9ae10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5ee6ffd640 0 client.15681 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 --2- 172.21.15.113:0/1966960963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564f61a90610 0x564f61a9a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 >> 172.21.15.113:0/1966960963 conn(0x564f61a8aa60 msgr2=0x564f61a85800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 shutdown_connections 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.306+0000 7f5f107e5740 1 -- 172.21.15.113:0/1966960963 wait complete. 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f107e5740 20 client.15681 trim_cache size 0 max 16384 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 1, 2024-04-22T07:15:04.312 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24473, 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 127.139195851, 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 22, 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 89, 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 88, 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 145.62474423800001 2024-04-22T07:15:04.313 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ef40a3ed0 msgr2=0x7f5ef40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ef40a3ed0 0x7f5ef40a6390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009a60 tx=0x7f5f00032570 comp rx=0 tx=0).stop 2024-04-22T07:15:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 msgr2=0x7f5f08198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 0x7f5f08198720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c970 tx=0x7f5ef800ce30 comp rx=0 tx=0).stop 2024-04-22T07:15:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 shutdown_connections 2024-04-22T07:15:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ef40a3ed0 0x7f5ef40a6390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f08108150 0x7f5f0819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f0806af10 0x7f5f08198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 --2- 172.21.15.113:0/3661669126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f0806a550 0x7f5f08198720 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 >> 172.21.15.113:0/3661669126 conn(0x7f5f08076c50 msgr2=0x7f5f080ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 shutdown_connections 2024-04-22T07:15:04.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.307+0000 7f5f0db15640 1 -- 172.21.15.113:0/3661669126 wait complete. 2024-04-22T07:15:04.315 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/122760049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.315 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3969923150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.315 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1937350640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- 172.21.15.113:0/3911658257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c101a40 msgr2=0x7f613c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- 172.21.15.113:0/3911658257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c101a40 0x7f613c105a80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- 172.21.15.113:0/3911658257 shutdown_connections 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- 172.21.15.113:0/3911658257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f613c106160 0x7f613c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- 172.21.15.113:0/3911658257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c101a40 0x7f613c105a80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- 172.21.15.113:0/3911658257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c101090 0x7f613c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- 172.21.15.113:0/3911658257 >> 172.21.15.113:0/3911658257 conn(0x7f613c077f50 msgr2=0x7f613c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- 172.21.15.113:0/3911658257 shutdown_connections 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- 172.21.15.113:0/3911658257 wait complete. 2024-04-22T07:15:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 Processor -- start 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- start start 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f613c101090 0x7f613c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 0x7f613c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c19bf40 0x7f613c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1111c0 con 0x7f613c106160 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f613c111040 con 0x7f613c101090 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f6143231640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f613c111340 con 0x7f613c19bf40 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f613bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 0x7f613c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.418+0000 7f613bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 0x7f613c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56900/0 (socket says 172.21.15.113:56900) 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 -- 172.21.15.113:0/3237679347 learned_addr learned my addr 172.21.15.113:0/3237679347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f6140fcd640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f613c101090 0x7f613c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 -- 172.21.15.113:0/3237679347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c19bf40 msgr2=0x7f613c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f61417ce640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c19bf40 0x7f613c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c19bf40 0x7f613c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 -- 172.21.15.113:0/3237679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f613c101090 msgr2=0x7f613c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f613c101090 0x7f613c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 -- 172.21.15.113:0/3237679347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c19ca30 con 0x7f613c106160 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f61417ce640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c19bf40 0x7f613c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f613bfff640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 0x7f613c197bc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f612802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f6139ffb640 1 -- 172.21.15.113:0/3237679347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280044d0 con 0x7f613c106160 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f6139ffb640 1 -- 172.21.15.113:0/3237679347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61280052f0 con 0x7f613c106160 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c19ccc0 con 0x7f613c106160 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f6139ffb640 1 -- 172.21.15.113:0/3237679347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128003df0 con 0x7f613c106160 2024-04-22T07:15:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.419+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c0693d0 con 0x7f613c106160 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.421+0000 7f6139ffb640 1 -- 172.21.15.113:0/3237679347 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f61280035f0 con 0x7f613c106160 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 Processor -- start 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 -- start start 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6139ffb640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f612c0a4070 0x7f612c0a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c675b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 0x55f03c681130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f03c5d9b10 con 0x55f03c67ece0 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f03c577240 con 0x55f03c676160 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6145e29740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f03c5d5ad0 con 0x55f03c6757b0 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6139ffb640 1 -- 172.21.15.113:0/3237679347 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f61280f4570 con 0x7f613c106160 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f61417ce640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f612c0a4070 0x7f612c0a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f61417ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 0x55f03c681130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f61417ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 0x55f03c681130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56914/0 (socket says 172.21.15.113:56914) 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f61417ce640 1 -- 172.21.15.113:0/3137019315 learned_addr learned my addr 172.21.15.113:0/3137019315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f613bfff640 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f6140fcd640 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c675b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f613bfff640 1 -- 172.21.15.113:0/3137019315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 msgr2=0x55f03c675b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f613bfff640 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c675b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f613bfff640 1 -- 172.21.15.113:0/3137019315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 msgr2=0x55f03c681130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f613bfff640 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 0x55f03c681130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f613bfff640 1 -- 172.21.15.113:0/3137019315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03c2ec2d0 con 0x55f03c676160 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.422+0000 7f61417ce640 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 0x55f03c681130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f61417ce640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f612c0a4070 0x7f612c0a6530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6134005c40 tx=0x7f6134009d70 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f613bfff640 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67e710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61280f1bc0 tx=0x7f6128003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6cbd41a84992f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f6126ffd640 1 -- 172.21.15.113:0/3137019315 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612804ede0 con 0x55f03c676160 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f6126ffd640 1 -- 172.21.15.113:0/3137019315 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61280c0ba0 con 0x55f03c676160 2024-04-22T07:15:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f6126ffd640 1 -- 172.21.15.113:0/3137019315 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612804ea20 con 0x55f03c676160 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f6145e29740 1 -- 172.21.15.113:0/3137019315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 msgr2=0x55f03c67e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.423+0000 7f6145e29740 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67e710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61280f1bc0 tx=0x7f6128003920 comp rx=0 tx=0).stop 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- 172.21.15.113:0/3137019315 shutdown_connections 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c67ece0 0x55f03c681130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67e710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 --2- 172.21.15.113:0/3137019315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c675b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- 172.21.15.113:0/3137019315 >> 172.21.15.113:0/3137019315 conn(0x55f03c66fa60 msgr2=0x55f03c66fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- 172.21.15.113:0/3137019315 shutdown_connections 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- 172.21.15.113:0/3137019315 wait complete. 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- start start 2024-04-22T07:15:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 Processor -- start 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c67bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c67ece0 0x55f03c67c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f03c577240 con 0x55f03c6757b0 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f03c562190 con 0x55f03c676160 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f03c681a10 con 0x55f03c67ece0 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6145e29740 10 client.0 _ll_register_callbacks cb 0x55f03c55d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6140fcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c67bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6140fcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c67bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56920/0 (socket says 172.21.15.113:56920) 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f6140fcd640 1 -- 172.21.15.113:0/927272280 learned_addr learned my addr 172.21.15.113:0/927272280 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.424+0000 7f613bfff640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f61417ce640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c67ece0 0x55f03c67c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6140fcd640 1 -- 172.21.15.113:0/927272280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c67ece0 msgr2=0x55f03c67c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6140fcd640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c67ece0 0x55f03c67c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6140fcd640 1 -- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 msgr2=0x55f03c67c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6140fcd640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6140fcd640 1 -- 172.21.15.113:0/927272280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03c3b66b0 con 0x55f03c6757b0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6140fcd640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c67bcb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6130010a50 tx=0x7f6130010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5561f407ce32007e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f611bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f611bfff640 1 -- 172.21.15.113:0/927272280 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613001e060 con 0x55f03c6757b0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f611bfff640 1 -- 172.21.15.113:0/927272280 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f613000fc30 con 0x55f03c6757b0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61280c0ba0 con 0x55f03c6757b0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f611bfff640 1 -- 172.21.15.113:0/927272280 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f613000fe30 con 0x55f03c6757b0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6145e29740 10 client.15693 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f03c68db00 con 0x55f03c6757b0 2024-04-22T07:15:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.425+0000 7f611bfff640 1 -- 172.21.15.113:0/927272280 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61280c0ba0 con 0x55f03c6757b0 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f611bfff640 1 -- 172.21.15.113:0/927272280 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f61300184e0 con 0x55f03c6757b0 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f6145e29740 10 client.15693 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f6145e29740 10 client.15693 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f6145e29740 20 client.15693 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f6145e29740 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c690150 0x55f03c6915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f6145e29740 4 client.15693 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f03c5e7530 con 0x55f03c690150 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f613bfff640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c690150 0x55f03c6915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f611bfff640 10 client.15693 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.426+0000 7f613bfff640 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c690150 0x55f03c6915c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980dd2fd06f8dba5 server_cookie=617ee4751d979512 in_seq=0 out_seq=0 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f611bfff640 1 -- 172.21.15.113:0/927272280 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f03c5e7530 con 0x55f03c690150 2024-04-22T07:15:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f611bfff640 10 client.15693 handle_command_reply: tid=0 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 client.15693 shutdown 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 msgr2=0x55f03c67bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c67bcb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6130010a50 tx=0x7f6130010f10 comp rx=0 tx=0).stop 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 shutdown_connections 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c67ece0 0x55f03c67c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c67c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c690150 0x55f03c6915c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 --2- 172.21.15.113:0/927272280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c67bcb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f611bfff640 0 client.15693 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 >> 172.21.15.113:0/927272280 conn(0x55f03c66fa60 msgr2=0x55f03c671bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 shutdown_connections 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 1 -- 172.21.15.113:0/927272280 wait complete. 2024-04-22T07:15:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.427+0000 7f6145e29740 20 client.15693 trim_cache size 0 max 16384 2024-04-22T07:15:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 Processor -- start 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 -- start start 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 0x55f03c67e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c67c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c678f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f03c68f900 con 0x55f03c66fa60 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f03c68ffd0 con 0x55f03c676160 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.445+0000 7f6145e29740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f03c6906a0 con 0x55f03c6757b0 2024-04-22T07:15:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 0x55f03c67e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 0x55f03c67e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56922/0 (socket says 172.21.15.113:56922) 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 -- 172.21.15.113:0/721275003 learned_addr learned my addr 172.21.15.113:0/721275003 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 -- 172.21.15.113:0/721275003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 msgr2=0x55f03c67c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c67c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 -- 172.21.15.113:0/721275003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 msgr2=0x55f03c678f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c678f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 -- 172.21.15.113:0/721275003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03c67a220 con 0x55f03c66fa60 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f6140fcd640 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 0x55f03c67e3e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f613000f870 tx=0x7f6130010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bad13c7712928fc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f61267fc640 1 -- 172.21.15.113:0/721275003 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000fc30 con 0x55f03c66fa60 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f61267fc640 1 -- 172.21.15.113:0/721275003 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6130018b20 con 0x55f03c66fa60 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.446+0000 7f61267fc640 1 -- 172.21.15.113:0/721275003 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61300048c0 con 0x55f03c66fa60 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- 172.21.15.113:0/721275003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 msgr2=0x55f03c67e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 0x55f03c67e3e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f613000f870 tx=0x7f6130010640 comp rx=0 tx=0).stop 2024-04-22T07:15:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- 172.21.15.113:0/721275003 shutdown_connections 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c676160 0x55f03c678f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c6757b0 0x55f03c67c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- 172.21.15.113:0/721275003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c66fa60 0x55f03c67e3e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- 172.21.15.113:0/721275003 >> 172.21.15.113:0/721275003 conn(0x55f03c6768b0 msgr2=0x55f03c67a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- 172.21.15.113:0/721275003 shutdown_connections 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- 172.21.15.113:0/721275003 wait complete. 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- start start 2024-04-22T07:15:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 Processor -- start 2024-04-22T07:15:04.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c66fa60 0x55f03c678e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c679350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c676160 0x55f03c697e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f03c691440 con 0x55f03c6757b0 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f03c690d70 con 0x55f03c66fa60 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.447+0000 7f6145e29740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f03c691b10 con 0x55f03c676160 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f6145e29740 10 client.0 _ll_register_callbacks cb 0x7f613000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c679350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c679350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56936/0 (socket says 172.21.15.113:56936) 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 -- 172.21.15.113:0/3744784796 learned_addr learned my addr 172.21.15.113:0/3744784796 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f61417ce640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c676160 0x55f03c697e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f6140fcd640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c66fa60 0x55f03c678e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 -- 172.21.15.113:0/3744784796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c676160 msgr2=0x55f03c697e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c676160 0x55f03c697e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 -- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c66fa60 msgr2=0x55f03c678e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c66fa60 0x55f03c678e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 -- 172.21.15.113:0/3744784796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f03c6983a0 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f613bfff640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c679350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f61280f1d00 tx=0x7f612802c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c9700ff8965ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f611bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f611bfff640 1 -- 172.21.15.113:0/3744784796 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612804ec00 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f611bfff640 1 -- 172.21.15.113:0/3744784796 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f61280054a0 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6130018b20 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f611bfff640 1 -- 172.21.15.113:0/3744784796 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f61280c0ba0 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f6145e29740 10 client.15699 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f03c6f6880 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.448+0000 7f611bfff640 1 -- 172.21.15.113:0/3744784796 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6130018b20 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f611bfff640 1 -- 172.21.15.113:0/3744784796 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6128042070 con 0x55f03c6757b0 2024-04-22T07:15:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6145e29740 10 client.15699 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6145e29740 10 client.15699 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6145e29740 20 client.15699 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6145e29740 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c67ece0 0x55f03c6fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6145e29740 4 client.15699 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "status"}) -- 0x55f03c5e13a0 con 0x55f03c67ece0 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6140fcd640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c67ece0 0x55f03c6fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f6140fcd640 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c67ece0 0x55f03c6fa050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea967fa8d2de0a9 server_cookie=112785b7aed9e0ca in_seq=0 out_seq=0 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.449+0000 7f611bfff640 10 client.15699 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f611bfff640 1 -- 172.21.15.113:0/3744784796 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f03c5e13a0 con 0x55f03c67ece0 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f611bfff640 10 client.15699 handle_command_reply: tid=0 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 client.15699 shutdown 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 msgr2=0x55f03c679350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c679350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f61280f1d00 tx=0x7f612802c1f0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 shutdown_connections 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f03c676160 0x55f03c697e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55f03c67ece0 0x55f03c6fa050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f03c6757b0 0x55f03c679350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 --2- 172.21.15.113:0/3744784796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f03c66fa60 0x55f03c678e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f611bfff640 0 client.15699 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 >> 172.21.15.113:0/3744784796 conn(0x55f03c6768b0 msgr2=0x55f03c66e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 shutdown_connections 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 1 -- 172.21.15.113:0/3744784796 wait complete. 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6145e29740 20 client.15699 trim_cache size 0 max 16384 2024-04-22T07:15:04.454 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "81035712-0076-11ef-bc93-c7b262605968", 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 24506, 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 129.293213552, 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 22, 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 89, 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 88, 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 134.93865177500001 2024-04-22T07:15:04.455 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f612c0a4070 msgr2=0x7f612c0a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f612c0a4070 0x7f612c0a6530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6134005c40 tx=0x7f6134009d70 comp rx=0 tx=0).stop 2024-04-22T07:15:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 msgr2=0x7f613c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 0x7f613c197bc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f612802cdf0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 shutdown_connections 2024-04-22T07:15:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f612c0a4070 0x7f612c0a6530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f613c19bf40 0x7f613c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f613c106160 0x7f613c197bc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 --2- 172.21.15.113:0/3237679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f613c101090 0x7f613c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.450+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 >> 172.21.15.113:0/3237679347 conn(0x7f613c077f50 msgr2=0x7f613c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.451+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 shutdown_connections 2024-04-22T07:15:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:04.451+0000 7f6143231640 1 -- 172.21.15.113:0/3237679347 wait complete. 2024-04-22T07:15:04.459 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-22T07:15:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/122760049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3969923150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1937350640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/122760049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3969923150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:15:04.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1937350640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:05.369 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:05 smithi113 ceph-mon[27933]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-04-22T07:15:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:05 smithi133 ceph-mon[31198]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-04-22T07:15:05.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:05 smithi154 ceph-mon[31862]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-04-22T07:15:07.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:07 smithi133 ceph-mon[31198]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-04-22T07:15:07.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:07 smithi154 ceph-mon[31862]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-04-22T07:15:07.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:07 smithi113 ceph-mon[27933]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-04-22T07:15:09.369 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:09 smithi133 ceph-mon[31198]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-04-22T07:15:09.370 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:09 smithi154 ceph-mon[31862]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-04-22T07:15:09.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:09 smithi113 ceph-mon[27933]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-04-22T07:15:11.556 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:11 smithi133 ceph-mon[31198]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-22T07:15:11.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:11 smithi154 ceph-mon[31862]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-22T07:15:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:11 smithi113 ceph-mon[27933]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-22T07:15:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:13 smithi133 ceph-mon[31198]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-04-22T07:15:13.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:13 smithi154 ceph-mon[31862]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-04-22T07:15:13.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:13 smithi113 ceph-mon[27933]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-04-22T07:15:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:15 smithi133 ceph-mon[31198]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-22T07:15:15.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:15 smithi154 ceph-mon[31862]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-22T07:15:15.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:15 smithi113 ceph-mon[27933]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-22T07:15:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:17 smithi133 ceph-mon[31198]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T07:15:17.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:17 smithi154 ceph-mon[31862]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T07:15:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:17 smithi113 ceph-mon[27933]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T07:15:19.460 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-22T07:15:19.460 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-04-22T07:15:19.460 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:19.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:19 smithi113 ceph-mon[27933]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-22T07:15:19.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:19 smithi133 ceph-mon[31198]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-22T07:15:19.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:19 smithi154 ceph-mon[31862]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-22T07:15:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 -- 172.21.15.113:0/541627472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14107260 msgr2=0x7f0a1410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 --2- 172.21.15.113:0/541627472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14107260 0x7f0a1410ee30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009ca0 tx=0x7f0a0c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 -- 172.21.15.113:0/541627472 shutdown_connections 2024-04-22T07:15:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 --2- 172.21.15.113:0/541627472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14107260 0x7f0a1410ee30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 --2- 172.21.15.113:0/541627472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102b40 0x7f0a14106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 --2- 172.21.15.113:0/541627472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a14102190 0x7f0a14102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 -- 172.21.15.113:0/541627472 >> 172.21.15.113:0/541627472 conn(0x7f0a140793e0 msgr2=0x7f0a140797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 -- 172.21.15.113:0/541627472 shutdown_connections 2024-04-22T07:15:19.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.573+0000 7f0a18950640 1 -- 172.21.15.113:0/541627472 wait complete. 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 Processor -- start 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 -- start start 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102190 0x7f0a1419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a14107260 0x7f0a1419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 0x7f0a14198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14112300 con 0x7f0a14198600 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14112180 con 0x7f0a14102190 2024-04-22T07:15:19.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a18950640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14112480 con 0x7f0a14107260 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a12d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 0x7f0a14198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a12d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 0x7f0a14198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60712/0 (socket says 172.21.15.113:60712) 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.574+0000 7f0a12d9d640 1 -- 172.21.15.113:0/3580654434 learned_addr learned my addr 172.21.15.113:0/3580654434 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a11d9b640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a14107260 0x7f0a1419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a1259c640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102190 0x7f0a1419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a12d9d640 1 -- 172.21.15.113:0/3580654434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a14107260 msgr2=0x7f0a1419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a12d9d640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a14107260 0x7f0a1419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a12d9d640 1 -- 172.21.15.113:0/3580654434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102190 msgr2=0x7f0a1419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a12d9d640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102190 0x7f0a1419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a12d9d640 1 -- 172.21.15.113:0/3580654434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a14199120 con 0x7f0a14198600 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a1259c640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102190 0x7f0a1419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a12d9d640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 0x7f0a14198ab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c037040 tx=0x7f0a0c035d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c039040 con 0x7f0a14198600 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a141993b0 con 0x7f0a14198600 2024-04-22T07:15:19.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0a0c044070 con 0x7f0a14198600 2024-04-22T07:15:19.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c004330 con 0x7f0a14198600 2024-04-22T07:15:19.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.575+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a141a5600 con 0x7f0a14198600 2024-04-22T07:15:19.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.576+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a14069230 con 0x7f0a14198600 2024-04-22T07:15:19.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.578+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f0a0c0044d0 con 0x7f0a14198600 2024-04-22T07:15:19.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.579+0000 7f0a037fe640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09fc0a3ed0 0x7f09fc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.579+0000 7f0a1259c640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09fc0a3ed0 0x7f09fc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.579+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f0a0c0f3680 con 0x7f0a14198600 2024-04-22T07:15:19.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.579+0000 7f0a1259c640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09fc0a3ed0 0x7f09fc0a6390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a080058c0 tx=0x7f0a08005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.582+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a0c0bfc20 con 0x7f0a14198600 2024-04-22T07:15:19.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.796+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a1410a4f0 con 0x7f0a14198600 2024-04-22T07:15:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.797+0000 7f0a037fe640 1 -- 172.21.15.113:0/3580654434 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f0a0c0bf520 con 0x7f0a14198600 2024-04-22T07:15:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:19.795 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:19.795 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09fc0a3ed0 msgr2=0x7f09fc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09fc0a3ed0 0x7f09fc0a6390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a080058c0 tx=0x7f0a08005850 comp rx=0 tx=0).stop 2024-04-22T07:15:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 msgr2=0x7f0a14198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 0x7f0a14198ab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c037040 tx=0x7f0a0c035d00 comp rx=0 tx=0).stop 2024-04-22T07:15:19.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 shutdown_connections 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09fc0a3ed0 0x7f09fc0a6390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a14198600 0x7f0a14198ab0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a14107260 0x7f0a1419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 --2- 172.21.15.113:0/3580654434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a14102190 0x7f0a1419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 >> 172.21.15.113:0/3580654434 conn(0x7f0a140793e0 msgr2=0x7f0a140ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 shutdown_connections 2024-04-22T07:15:19.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.799+0000 7f0a18950640 1 -- 172.21.15.113:0/3580654434 wait complete. 2024-04-22T07:15:19.808 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 78.85 --await' 2024-04-22T07:15:19.809 DEBUG:teuthology.orchestra.run.smithi113:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24506 quiesce db / --timeout 60 --expiration 78.85 --await 2024-04-22T07:15:19.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 -- 172.21.15.113:0/3766524925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 msgr2=0x7fc28010bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 --2- 172.21.15.113:0/3766524925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28010bfb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc274009900 tx=0x7fc27402bcd0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 -- 172.21.15.113:0/3766524925 shutdown_connections 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 --2- 172.21.15.113:0/3766524925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28010c580 0x7fc28010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 --2- 172.21.15.113:0/3766524925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28010bfb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 --2- 172.21.15.113:0/3766524925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2801031b0 0x7fc280103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 -- 172.21.15.113:0/3766524925 >> 172.21.15.113:0/3766524925 conn(0x7fc2800771d0 msgr2=0x7fc2800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 -- 172.21.15.113:0/3766524925 shutdown_connections 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.958+0000 7fc28636d640 1 -- 172.21.15.113:0/3766524925 wait complete. 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 Processor -- start 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 -- start start 2024-04-22T07:15:19.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2801031b0 0x7fc28019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28010c580 0x7fc280198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc280112300 con 0x7fc280103ad0 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc280112180 con 0x7fc28010c580 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc28636d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc280112480 con 0x7fc2801031b0 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60740/0 (socket says 172.21.15.113:60740) 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.959+0000 7fc27f7fe640 1 -- 172.21.15.113:0/474617737 learned_addr learned my addr 172.21.15.113:0/474617737 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27ffff640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2801031b0 0x7fc28019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc28490a640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28010c580 0x7fc280198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27f7fe640 1 -- 172.21.15.113:0/474617737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2801031b0 msgr2=0x7fc28019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2801031b0 0x7fc28019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27f7fe640 1 -- 172.21.15.113:0/474617737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28010c580 msgr2=0x7fc280198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28010c580 0x7fc280198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27f7fe640 1 -- 172.21.15.113:0/474617737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc280198f20 con 0x7fc280103ad0 2024-04-22T07:15:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28019eae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc274006010 tx=0x7fc27402ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27d7fa640 1 -- 172.21.15.113:0/474617737 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274043020 con 0x7fc280103ad0 2024-04-22T07:15:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2801991b0 con 0x7fc280103ad0 2024-04-22T07:15:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27d7fa640 1 -- 172.21.15.113:0/474617737 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc2740042d0 con 0x7fc280103ad0 2024-04-22T07:15:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc27d7fa640 1 -- 172.21.15.113:0/474617737 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274003b20 con 0x7fc280103ad0 2024-04-22T07:15:19.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.960+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2801a5520 con 0x7fc280103ad0 2024-04-22T07:15:19.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.962+0000 7fc27d7fa640 1 -- 172.21.15.113:0/474617737 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fc274046020 con 0x7fc280103ad0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.962+0000 7fc289034740 1 Processor -- start 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 -- start start 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27d7fa640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2680a3ed0 0x7fc2680a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03239da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a03242980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558a0319a0b0 con 0x558a032399c0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a0314b680 con 0x558a0323a2e0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc289034740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558a03157c60 con 0x558a03242ec0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc28490a640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2680a3ed0 0x7fc2680a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03239da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27d7fa640 1 -- 172.21.15.113:0/474617737 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc2740f2bb0 con 0x7fc280103ad0 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03239da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60742/0 (socket says 172.21.15.113:60742) 2024-04-22T07:15:19.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 -- 172.21.15.113:0/1357336756 learned_addr learned my addr 172.21.15.113:0/1357336756 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a03242980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc28490a640 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 -- 172.21.15.113:0/1357336756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 msgr2=0x558a032452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 -- 172.21.15.113:0/1357336756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 msgr2=0x558a03242980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a03242980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 -- 172.21.15.113:0/1357336756 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a02f7cc90 con 0x558a032399c0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc28490a640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2680a3ed0 0x7fc2680a6390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc278005cc0 tx=0x7fc278012960 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.963+0000 7fc27ffff640 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03239da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc27000dc30 tx=0x7fc27000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bb9aeb28835da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc2667fc640 1 -- 172.21.15.113:0/1357336756 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270011440 con 0x558a032399c0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc2667fc640 1 -- 172.21.15.113:0/1357336756 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc270011ab0 con 0x558a032399c0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc2667fc640 1 -- 172.21.15.113:0/1357336756 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270010a10 con 0x558a032399c0 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 -- 172.21.15.113:0/1357336756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 msgr2=0x558a03239da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03239da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc27000dc30 tx=0x7fc27000f730 comp rx=0 tx=0).stop 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 -- 172.21.15.113:0/1357336756 shutdown_connections 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a03242980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 --2- 172.21.15.113:0/1357336756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03239da0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 -- 172.21.15.113:0/1357336756 >> 172.21.15.113:0/1357336756 conn(0x558a032340f0 msgr2=0x558a032344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 -- 172.21.15.113:0/1357336756 shutdown_connections 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.964+0000 7fc289034740 1 -- 172.21.15.113:0/1357336756 wait complete. 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 -- start start 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 Processor -- start 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a032399c0 0x558a0323ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a032404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 0x558a03240a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558a0319d400 con 0x558a03242ec0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a03159840 con 0x558a0323a2e0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558a031a2720 con 0x558a032399c0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc289034740 10 client.0 _ll_register_callbacks cb 0x558a031aaea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 0x558a03240a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 0x558a03240a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60750/0 (socket says 172.21.15.113:60750) 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 -- 172.21.15.113:0/46540303 learned_addr learned my addr 172.21.15.113:0/46540303 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a032404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc27ffff640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a032399c0 0x558a0323ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 -- 172.21.15.113:0/46540303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a032399c0 msgr2=0x558a0323ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a032399c0 0x558a0323ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 -- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 msgr2=0x558a032404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a032404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc28490a640 1 -- 172.21.15.113:0/46540303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a02fdd420 con 0x558a03242ec0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a032404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.965+0000 7fc27ffff640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a032399c0 0x558a0323ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc28490a640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 0x558a03240a10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc278010df0 tx=0x7fc278004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1215972346952ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc25f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc25f7fe640 1 -- 172.21.15.113:0/46540303 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278004b20 con 0x558a03242ec0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc25f7fe640 1 -- 172.21.15.113:0/46540303 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc278004cc0 con 0x558a03242ec0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc270011ab0 con 0x558a03242ec0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc25f7fe640 1 -- 172.21.15.113:0/46540303 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc278003970 con 0x558a03242ec0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 10 client.15717 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558a02e70580 con 0x558a03242ec0 2024-04-22T07:15:19.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc25f7fe640 1 -- 172.21.15.113:0/46540303 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc270011ab0 con 0x558a03242ec0 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc25f7fe640 1 -- 172.21.15.113:0/46540303 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc27802a530 con 0x558a03242ec0 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 10 client.15717 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 10 client.15717 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 20 client.15717 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a03252c00 0x558a03255060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 4 client.15717 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.966+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a031abb10 con 0x558a03252c00 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.967+0000 7fc27ffff640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a03252c00 0x558a03255060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.967+0000 7fc27ffff640 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a03252c00 0x558a03255060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda8ad26a35fd202 server_cookie=bbd63ae7ec148ced in_seq=0 out_seq=0 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.967+0000 7fc25f7fe640 10 client.15717 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc25f7fe640 1 -- 172.21.15.113:0/46540303 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a031abb10 con 0x558a03252c00 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc25f7fe640 10 client.15717 handle_command_reply: tid=0 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 client.15717 shutdown 2024-04-22T07:15:19.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 msgr2=0x558a03240a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 0x558a03240a10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc278010df0 tx=0x7fc278004ab0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 shutdown_connections 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a03252c00 0x558a03255060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a03242ec0 0x558a03240a10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a032404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc25f7fe640 0 client.15717 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 --2- 172.21.15.113:0/46540303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a032399c0 0x558a0323ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 >> 172.21.15.113:0/46540303 conn(0x558a032340f0 msgr2=0x558a03235e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 shutdown_connections 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 1 -- 172.21.15.113:0/46540303 wait complete. 2024-04-22T07:15:19.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.968+0000 7fc289034740 20 client.15717 trim_cache size 0 max 16384 2024-04-22T07:15:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.984+0000 7fc289034740 1 Processor -- start 2024-04-22T07:15:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.984+0000 7fc289034740 1 -- start start 2024-04-22T07:15:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc289034740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03232fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc289034740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a0324e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc289034740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a03235c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc289034740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2801ade40 con 0x558a032399c0 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc289034740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2801abc90 con 0x558a0323a2e0 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc289034740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2801ad770 con 0x558a03242ec0 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03232fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03232fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60760/0 (socket says 172.21.15.113:60760) 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 -- 172.21.15.113:0/3909494591 learned_addr learned my addr 172.21.15.113:0/3909494591 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a0324e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 -- 172.21.15.113:0/3909494591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 msgr2=0x558a03235c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a03235c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 -- 172.21.15.113:0/3909494591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 msgr2=0x558a0324e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a0324e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 -- 172.21.15.113:0/3909494591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a0323a110 con 0x558a032399c0 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a0324e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc27ffff640 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03232fb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc27000dbd0 tx=0x7fc2700103f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc6f587972a5f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc265ffb640 1 -- 172.21.15.113:0/3909494591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27000dd20 con 0x558a032399c0 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc265ffb640 1 -- 172.21.15.113:0/3909494591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc27002e070 con 0x558a032399c0 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.985+0000 7fc265ffb640 1 -- 172.21.15.113:0/3909494591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270011440 con 0x558a032399c0 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 -- 172.21.15.113:0/3909494591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 msgr2=0x558a03232fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03232fb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc27000dbd0 tx=0x7fc2700103f0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 -- 172.21.15.113:0/3909494591 shutdown_connections 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a03235c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a0323a2e0 0x558a0324e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 --2- 172.21.15.113:0/3909494591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a032399c0 0x558a03232fb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 -- 172.21.15.113:0/3909494591 >> 172.21.15.113:0/3909494591 conn(0x558a032340f0 msgr2=0x558a03235520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:19.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 -- 172.21.15.113:0/3909494591 shutdown_connections 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 -- 172.21.15.113:0/3909494591 wait complete. 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 -- start start 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.986+0000 7fc289034740 1 Processor -- start 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a032399c0 0x558a0323d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 0x558a0323d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2801ac9d0 con 0x558a0323a2e0 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2801ac300 con 0x558a032399c0 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2801ad0a0 con 0x558a03242ec0 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 10 client.0 _ll_register_callbacks cb 0x7fc278010630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 0x558a0323d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 0x558a0323d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60770/0 (socket says 172.21.15.113:60770) 2024-04-22T07:15:19.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 -- 172.21.15.113:0/2211636201 learned_addr learned my addr 172.21.15.113:0/2211636201 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27ffff640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a032399c0 0x558a0323d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc28490a640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 -- 172.21.15.113:0/2211636201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 msgr2=0x558a032b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 -- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a032399c0 msgr2=0x558a0323d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a032399c0 0x558a0323d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 -- 172.21.15.113:0/2211636201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a032b8730 con 0x558a0323a2e0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc27f7fe640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 0x558a0323d570 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc274000c00 tx=0x7fc2740be550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c0260567002714 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc25f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc25f7fe640 1 -- 172.21.15.113:0/2211636201 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740beae0 con 0x558a0323a2e0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc25f7fe640 1 -- 172.21.15.113:0/2211636201 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc2740a2580 con 0x558a0323a2e0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.987+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27002e070 con 0x558a0323a2e0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc25f7fe640 1 -- 172.21.15.113:0/2211636201 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc2740c2e90 con 0x558a0323a2e0 2024-04-22T07:15:19.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 10 client.15723 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558a032b9860 con 0x558a0323a2e0 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc25f7fe640 1 -- 172.21.15.113:0/2211636201 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc27002e070 con 0x558a0323a2e0 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc25f7fe640 1 -- 172.21.15.113:0/2211636201 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc2740c7230 con 0x558a0323a2e0 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 10 client.15723 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 10 client.15723 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 20 client.15723 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a032b9fb0 0x558a032bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 4 client.15723 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.85, "await": true}] 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.85, "await": true}) -- 0x558a031a5980 con 0x558a032b9fb0 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc27ffff640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a032b9fb0 0x558a032bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc25f7fe640 10 client.15723 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:19.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:19.988+0000 7fc27ffff640 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a032b9fb0 0x558a032bc3d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbf3a772035e169 server_cookie=6227647087ca46f2 in_seq=0 out_seq=0 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.002+0000 7fc25f7fe640 1 -- 172.21.15.113:0/2211636201 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x558a031a5980 con 0x558a032b9fb0 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.002+0000 7fc25f7fe640 10 client.15723 handle_command_reply: tid=0 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.002+0000 7fc289034740 1 client.15723 shutdown 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 msgr2=0x558a0323d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 0x558a0323d570 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc274000c00 tx=0x7fc2740be550 comp rx=0 tx=0).stop 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 shutdown_connections 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a03242ec0 0x558a032b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x558a032b9fb0 0x558a032bc3d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a0323a2e0 0x558a0323d570 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 --2- 172.21.15.113:0/2211636201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a032399c0 0x558a0323d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 >> 172.21.15.113:0/2211636201 conn(0x558a032340f0 msgr2=0x558a03235db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc25f7fe640 0 client.15723 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 shutdown_connections 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 1 -- 172.21.15.113:0/2211636201 wait complete. 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc289034740 20 client.15723 trim_cache size 0 max 16384 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "epoch": 22, 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "leader": 24506, 2024-04-22T07:15:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "set_version": 2, 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "sets": { 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "e298081e": { 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "version": 2, 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "age_ref": 0.0, 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "QUIESCED", 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "timeout": 60.0, 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "expiration": 78.8, 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "members": { 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "file:/": { 2024-04-22T07:15:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "excluded": false, 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "QUIESCED", 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:20.004 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2680a3ed0 msgr2=0x7fc2680a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2680a3ed0 0x7fc2680a6390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc278005cc0 tx=0x7fc278012960 comp rx=0 tx=0).stop 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 msgr2=0x7fc28019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28019eae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc274006010 tx=0x7fc27402ce90 comp rx=0 tx=0).stop 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 shutdown_connections 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2680a3ed0 0x7fc2680a6390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc28010c580 0x7fc280198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc280103ad0 0x7fc28019eae0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 --2- 172.21.15.113:0/474617737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2801031b0 0x7fc28019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:20.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 >> 172.21.15.113:0/474617737 conn(0x7fc2800771d0 msgr2=0x7fc2800fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:20.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 shutdown_connections 2024-04-22T07:15:20.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:20.003+0000 7fc28636d640 1 -- 172.21.15.113:0/474617737 wait complete. 2024-04-22T07:15:20.012 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: e298081e, quiesce took 0.6 seconds. Will release after: 18.85 2024-04-22T07:15:20.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:20 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3580654434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:20.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:20 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3580654434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:20.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:20 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3580654434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:21.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:21 smithi133 ceph-mon[31198]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:15:21.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:21 smithi154 ceph-mon[31862]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:15:21.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:21 smithi113 ceph-mon[27933]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:15:23.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:23 smithi133 ceph-mon[31198]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:23 smithi154 ceph-mon[31862]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:23 smithi113 ceph-mon[27933]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:25.387 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:25 smithi113 ceph-mon[27933]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-22T07:15:25.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:25 smithi133 ceph-mon[31198]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-22T07:15:25.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:25 smithi154 ceph-mon[31862]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-22T07:15:26.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:15:26.686 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:15:26.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 -- 172.21.15.113:0/2578532289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241080d0 msgr2=0x7fe1241084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 --2- 172.21.15.113:0/2578532289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241080d0 0x7fe1241084b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe114009cd0 tx=0x7fe11402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 -- 172.21.15.113:0/2578532289 shutdown_connections 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 --2- 172.21.15.113:0/2578532289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe124100e40 0x7fe12410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 --2- 172.21.15.113:0/2578532289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1241004c0 0x7fe124100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 --2- 172.21.15.113:0/2578532289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241080d0 0x7fe1241084b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 -- 172.21.15.113:0/2578532289 >> 172.21.15.113:0/2578532289 conn(0x7fe124077360 msgr2=0x7fe1240fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 -- 172.21.15.113:0/2578532289 shutdown_connections 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 -- 172.21.15.113:0/2578532289 wait complete. 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 Processor -- start 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.790+0000 7fe129938640 1 -- start start 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 0x7fe12419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe124100e40 0x7fe12419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1241080d0 0x7fe124198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241123f0 con 0x7fe1241004c0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112270 con 0x7fe1241080d0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112570 con 0x7fe124100e40 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 0x7fe12419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 0x7fe12419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57554/0 (socket says 172.21.15.113:57554) 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 -- 172.21.15.113:0/3095956355 learned_addr learned my addr 172.21.15.113:0/3095956355 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe1227fc640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe124100e40 0x7fe12419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 -- 172.21.15.113:0/3095956355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe124100e40 msgr2=0x7fe12419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe124100e40 0x7fe12419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 -- 172.21.15.113:0/3095956355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1241080d0 msgr2=0x7fe124198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1241080d0 0x7fe124198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 -- 172.21.15.113:0/3095956355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe124198f60 con 0x7fe1241004c0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe122ffd640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 0x7fe12419e620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe11402c700 tx=0x7fe114032480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe128936640 1 -- 172.21.15.113:0/3095956355 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114039040 con 0x7fe1241004c0 2024-04-22T07:15:26.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1241991f0 con 0x7fe1241004c0 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe128936640 1 -- 172.21.15.113:0/3095956355 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe114044070 con 0x7fe1241004c0 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe128936640 1 -- 172.21.15.113:0/3095956355 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114004280 con 0x7fe1241004c0 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.791+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1241a5470 con 0x7fe1241004c0 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe128936640 1 -- 172.21.15.113:0/3095956355 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fe114003970 con 0x7fe1241004c0 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 Processor -- start 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 -- start start 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4853960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559cd47b81e0 con 0x559cd485ca80 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559cd4755390 con 0x559cd4853580 2024-04-22T07:15:26.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.794+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559cd47b41a0 con 0x559cd4853ea0 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57562/0 (socket says 172.21.15.113:57562) 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 -- 172.21.15.113:0/217260990 learned_addr learned my addr 172.21.15.113:0/217260990 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe128936640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe10c0a3ed0 0x7fe10c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe122ffd640 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1237fe640 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4853960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 -- 172.21.15.113:0/217260990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 msgr2=0x559cd485c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 -- 172.21.15.113:0/217260990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 msgr2=0x559cd4853960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4853960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1237fe640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe10c0a3ed0 0x7fe10c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 -- 172.21.15.113:0/217260990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cd45947d0 con 0x559cd485ca80 2024-04-22T07:15:26.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe122ffd640 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe128936640 1 -- 172.21.15.113:0/3095956355 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe1140f2ee0 con 0x7fe1241004c0 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1227fc640 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485ee70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe118009bf0 tx=0x7fe118010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e9ce54a62b9a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe109ffb640 1 -- 172.21.15.113:0/217260990 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11801b480 con 0x559cd485ca80 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe1237fe640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe10c0a3ed0 0x7fe10c0a6390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009e10 tx=0x7fe11c0046c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe109ffb640 1 -- 172.21.15.113:0/217260990 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe11801baf0 con 0x559cd485ca80 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.795+0000 7fe109ffb640 1 -- 172.21.15.113:0/217260990 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118012b50 con 0x559cd485ca80 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 -- 172.21.15.113:0/217260990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 msgr2=0x559cd485ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485ee70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe118009bf0 tx=0x7fe118010740 comp rx=0 tx=0).stop 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 -- 172.21.15.113:0/217260990 shutdown_connections 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485ee70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/217260990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4853960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 -- 172.21.15.113:0/217260990 >> 172.21.15.113:0/217260990 conn(0x559cd484dcb0 msgr2=0x559cd484e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 -- 172.21.15.113:0/217260990 shutdown_connections 2024-04-22T07:15:26.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 -- 172.21.15.113:0/217260990 wait complete. 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 -- start start 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.796+0000 7fe12c5fd740 1 Processor -- start 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4859ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559cd485f870 con 0x559cd485ca80 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559cd47712f0 con 0x559cd4853580 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559cd485f9f0 con 0x559cd4853ea0 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 10 client.0 _ll_register_callbacks cb 0x559cd477e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57570/0 (socket says 172.21.15.113:57570) 2024-04-22T07:15:26.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 -- 172.21.15.113:0/1799916101 learned_addr learned my addr 172.21.15.113:0/1799916101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1227fc640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe122ffd640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4859ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 -- 172.21.15.113:0/1799916101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 msgr2=0x559cd485a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 -- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 msgr2=0x559cd4859ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4859ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 -- 172.21.15.113:0/1799916101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cd44ca3f0 con 0x559cd485ca80 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe122ffd640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4859ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe1237fe640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485a550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe11c003870 tx=0x7fe11c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fc566239277662 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe102ffd640 1 -- 172.21.15.113:0/1799916101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c023020 con 0x559cd485ca80 2024-04-22T07:15:26.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe102ffd640 1 -- 172.21.15.113:0/1799916101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe11c010ba0 con 0x559cd485ca80 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.797+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11801baf0 con 0x559cd485ca80 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe102ffd640 1 -- 172.21.15.113:0/1799916101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe11c02e050 con 0x559cd485ca80 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 10 client.15735 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559cd448a0f0 con 0x559cd485ca80 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe102ffd640 1 -- 172.21.15.113:0/1799916101 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe11801baf0 con 0x559cd485ca80 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe102ffd640 1 -- 172.21.15.113:0/1799916101 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe11c00f3c0 con 0x559cd485ca80 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 10 client.15735 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 20 client.15735 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd486cab0 0x559cd486ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 4 client.15735 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cd47c5c00 con 0x559cd486cab0 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.798+0000 7fe122ffd640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd486cab0 0x559cd486ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe122ffd640 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd486cab0 0x559cd486ef00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20605530a7711607 server_cookie=2340cdadd9fbe41c in_seq=0 out_seq=0 2024-04-22T07:15:26.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe102ffd640 10 client.15735 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe102ffd640 1 -- 172.21.15.113:0/1799916101 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559cd47c5c00 con 0x559cd486cab0 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe102ffd640 10 client.15735 handle_command_reply: tid=0 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 client.15735 shutdown 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 msgr2=0x559cd485a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485a550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe11c003870 tx=0x7fe11c005f90 comp rx=0 tx=0).stop 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 shutdown_connections 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd486cab0 0x559cd486ef00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd485a550 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853ea0 0x559cd485a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe102ffd640 0 client.15735 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/1799916101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853580 0x559cd4859ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.799+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 >> 172.21.15.113:0/1799916101 conn(0x559cd484dcb0 msgr2=0x559cd484fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.800+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 shutdown_connections 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.800+0000 7fe12c5fd740 1 -- 172.21.15.113:0/1799916101 wait complete. 2024-04-22T07:15:26.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.800+0000 7fe12c5fd740 20 client.15735 trim_cache size 0 max 16384 2024-04-22T07:15:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 Processor -- start 2024-04-22T07:15:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 -- start start 2024-04-22T07:15:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 0x559cd485c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 0x559cd4868730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd485ca80 0x559cd484e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241b04c0 con 0x559cd4853ea0 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241add90 con 0x559cd485ca80 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241abbe0 con 0x559cd4853580 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 0x559cd4868730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 0x559cd4868730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57580/0 (socket says 172.21.15.113:57580) 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 -- 172.21.15.113:0/225810581 learned_addr learned my addr 172.21.15.113:0/225810581 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 -- 172.21.15.113:0/225810581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 msgr2=0x559cd485c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 0x559cd485c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 -- 172.21.15.113:0/225810581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd485ca80 msgr2=0x559cd484e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd485ca80 0x559cd484e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 -- 172.21.15.113:0/225810581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cd4853cd0 con 0x559cd4853ea0 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.817+0000 7fe1227fc640 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 0x559cd4868730 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe118009bf0 tx=0x7fe118012570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bcee9c82b8bd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe1097fa640 1 -- 172.21.15.113:0/225810581 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1180095b0 con 0x559cd4853ea0 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe1097fa640 1 -- 172.21.15.113:0/225810581 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe118005510 con 0x559cd4853ea0 2024-04-22T07:15:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe1097fa640 1 -- 172.21.15.113:0/225810581 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1180035f0 con 0x559cd4853ea0 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 -- 172.21.15.113:0/225810581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 msgr2=0x559cd4868730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 0x559cd4868730 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe118009bf0 tx=0x7fe118012570 comp rx=0 tx=0).stop 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 -- 172.21.15.113:0/225810581 shutdown_connections 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd485ca80 0x559cd484e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd4853ea0 0x559cd4868730 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/225810581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 0x559cd485c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 -- 172.21.15.113:0/225810581 >> 172.21.15.113:0/225810581 conn(0x559cd484dcb0 msgr2=0x559cd484c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 -- 172.21.15.113:0/225810581 shutdown_connections 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 -- 172.21.15.113:0/225810581 wait complete. 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.818+0000 7fe12c5fd740 1 -- start start 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 Processor -- start 2024-04-22T07:15:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 0x559cd4856840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853ea0 0x559cd4856d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd48572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ac250 con 0x559cd485ca80 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ad6c0 con 0x559cd4853ea0 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ac920 con 0x559cd4853580 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe12c5fd740 10 client.0 _ll_register_callbacks cb 0x7fe11c0099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd48572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd48572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57584/0 (socket says 172.21.15.113:57584) 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 -- 172.21.15.113:0/2696164489 learned_addr learned my addr 172.21.15.113:0/2696164489 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1227fc640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853ea0 0x559cd4856d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 -- 172.21.15.113:0/2696164489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 msgr2=0x559cd4856840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 0x559cd4856840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 -- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853ea0 msgr2=0x559cd4856d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853ea0 0x559cd4856d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 -- 172.21.15.113:0/2696164489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cd48d20f0 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1227fc640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853ea0 0x559cd4856d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe1237fe640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd48572c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00f860 tx=0x7fe11c00fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88317261abd54bed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.819+0000 7fe102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe102ffd640 1 -- 172.21.15.113:0/2696164489 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c023070 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe102ffd640 1 -- 172.21.15.113:0/2696164489 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fe11c0107f0 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe118005510 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe102ffd640 1 -- 172.21.15.113:0/2696164489 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe11c042050 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 10 client.15741 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559cd48d33e0 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe102ffd640 1 -- 172.21.15.113:0/2696164489 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe118005510 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe102ffd640 1 -- 172.21.15.113:0/2696164489 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe11c03b8f0 con 0x559cd485ca80 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 10 client.15741 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 20 client.15741 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd48d3b30 0x559cd48d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 4 client.15741 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cd47bfa70 con 0x559cd48d3b30 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.820+0000 7fe122ffd640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd48d3b30 0x559cd48d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe102ffd640 10 client.15741 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe122ffd640 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd48d3b30 0x559cd48d5f50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9beae47444fc2e3 server_cookie=451eceb8beb17854 in_seq=0 out_seq=0 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe102ffd640 1 -- 172.21.15.113:0/2696164489 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559cd47bfa70 con 0x559cd48d3b30 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe102ffd640 10 client.15741 handle_command_reply: tid=0 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 client.15741 shutdown 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 msgr2=0x559cd48572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd48572c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00f860 tx=0x7fe11c00fd00 comp rx=0 tx=0).stop 2024-04-22T07:15:26.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 shutdown_connections 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559cd48d3b30 0x559cd48d5f50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559cd485ca80 0x559cd48572c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe102ffd640 0 client.15741 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cd4853ea0 0x559cd4856d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 --2- 172.21.15.113:0/2696164489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559cd4853580 0x559cd4856840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 >> 172.21.15.113:0/2696164489 conn(0x559cd484dcb0 msgr2=0x559cd484cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 shutdown_connections 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 1 -- 172.21.15.113:0/2696164489 wait complete. 2024-04-22T07:15:26.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.821+0000 7fe12c5fd740 20 client.15741 trim_cache size 0 max 16384 2024-04-22T07:15:26.827 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:26.827 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:15:26.827 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:15:26.827 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe10c0a3ed0 msgr2=0x7fe10c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe10c0a3ed0 0x7fe10c0a6390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009e10 tx=0x7fe11c0046c0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 msgr2=0x7fe12419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 0x7fe12419e620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe11402c700 tx=0x7fe114032480 comp rx=0 tx=0).stop 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 shutdown_connections 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe10c0a3ed0 0x7fe10c0a6390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1241080d0 0x7fe124198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe124100e40 0x7fe12419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 --2- 172.21.15.113:0/3095956355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1241004c0 0x7fe12419e620 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 >> 172.21.15.113:0/3095956355 conn(0x7fe124077360 msgr2=0x7fe124104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 shutdown_connections 2024-04-22T07:15:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.822+0000 7fe129938640 1 -- 172.21.15.113:0/3095956355 wait complete. 2024-04-22T07:15:26.830 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:15:26.830 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 -- 172.21.15.113:0/1179908764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 msgr2=0x7ff9fc101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/1179908764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc101590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009900 tx=0x7ff9f002bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 -- 172.21.15.113:0/1179908764 shutdown_connections 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/1179908764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc101ad0 0x7ff9fc10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/1179908764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc101590 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/1179908764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc106f10 0x7ff9fc1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 -- 172.21.15.113:0/1179908764 >> 172.21.15.113:0/1179908764 conn(0x7ff9fc077e30 msgr2=0x7ff9fc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 -- 172.21.15.113:0/1179908764 shutdown_connections 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.931+0000 7ffa01eb3640 1 -- 172.21.15.113:0/1179908764 wait complete. 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 Processor -- start 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 -- start start 2024-04-22T07:15:26.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc101ad0 0x7ff9fc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc106f10 0x7ff9fc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc112380 con 0x7ff9fc101130 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc112200 con 0x7ff9fc106f10 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ffa01eb3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc112500 con 0x7ff9fc101ad0 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ff9fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ff9fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57610/0 (socket says 172.21.15.113:57610) 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.932+0000 7ff9fb7fe640 1 -- 172.21.15.113:0/4209270321 learned_addr learned my addr 172.21.15.113:0/4209270321 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9faffd640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc101ad0 0x7ff9fc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fbfff640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc106f10 0x7ff9fc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fb7fe640 1 -- 172.21.15.113:0/4209270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc101ad0 msgr2=0x7ff9fc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fb7fe640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc101ad0 0x7ff9fc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fb7fe640 1 -- 172.21.15.113:0/4209270321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc106f10 msgr2=0x7ff9fc198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fb7fe640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc106f10 0x7ff9fc198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fb7fe640 1 -- 172.21.15.113:0/4209270321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc198fe0 con 0x7ff9fc101130 2024-04-22T07:15:26.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9faffd640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc101ad0 0x7ff9fc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fbfff640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc106f10 0x7ff9fc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9fb7fe640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc19e610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00b4a0 tx=0x7ff9ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec014070 con 0x7ff9fc101130 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff9ec010ae0 con 0x7ff9fc101130 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc1992d0 con 0x7ff9fc101130 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec017ce0 con 0x7ff9fc101130 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.933+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc1a5590 con 0x7ff9fc101130 2024-04-22T07:15:26.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.934+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9fc1025f0 con 0x7ff9fc101130 2024-04-22T07:15:26.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.935+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7ff9ec017070 con 0x7ff9fc101130 2024-04-22T07:15:26.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.936+0000 7ff9f8ff9640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff9e40a3ed0 0x7ff9e40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:26.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.936+0000 7ff9faffd640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff9e40a3ed0 0x7ff9e40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:26.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.936+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff9ec0d01d0 con 0x7ff9fc101130 2024-04-22T07:15:26.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.936+0000 7ff9faffd640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff9e40a3ed0 0x7ff9e40a6390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc199f20 tx=0x7ff9f002c7b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:26.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:26.939+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9ec09c7b0 con 0x7ff9fc101130 2024-04-22T07:15:27.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.156+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9fc069230 con 0x7ff9fc101130 2024-04-22T07:15:27.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.157+0000 7ff9f8ff9640 1 -- 172.21.15.113:0/4209270321 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7ff9fc069230 con 0x7ff9fc101130 2024-04-22T07:15:27.155 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:27.155 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:27.155 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff9e40a3ed0 msgr2=0x7ff9e40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff9e40a3ed0 0x7ff9e40a6390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc199f20 tx=0x7ff9f002c7b0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 msgr2=0x7ff9fc19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc19e610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00b4a0 tx=0x7ff9ec00b960 comp rx=0 tx=0).stop 2024-04-22T07:15:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 shutdown_connections 2024-04-22T07:15:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff9e40a3ed0 0x7ff9e40a6390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc106f10 0x7ff9fc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc101ad0 0x7ff9fc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 --2- 172.21.15.113:0/4209270321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9fc101130 0x7ff9fc19e610 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 >> 172.21.15.113:0/4209270321 conn(0x7ff9fc077e30 msgr2=0x7ff9fc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 shutdown_connections 2024-04-22T07:15:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.159+0000 7ffa01eb3640 1 -- 172.21.15.113:0/4209270321 wait complete. 2024-04-22T07:15:27.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.272+0000 7fa07a1f4640 1 -- 172.21.15.113:0/944817701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 msgr2=0x7fa07410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.272+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/944817701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa07410dd20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009ca0 tx=0x7fa06c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- 172.21.15.113:0/944817701 shutdown_connections 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/944817701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa07410dd20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/944817701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0741027e0 0x7fa074106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/944817701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa074101e30 0x7fa074102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- 172.21.15.113:0/944817701 >> 172.21.15.113:0/944817701 conn(0x7fa074076150 msgr2=0x7fa074076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- 172.21.15.113:0/944817701 shutdown_connections 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- 172.21.15.113:0/944817701 wait complete. 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 Processor -- start 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- start start 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074101e30 0x7fa0741a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa0741a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa07419a8a0 0x7fa07419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741111f0 con 0x7fa0741072f0 2024-04-22T07:15:27.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa074111070 con 0x7fa07419a8a0 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa07a1f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa074111370 con 0x7fa074101e30 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa0741a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa0741a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57640/0 (socket says 172.21.15.113:57640) 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.273+0000 7fa06bfff640 1 -- 172.21.15.113:0/1816864883 learned_addr learned my addr 172.21.15.113:0/1816864883 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa0737fe640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074101e30 0x7fa0741a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa06bfff640 1 -- 172.21.15.113:0/1816864883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074101e30 msgr2=0x7fa0741a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074101e30 0x7fa0741a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa073fff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa07419a8a0 0x7fa07419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa06bfff640 1 -- 172.21.15.113:0/1816864883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa07419a8a0 msgr2=0x7fa07419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa07419a8a0 0x7fa07419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa06bfff640 1 -- 172.21.15.113:0/1816864883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07419b3c0 con 0x7fa0741072f0 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa073fff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa07419a8a0 0x7fa07419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:27.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa0741a0c30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d740 tx=0x7fa05c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa0717fa640 1 -- 172.21.15.113:0/1816864883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c017070 con 0x7fa0741072f0 2024-04-22T07:15:27.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa0717fa640 1 -- 172.21.15.113:0/1816864883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa05c00de50 con 0x7fa0741072f0 2024-04-22T07:15:27.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07419b6b0 con 0x7fa0741072f0 2024-04-22T07:15:27.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa0717fa640 1 -- 172.21.15.113:0/1816864883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c004d90 con 0x7fa0741072f0 2024-04-22T07:15:27.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.274+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741a78b0 con 0x7fa0741072f0 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa0717fa640 1 -- 172.21.15.113:0/1816864883 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fa05c004f30 con 0x7fa0741072f0 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 Processor -- start 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 -- start start 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd164d0 0x563b6dd168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd16e80 0x563b6dd1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd21e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563b6dc7b1f0 con 0x563b6dd1fa00 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563b6dc184b0 con 0x563b6dd164d0 2024-04-22T07:15:27.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa07cde6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563b6dc771b0 con 0x563b6dd16e80 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd21e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd21e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57646/0 (socket says 172.21.15.113:57646) 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa06bfff640 1 -- 172.21.15.113:0/1250457083 learned_addr learned my addr 172.21.15.113:0/1250457083 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa0717fa640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa0600a3ed0 0x7fa0600a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa073fff640 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd164d0 0x563b6dd168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.277+0000 7fa0737fe640 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd16e80 0x563b6dd1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa073fff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa0600a3ed0 0x7fa0600a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa0717fa640 1 -- 172.21.15.113:0/1816864883 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa05c0d04c0 con 0x7fa0741072f0 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa06bfff640 1 -- 172.21.15.113:0/1250457083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd16e80 msgr2=0x563b6dd1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd16e80 0x563b6dd1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa06bfff640 1 -- 172.21.15.113:0/1250457083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd164d0 msgr2=0x563b6dd168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd164d0 0x563b6dd168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa06bfff640 1 -- 172.21.15.113:0/1250457083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6dc0a1b0 con 0x563b6dd1fa00 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa0737fe640 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd16e80 0x563b6dd1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:27.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd21e50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa05c09bf50 tx=0x7fa05c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a4f09096cad15d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa069ffb640 1 -- 172.21.15.113:0/1250457083 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c020070 con 0x563b6dd1fa00 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa073fff640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa0600a3ed0 0x7fa0600a6390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa06c004af0 tx=0x7fa06c02c0f0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa069ffb640 1 -- 172.21.15.113:0/1250457083 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa05c0a0600 con 0x563b6dd1fa00 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.278+0000 7fa069ffb640 1 -- 172.21.15.113:0/1250457083 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c097fa0 con 0x563b6dd1fa00 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- 172.21.15.113:0/1250457083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 msgr2=0x563b6dd21e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd21e50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa05c09bf50 tx=0x7fa05c009c30 comp rx=0 tx=0).stop 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- 172.21.15.113:0/1250457083 shutdown_connections 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd21e50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd16e80 0x563b6dd1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1250457083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd164d0 0x563b6dd168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- 172.21.15.113:0/1250457083 >> 172.21.15.113:0/1250457083 conn(0x563b6dd10ba0 msgr2=0x563b6dd10f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- 172.21.15.113:0/1250457083 shutdown_connections 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- 172.21.15.113:0/1250457083 wait complete. 2024-04-22T07:15:27.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- start start 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 Processor -- start 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd1f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd1c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd1cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.279+0000 7fa07cde6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563b6dd22910 con 0x563b6dd1fa00 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa07cde6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563b6dc34500 con 0x563b6dd16e80 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa07cde6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563b6dd22a90 con 0x563b6dd164d0 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa07cde6740 10 client.0 _ll_register_callbacks cb 0x563b6dc02b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd1cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd1cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57648/0 (socket says 172.21.15.113:57648) 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 -- 172.21.15.113:0/1197532401 learned_addr learned my addr 172.21.15.113:0/1197532401 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa0737fe640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd1f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa06bfff640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd1c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 -- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 msgr2=0x563b6dd1f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd1f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 -- 172.21.15.113:0/1197532401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 msgr2=0x563b6dd1c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd1c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 -- 172.21.15.113:0/1197532401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6d98d3f0 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa073fff640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd1cd20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa06c036000 tx=0x7fa06c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8056a21b6e381e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa052ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa052ffd640 1 -- 172.21.15.113:0/1197532401 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c039030 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa052ffd640 1 -- 172.21.15.113:0/1197532401 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa06c004540 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c0a0600 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa052ffd640 1 -- 172.21.15.113:0/1197532401 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa06c05b050 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa07cde6740 10 client.15759 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.280+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563b6da577d0 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa052ffd640 1 -- 172.21.15.113:0/1197532401 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa05c0a0600 con 0x563b6dd1fa00 2024-04-22T07:15:27.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa052ffd640 1 -- 172.21.15.113:0/1197532401 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa06c034c20 con 0x563b6dd1fa00 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa07cde6740 10 client.15759 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa07cde6740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa07cde6740 20 client.15759 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd2fba0 0x563b6dd31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa07cde6740 4 client.15759 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b6dc1a930 con 0x563b6dd2fba0 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa0737fe640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd2fba0 0x563b6dd31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa052ffd640 10 client.15759 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.281+0000 7fa0737fe640 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd2fba0 0x563b6dd31fd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df9986aab24de447 server_cookie=f0ef9adbc18520df in_seq=0 out_seq=0 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa052ffd640 1 -- 172.21.15.113:0/1197532401 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b6dc1a930 con 0x563b6dd2fba0 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa052ffd640 10 client.15759 handle_command_reply: tid=0 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 client.15759 shutdown 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 msgr2=0x563b6dd1cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd1cd20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa06c036000 tx=0x7fa06c0095c0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 shutdown_connections 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd2fba0 0x563b6dd31fd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd1fa00 0x563b6dd1cd20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd1c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa052ffd640 0 client.15759 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 --2- 172.21.15.113:0/1197532401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd1f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 >> 172.21.15.113:0/1197532401 conn(0x563b6dd10ba0 msgr2=0x563b6dd128f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 shutdown_connections 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.282+0000 7fa07cde6740 1 -- 172.21.15.113:0/1197532401 wait complete. 2024-04-22T07:15:27.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.283+0000 7fa07cde6740 20 client.15759 trim_cache size 0 max 16384 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.295+0000 7fa07cde6740 1 Processor -- start 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.295+0000 7fa07cde6740 1 -- start start 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd2b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 0x563b6dd1ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd1fa00 0x563b6dd1f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa07cde6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741b2770 con 0x563b6dd16e80 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa07cde6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741b0040 con 0x563b6dd1fa00 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa07cde6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741ade90 con 0x563b6dd164d0 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 0x563b6dd1ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 0x563b6dd1ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57658/0 (socket says 172.21.15.113:57658) 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 -- 172.21.15.113:0/253665545 learned_addr learned my addr 172.21.15.113:0/253665545 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 -- 172.21.15.113:0/253665545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 msgr2=0x563b6dd2b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd2b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 -- 172.21.15.113:0/253665545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd1fa00 msgr2=0x563b6dd1f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd1fa00 0x563b6dd1f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 -- 172.21.15.113:0/253665545 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6dd103f0 con 0x563b6dd16e80 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa06bfff640 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 0x563b6dd1ee40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0a4990 tx=0x7fa05c005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6aa96915886203 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa0697fa640 1 -- 172.21.15.113:0/253665545 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c020040 con 0x563b6dd16e80 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa0697fa640 1 -- 172.21.15.113:0/253665545 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa05c09c250 con 0x563b6dd16e80 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.296+0000 7fa0697fa640 1 -- 172.21.15.113:0/253665545 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0d02a0 con 0x563b6dd16e80 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 -- 172.21.15.113:0/253665545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 msgr2=0x563b6dd1ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 0x563b6dd1ee40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0a4990 tx=0x7fa05c005af0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 -- 172.21.15.113:0/253665545 shutdown_connections 2024-04-22T07:15:27.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd1fa00 0x563b6dd1f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd16e80 0x563b6dd1ee40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- 172.21.15.113:0/253665545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd164d0 0x563b6dd2b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 -- 172.21.15.113:0/253665545 >> 172.21.15.113:0/253665545 conn(0x563b6dd10ba0 msgr2=0x563b6dd0fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 -- 172.21.15.113:0/253665545 shutdown_connections 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 -- 172.21.15.113:0/253665545 wait complete. 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 -- start start 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 Processor -- start 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 0x563b6dd194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.297+0000 7fa07cde6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd1fa00 0x563b6dd19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741ae500 con 0x563b6dd164d0 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741af970 con 0x563b6dd16e80 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0741aebd0 con 0x563b6dd1fa00 2024-04-22T07:15:27.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 10 client.0 _ll_register_callbacks cb 0x7fa06c032a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 0x563b6dd194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 0x563b6dd194b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57670/0 (socket says 172.21.15.113:57670) 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 -- 172.21.15.113:0/2274305018 learned_addr learned my addr 172.21.15.113:0/2274305018 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa073fff640 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd1fa00 0x563b6dd19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 -- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd1fa00 msgr2=0x563b6dd19f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd1fa00 0x563b6dd19f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 -- 172.21.15.113:0/2274305018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 msgr2=0x563b6dd199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 -- 172.21.15.113:0/2274305018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6dd95230 con 0x563b6dd164d0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa0737fe640 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 0x563b6dd194b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x563b6dd17280 tx=0x7fa0640044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34b13393e8cc039 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa052ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa052ffd640 1 -- 172.21.15.113:0/2274305018 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064022020 con 0x563b6dd164d0 2024-04-22T07:15:27.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa052ffd640 1 -- 172.21.15.113:0/2274305018 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fa06401e620 con 0x563b6dd164d0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c09c250 con 0x563b6dd164d0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa052ffd640 1 -- 172.21.15.113:0/2274305018 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa06402d050 con 0x563b6dd164d0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 10 client.15765 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563b6dd96590 con 0x563b6dd164d0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.298+0000 7fa052ffd640 1 -- 172.21.15.113:0/2274305018 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa05c09c250 con 0x563b6dd164d0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa052ffd640 1 -- 172.21.15.113:0/2274305018 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa064011440 con 0x563b6dd164d0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 10 client.15765 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 20 client.15765 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd96d60 0x563b6dd99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 4 client.15765 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 --> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b6dc88c10 con 0x563b6dd96d60 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa06bfff640 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd96d60 0x563b6dd99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa052ffd640 10 client.15765 ms_handle_connect on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:27.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa06bfff640 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd96d60 0x563b6dd99180 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d1bbb837afadacc server_cookie=3469bb38415ec0e5 in_seq=0 out_seq=0 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa052ffd640 1 -- 172.21.15.113:0/2274305018 <== mds.2 v2:172.21.15.154:6838/3404626603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b6dc88c10 con 0x563b6dd96d60 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa052ffd640 10 client.15765 handle_command_reply: tid=0 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.299+0000 7fa07cde6740 1 client.15765 shutdown 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 msgr2=0x563b6dd194b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 0x563b6dd194b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x563b6dd17280 tx=0x7fa0640044d0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 shutdown_connections 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:6838/3404626603,v1:172.21.15.154:6839/3404626603] conn(0x563b6dd96d60 0x563b6dd99180 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563b6dd1fa00 0x563b6dd19f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b6dd16e80 0x563b6dd199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 --2- 172.21.15.113:0/2274305018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563b6dd164d0 0x563b6dd194b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa052ffd640 0 client.15765 ms_handle_reset on v2:172.21.15.154:6838/3404626603 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 >> 172.21.15.113:0/2274305018 conn(0x563b6dd10ba0 msgr2=0x563b6dd0fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 shutdown_connections 2024-04-22T07:15:27.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 1 -- 172.21.15.113:0/2274305018 wait complete. 2024-04-22T07:15:27.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07cde6740 20 client.15765 trim_cache size 0 max 16384 2024-04-22T07:15:27.305 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa0600a3ed0 msgr2=0x7fa0600a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa0600a3ed0 0x7fa0600a6390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa06c004af0 tx=0x7fa06c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 msgr2=0x7fa0741a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa0741a0c30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d740 tx=0x7fa05c00dc00 comp rx=0 tx=0).stop 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 shutdown_connections 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa0600a3ed0 0x7fa0600a6390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa07419a8a0 0x7fa07419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0741072f0 0x7fa0741a0c30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 --2- 172.21.15.113:0/1816864883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa074101e30 0x7fa0741a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 >> 172.21.15.113:0/1816864883 conn(0x7fa074076150 msgr2=0x7fa074100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 shutdown_connections 2024-04-22T07:15:27.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.300+0000 7fa07a1f4640 1 -- 172.21.15.113:0/1816864883 wait complete. 2024-04-22T07:15:27.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:15:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 -- 172.21.15.113:0/3984195513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a4102b40 msgr2=0x7f92a4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 --2- 172.21.15.113:0/3984195513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a4102b40 0x7f92a4106b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f929402bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 -- 172.21.15.113:0/3984195513 shutdown_connections 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 --2- 172.21.15.113:0/3984195513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a4107260 0x7f92a410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 --2- 172.21.15.113:0/3984195513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a4102b40 0x7f92a4106b80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 --2- 172.21.15.113:0/3984195513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a4102190 0x7f92a4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 -- 172.21.15.113:0/3984195513 >> 172.21.15.113:0/3984195513 conn(0x7f92a40793a0 msgr2=0x7f92a40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 -- 172.21.15.113:0/3984195513 shutdown_connections 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.411+0000 7f92ab75f640 1 -- 172.21.15.113:0/3984195513 wait complete. 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 Processor -- start 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 -- start start 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a4102190 0x7f92a4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a4107260 0x7f92a4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 0x7f92a419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41122c0 con 0x7f92a419d000 2024-04-22T07:15:27.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4112140 con 0x7f92a4107260 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92ab75f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4112440 con 0x7f92a4102190 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 0x7f92a419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 0x7f92a419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57682/0 (socket says 172.21.15.113:57682) 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 -- 172.21.15.113:0/370803057 learned_addr learned my addr 172.21.15.113:0/370803057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a94fb640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a4102190 0x7f92a4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a4107260 0x7f92a4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 -- 172.21.15.113:0/370803057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a4102190 msgr2=0x7f92a4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a4102190 0x7f92a4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 -- 172.21.15.113:0/370803057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a4107260 msgr2=0x7f92a4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a4107260 0x7f92a4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.412+0000 7f92a9cfc640 1 -- 172.21.15.113:0/370803057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a419daf0 con 0x7f92a419d000 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.413+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 0x7f92a419d4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f929c00cce0 tx=0x7f929c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.413+0000 7f929a7fc640 1 -- 172.21.15.113:0/370803057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c00f450 con 0x7f92a419d000 2024-04-22T07:15:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.413+0000 7f929a7fc640 1 -- 172.21.15.113:0/370803057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f929c00fac0 con 0x7f92a419d000 2024-04-22T07:15:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.413+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a419dde0 con 0x7f92a419d000 2024-04-22T07:15:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.413+0000 7f929a7fc640 1 -- 172.21.15.113:0/370803057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c018b20 con 0x7f92a419d000 2024-04-22T07:15:27.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.413+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41a5550 con 0x7f92a419d000 2024-04-22T07:15:27.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.415+0000 7f929a7fc640 1 -- 172.21.15.113:0/370803057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f929c00fc70 con 0x7f92a419d000 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.415+0000 7f92ae42a740 1 Processor -- start 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.415+0000 7f92ae42a740 1 -- start start 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f929a7fc640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f92900a3ed0 0x7f92900a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23695a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236963890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f2369642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92ae42a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f2368bf1f0 con 0x55f23695a3b0 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92ae42a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f23685c4b0 con 0x55f236963e60 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92ae42a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f2368bb1b0 con 0x55f23695ad60 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23695a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f92900a3ed0 0x7f92900a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23695a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57684/0 (socket says 172.21.15.113:57684) 2024-04-22T07:15:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f929a7fc640 1 -- 172.21.15.113:0/370803057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f929c0d0ec0 con 0x7f92a419d000 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 -- 172.21.15.113:0/1187065924 learned_addr learned my addr 172.21.15.113:0/1187065924 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f2369642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236963890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 -- 172.21.15.113:0/1187065924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 msgr2=0x55f236963890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236963890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 -- 172.21.15.113:0/1187065924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 msgr2=0x55f2369642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f2369642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 -- 172.21.15.113:0/1187065924 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f23684e1b0 con 0x55f23695a3b0 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236963890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a94fb640 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23695a790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f92a000ad00 tx=0x7f92a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7976ac66d6884f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f92900a3ed0 0x7f92900a6390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9294006010 tx=0x7f929402c9b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f928f7fe640 1 -- 172.21.15.113:0/1187065924 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a001d070 con 0x55f23695a3b0 2024-04-22T07:15:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f928f7fe640 1 -- 172.21.15.113:0/1187065924 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f92a0018b10 con 0x55f23695a3b0 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.416+0000 7f928f7fe640 1 -- 172.21.15.113:0/1187065924 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0010c40 con 0x55f23695a3b0 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 -- 172.21.15.113:0/1187065924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 msgr2=0x55f23695a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23695a790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f92a000ad00 tx=0x7f92a000f790 comp rx=0 tx=0).stop 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 -- 172.21.15.113:0/1187065924 shutdown_connections 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f2369642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236963890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 --2- 172.21.15.113:0/1187065924 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23695a790 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 -- 172.21.15.113:0/1187065924 >> 172.21.15.113:0/1187065924 conn(0x55f236954ba0 msgr2=0x55f236954f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 -- 172.21.15.113:0/1187065924 shutdown_connections 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 -- 172.21.15.113:0/1187065924 wait complete. 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.417+0000 7f92ae42a740 1 -- start start 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 Processor -- start 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23696a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236967600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236967b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f23696ada0 con 0x55f23695a3b0 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f236878500 con 0x55f236963e60 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f23696af20 con 0x55f23695ad60 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92ae42a740 10 client.0 _ll_register_callbacks cb 0x55f236846b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23696a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23696a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57696/0 (socket says 172.21.15.113:57696) 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 -- 172.21.15.113:0/986885096 learned_addr learned my addr 172.21.15.113:0/986885096 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236967b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236967600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 -- 172.21.15.113:0/986885096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 msgr2=0x55f236967600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236967600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 -- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 msgr2=0x55f236967b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236967b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.418+0000 7f92a94fb640 1 -- 172.21.15.113:0/986885096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2365d13f0 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92a94fb640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23696a270 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f92a00107e0 tx=0x7f92a0011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229a004e4eaca756 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f928cff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f928cff9640 1 -- 172.21.15.113:0/986885096 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a001d050 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f928cff9640 1 -- 172.21.15.113:0/986885096 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f92a0018640 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a0018b10 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f928cff9640 1 -- 172.21.15.113:0/986885096 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f92a0032050 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 10 client.15777 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f23669b7d0 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f928cff9640 1 -- 172.21.15.113:0/986885096 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f92a0018b10 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f928cff9640 1 -- 172.21.15.113:0/986885096 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f92a00046c0 con 0x55f23695a3b0 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 10 client.15777 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:15:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 20 client.15777 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f236977ed0 0x55f23697a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 4 client.15777 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.419+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f23685e930 con 0x55f236977ed0 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.420+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f236977ed0 0x55f23697a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.420+0000 7f928cff9640 10 client.15777 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.420+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f236977ed0 0x55f23697a2c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eced6461f701313 server_cookie=65f9e523da7f179b in_seq=0 out_seq=0 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f928cff9640 1 -- 172.21.15.113:0/986885096 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f23685e930 con 0x55f236977ed0 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f928cff9640 10 client.15777 handle_command_reply: tid=0 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 client.15777 shutdown 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 msgr2=0x55f23696a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23696a270 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f92a00107e0 tx=0x7f92a0011b80 comp rx=0 tx=0).stop 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 shutdown_connections 2024-04-22T07:15:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f236977ed0 0x55f23697a2c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236967b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f928cff9640 0 client.15777 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695ad60 0x55f236967600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 --2- 172.21.15.113:0/986885096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695a3b0 0x55f23696a270 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 >> 172.21.15.113:0/986885096 conn(0x55f236954ba0 msgr2=0x55f2369568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 shutdown_connections 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 1 -- 172.21.15.113:0/986885096 wait complete. 2024-04-22T07:15:27.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.421+0000 7f92ae42a740 20 client.15777 trim_cache size 0 max 16384 2024-04-22T07:15:27.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 Processor -- start 2024-04-22T07:15:27.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 -- start start 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695a3b0 0x55f236973ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236966760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236965250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41ae1b0 con 0x55f23695ad60 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41aba80 con 0x55f236963e60 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92ae42a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41a98d0 con 0x55f23695a3b0 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92a8cfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236966760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92a8cfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236966760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57702/0 (socket says 172.21.15.113:57702) 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.434+0000 7f92a8cfa640 1 -- 172.21.15.113:0/3004167207 learned_addr learned my addr 172.21.15.113:0/3004167207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f92a8cfa640 1 -- 172.21.15.113:0/3004167207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695a3b0 msgr2=0x55f236973ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695a3b0 0x55f236973ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f92a8cfa640 1 -- 172.21.15.113:0/3004167207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 msgr2=0x55f236965250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236965250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f92a8cfa640 1 -- 172.21.15.113:0/3004167207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f23696af10 con 0x55f23695ad60 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236966760 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f92a406b820 tx=0x7f9294044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f479ef5513927c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f928effd640 1 -- 172.21.15.113:0/3004167207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929405e070 con 0x55f23695ad60 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f928effd640 1 -- 172.21.15.113:0/3004167207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9294004490 con 0x55f23695ad60 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.435+0000 7f928effd640 1 -- 172.21.15.113:0/3004167207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294034830 con 0x55f23695ad60 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- 172.21.15.113:0/3004167207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 msgr2=0x55f236966760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236966760 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f92a406b820 tx=0x7f9294044000 comp rx=0 tx=0).stop 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- 172.21.15.113:0/3004167207 shutdown_connections 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f236963e60 0x55f236965250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236966760 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- 172.21.15.113:0/3004167207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23695a3b0 0x55f236973ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- 172.21.15.113:0/3004167207 >> 172.21.15.113:0/3004167207 conn(0x55f236954ba0 msgr2=0x55f236955f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- 172.21.15.113:0/3004167207 shutdown_connections 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- 172.21.15.113:0/3004167207 wait complete. 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- start start 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 Processor -- start 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f23695a3b0 0x55f236970720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236964840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f236963e60 0x55f236964f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41a9f40 con 0x55f23695ad60 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41ab3b0 con 0x55f23695a3b0 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41aa610 con 0x55f236963e60 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92ae42a740 10 client.0 _ll_register_callbacks cb 0x7f92a000ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.436+0000 7f92a8cfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236964840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236964840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57712/0 (socket says 172.21.15.113:57712) 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 -- 172.21.15.113:0/656537762 learned_addr learned my addr 172.21.15.113:0/656537762 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f236963e60 0x55f236964f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 -- 172.21.15.113:0/656537762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f236963e60 msgr2=0x55f236964f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f236963e60 0x55f236964f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 -- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f23695a3b0 msgr2=0x55f236970720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a94fb640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f23695a3b0 0x55f236970720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f23695a3b0 0x55f236970720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 -- 172.21.15.113:0/656537762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2369dd100 con 0x55f23695ad60 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a9cfc640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f236963e60 0x55f236964f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92a8cfa640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236964840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f92940096c0 tx=0x7f9294003750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bcb13e633e9de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f928cff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f928cff9640 1 -- 172.21.15.113:0/656537762 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929405e030 con 0x55f23695ad60 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9294004490 con 0x55f23695ad60 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f928cff9640 1 -- 172.21.15.113:0/656537762 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f9294054040 con 0x55f23695ad60 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f928cff9640 1 -- 172.21.15.113:0/656537762 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9294003a00 con 0x55f23695ad60 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92ae42a740 10 client.15783 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:27.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f2369de2c0 con 0x55f23695ad60 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.437+0000 7f928cff9640 1 -- 172.21.15.113:0/656537762 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9294004490 con 0x55f23695ad60 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f928cff9640 1 -- 172.21.15.113:0/656537762 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f92940654c0 con 0x55f23695ad60 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 10 client.15783 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 20 client.15783 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f2369de9e0 0x55f2369e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 4 client.15783 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 --> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2368ccc10 con 0x55f2369de9e0 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92a94fb640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f2369de9e0 0x55f2369e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f928cff9640 10 client.15783 ms_handle_connect on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92a94fb640 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f2369de9e0 0x55f2369e0e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff5d7e10a88cfbfe server_cookie=72008609b8bf9a13 in_seq=0 out_seq=0 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f928cff9640 1 -- 172.21.15.113:0/656537762 <== mds.1 v2:172.21.15.133:6836/65907218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2368ccc10 con 0x55f2369de9e0 2024-04-22T07:15:27.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f928cff9640 10 client.15783 handle_command_reply: tid=0 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 1 client.15783 shutdown 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.438+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 msgr2=0x55f236964840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236964840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f92940096c0 tx=0x7f9294003750 comp rx=0 tx=0).stop 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 shutdown_connections 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:6836/65907218,v1:172.21.15.133:6837/65907218] conn(0x55f2369de9e0 0x55f2369e0e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f236963e60 0x55f236964f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f23695ad60 0x55f236964840 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 --2- 172.21.15.113:0/656537762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f23695a3b0 0x55f236970720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 >> 172.21.15.113:0/656537762 conn(0x55f236954ba0 msgr2=0x55f236953c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f928cff9640 0 client.15783 ms_handle_reset on v2:172.21.15.133:6836/65907218 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 shutdown_connections 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 1 -- 172.21.15.113:0/656537762 wait complete. 2024-04-22T07:15:27.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ae42a740 20 client.15783 trim_cache size 0 max 16384 2024-04-22T07:15:27.444 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f92900a3ed0 msgr2=0x7f92900a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f92900a3ed0 0x7f92900a6390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9294006010 tx=0x7f929402c9b0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 msgr2=0x7f92a419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 0x7f92a419d4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f929c00cce0 tx=0x7f929c00a620 comp rx=0 tx=0).stop 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 shutdown_connections 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f92900a3ed0 0x7f92900a6390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92a419d000 0x7f92a419d4b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a4107260 0x7f92a4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 --2- 172.21.15.113:0/370803057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a4102190 0x7f92a4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 >> 172.21.15.113:0/370803057 conn(0x7f92a40793a0 msgr2=0x7f92a40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 shutdown_connections 2024-04-22T07:15:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.439+0000 7f92ab75f640 1 -- 172.21.15.113:0/370803057 wait complete. 2024-04-22T07:15:27.446 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:27 smithi113 ceph-mon[27933]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:27.447 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:15:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.549+0000 7fc80a848640 1 -- 172.21.15.113:0/2625260310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc80410c510 msgr2=0x7fc80410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.549+0000 7fc80a848640 1 --2- 172.21.15.113:0/2625260310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc80410c510 0x7fc80410e9b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009ca0 tx=0x7fc7fc02c230 comp rx=0 tx=0).stop 2024-04-22T07:15:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- 172.21.15.113:0/2625260310 shutdown_connections 2024-04-22T07:15:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 --2- 172.21.15.113:0/2625260310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc80410c510 0x7fc80410e9b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 --2- 172.21.15.113:0/2625260310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc804103af0 0x7fc80410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 --2- 172.21.15.113:0/2625260310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8041031d0 0x7fc8041035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- 172.21.15.113:0/2625260310 >> 172.21.15.113:0/2625260310 conn(0x7fc804077f50 msgr2=0x7fc804078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- 172.21.15.113:0/2625260310 shutdown_connections 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- 172.21.15.113:0/2625260310 wait complete. 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 Processor -- start 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- start start 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8041031d0 0x7fc80419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 0x7fc80419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc80410c510 0x7fc8041985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041122e0 con 0x7fc804103af0 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc804112160 con 0x7fc8041031d0 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc80a848640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc804112460 con 0x7fc80410c510 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.550+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 0x7fc80419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 0x7fc80419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57732/0 (socket says 172.21.15.113:57732) 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 -- 172.21.15.113:0/2874493271 learned_addr learned my addr 172.21.15.113:0/2874493271 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc803fff640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8041031d0 0x7fc80419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc808de5640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc80410c510 0x7fc8041985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 -- 172.21.15.113:0/2874493271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc80410c510 msgr2=0x7fc8041985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc80410c510 0x7fc8041985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 -- 172.21.15.113:0/2874493271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8041031d0 msgr2=0x7fc80419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8041031d0 0x7fc80419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 -- 172.21.15.113:0/2874493271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc804198d60 con 0x7fc804103af0 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc808de5640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc80410c510 0x7fc8041985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8037fe640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 0x7fc80419e970 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800e9f0 tx=0x7fc7f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8017fa640 1 -- 172.21.15.113:0/2874493271 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800cd80 con 0x7fc804103af0 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8017fa640 1 -- 172.21.15.113:0/2874493271 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7f80054f0 con 0x7fc804103af0 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc804199050 con 0x7fc804103af0 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc8017fa640 1 -- 172.21.15.113:0/2874493271 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8010690 con 0x7fc804103af0 2024-04-22T07:15:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.551+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8041a52f0 con 0x7fc804103af0 2024-04-22T07:15:27.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.553+0000 7fc8017fa640 1 -- 172.21.15.113:0/2874493271 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fc7f80046c0 con 0x7fc804103af0 2024-04-22T07:15:27.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 Processor -- start 2024-04-22T07:15:27.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 -- start start 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7ca1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c6f7bfb1f0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c6f7b984b0 con 0x55c6f7c96e80 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc80d432740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c6f7bf71b0 con 0x55c6f7c964d0 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc8017fa640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc7ec0a3ed0 0x7fc7ec0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7ca1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7ca1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57746/0 (socket says 172.21.15.113:57746) 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc8037fe640 1 -- 172.21.15.113:0/2529341567 learned_addr learned my addr 172.21.15.113:0/2529341567 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc808de5640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc7ec0a3ed0 0x7fc7ec0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc803fff640 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc808de5640 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc8017fa640 1 -- 172.21.15.113:0/2874493271 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc7f80d0e60 con 0x7fc804103af0 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc803fff640 1 -- 172.21.15.113:0/2529341567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 msgr2=0x55c6f7c968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc803fff640 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc803fff640 1 -- 172.21.15.113:0/2529341567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 msgr2=0x55c6f7ca1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc803fff640 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7ca1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc803fff640 1 -- 172.21.15.113:0/2529341567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f7b8a1b0 con 0x55c6f7c96e80 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.554+0000 7fc8037fe640 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7ca1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.555+0000 7fc808de5640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc7ec0a3ed0 0x7fc7ec0a6390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc032690 tx=0x7fc7fc0323c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.555+0000 7fc803fff640 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9f430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bbf0 tx=0x7fc7f40103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92247abae95c518 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.555+0000 7fc7ea7fc640 1 -- 172.21.15.113:0/2529341567 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4004030 con 0x55c6f7c96e80 2024-04-22T07:15:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.555+0000 7fc7ea7fc640 1 -- 172.21.15.113:0/2529341567 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7f40041d0 con 0x55c6f7c96e80 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.555+0000 7fc7ea7fc640 1 -- 172.21.15.113:0/2529341567 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40116a0 con 0x55c6f7c96e80 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- 172.21.15.113:0/2529341567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 msgr2=0x55c6f7c9f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9f430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bbf0 tx=0x7fc7f40103d0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- 172.21.15.113:0/2529341567 shutdown_connections 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7ca1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9f430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- 172.21.15.113:0/2529341567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- 172.21.15.113:0/2529341567 >> 172.21.15.113:0/2529341567 conn(0x55c6f7c90ba0 msgr2=0x55c6f7c90f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- 172.21.15.113:0/2529341567 shutdown_connections 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- 172.21.15.113:0/2529341567 wait complete. 2024-04-22T07:15:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- start start 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 Processor -- start 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c6f7ca2970 con 0x55c6f7c9fa00 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c6f7bb4500 con 0x55c6f7c96e80 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c6f7ca2af0 con 0x55c6f7c964d0 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.556+0000 7fc80d432740 10 client.0 _ll_register_callbacks cb 0x55c6f7b82b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57760/0 (socket says 172.21.15.113:57760) 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 -- 172.21.15.113:0/2401470844 learned_addr learned my addr 172.21.15.113:0/2401470844 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc803fff640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc8037fe640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 -- 172.21.15.113:0/2401470844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 msgr2=0x55c6f7c9f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c9f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 -- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 msgr2=0x55c6f7c9c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 -- 172.21.15.113:0/2401470844 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f790d3f0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc808de5640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9cc10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc036000 tx=0x7fc7fc045070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b88bb0f6d5b529 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc7e37fe640 1 -- 172.21.15.113:0/2401470844 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc049040 con 0x55c6f7c9fa00 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc7e37fe640 1 -- 172.21.15.113:0/2401470844 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc7fc004040 con 0x55c6f7c9fa00 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f40041d0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc7e37fe640 1 -- 172.21.15.113:0/2401470844 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc05b050 con 0x55c6f7c9fa00 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc80d432740 10 client.15795 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c6f79d77d0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.557+0000 7fc7e37fe640 1 -- 172.21.15.113:0/2401470844 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7f40041d0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc7e37fe640 1 -- 172.21.15.113:0/2401470844 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc7fc04c070 con 0x55c6f7c9fa00 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc80d432740 10 client.15795 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc80d432740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc80d432740 20 client.15795 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc80d432740 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7cafc70 0x55c6f7cb20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc80d432740 4 client.15795 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f7b9a930 con 0x55c6f7cafc70 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc803fff640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7cafc70 0x55c6f7cb20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc7e37fe640 10 client.15795 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.558+0000 7fc803fff640 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7cafc70 0x55c6f7cb20a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d87342ff8df6f8 server_cookie=758e5481506472c3 in_seq=0 out_seq=0 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc7e37fe640 1 -- 172.21.15.113:0/2401470844 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6f7b9a930 con 0x55c6f7cafc70 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc7e37fe640 10 client.15795 handle_command_reply: tid=0 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 client.15795 shutdown 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 msgr2=0x55c6f7c9cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9cc10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc036000 tx=0x7fc7fc045070 comp rx=0 tx=0).stop 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 shutdown_connections 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7cafc70 0x55c6f7cb20a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9cc10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc7e37fe640 0 client.15795 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 --2- 172.21.15.113:0/2401470844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c964d0 0x55c6f7c9f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 >> 172.21.15.113:0/2401470844 conn(0x55c6f7c90ba0 msgr2=0x55c6f7c928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 shutdown_connections 2024-04-22T07:15:27.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 1 -- 172.21.15.113:0/2401470844 wait complete. 2024-04-22T07:15:27.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.559+0000 7fc80d432740 20 client.15795 trim_cache size 0 max 16384 2024-04-22T07:15:27.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:27 smithi154 ceph-mon[31862]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:27.570 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:27 smithi133 ceph-mon[31198]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:27.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 Processor -- start 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 -- start start 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7cab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041b0460 con 0x55c6f7c96e80 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041add30 con 0x55c6f7c964d0 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.572+0000 7fc80d432740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041abb80 con 0x55c6f7c9fa00 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57772/0 (socket says 172.21.15.113:57772) 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 -- 172.21.15.113:0/572401367 learned_addr learned my addr 172.21.15.113:0/572401367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc808de5640 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 -- 172.21.15.113:0/572401367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c9fa00 msgr2=0x55c6f7c9f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc803fff640 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7cab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 -- 172.21.15.113:0/572401367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 msgr2=0x55c6f7cab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7cab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 -- 172.21.15.113:0/572401367 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f7c956f0 con 0x55c6f7c96e80 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc8037fe640 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9edc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c6a0 tx=0x7fc7f80d32b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb01aa8ee8c54759 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc7e9ffb640 1 -- 172.21.15.113:0/572401367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f809cbf0 con 0x55c6f7c96e80 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc7e9ffb640 1 -- 172.21.15.113:0/572401367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7f80a0fa0 con 0x55c6f7c96e80 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.573+0000 7fc7e9ffb640 1 -- 172.21.15.113:0/572401367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80a5630 con 0x55c6f7c96e80 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- 172.21.15.113:0/572401367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 msgr2=0x55c6f7c9edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9edc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c6a0 tx=0x7fc7f80d32b0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- 172.21.15.113:0/572401367 shutdown_connections 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c9f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c96e80 0x55c6f7c9edc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- 172.21.15.113:0/572401367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7cab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- 172.21.15.113:0/572401367 >> 172.21.15.113:0/572401367 conn(0x55c6f7c90ba0 msgr2=0x55c6f7c8fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- 172.21.15.113:0/572401367 shutdown_connections 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- 172.21.15.113:0/572401367 wait complete. 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- start start 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 Processor -- start 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7c993c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c96e80 0x55c6f7c99900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041ac1f0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041ad660 con 0x55c6f7c964d0 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041ac8c0 con 0x55c6f7c96e80 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc80d432740 10 client.0 _ll_register_callbacks cb 0x7fc7fc004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc808de5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc808de5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c99e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57786/0 (socket says 172.21.15.113:57786) 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.574+0000 7fc808de5640 1 -- 172.21.15.113:0/1698521659 learned_addr learned my addr 172.21.15.113:0/1698521659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc8037fe640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c96e80 0x55c6f7c99900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc803fff640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7c993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc808de5640 1 -- 172.21.15.113:0/1698521659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c96e80 msgr2=0x55c6f7c99900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc808de5640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c96e80 0x55c6f7c99900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc808de5640 1 -- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 msgr2=0x55c6f7c993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc808de5640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7c993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc808de5640 1 -- 172.21.15.113:0/1698521659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f7d15260 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc808de5640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c99e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02c770 tx=0x7fc7fc0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11faba347feb894 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc7e37fe640 1 -- 172.21.15.113:0/1698521659 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc039070 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc7e37fe640 1 -- 172.21.15.113:0/1698521659 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fc7fc032cc0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f80a0fa0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc7e37fe640 1 -- 172.21.15.113:0/1698521659 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc0059f0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc80d432740 10 client.15801 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c6f7d16560 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc7e37fe640 1 -- 172.21.15.113:0/1698521659 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7f80a0fa0 con 0x55c6f7c9fa00 2024-04-22T07:15:27.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc7e37fe640 1 -- 172.21.15.113:0/1698521659 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc7fc04c040 con 0x55c6f7c9fa00 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc80d432740 10 client.15801 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.575+0000 7fc80d432740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc80d432740 20 client.15801 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc80d432740 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7d16bd0 0x55c6f7d18fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc80d432740 4 client.15801 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6f7c08c10 con 0x55c6f7d16bd0 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc803fff640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7d16bd0 0x55c6f7d18fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc7e37fe640 10 client.15801 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc803fff640 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7d16bd0 0x55c6f7d18fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53ca68f4cc60904 server_cookie=899dc37e82a696c5 in_seq=0 out_seq=0 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc7e37fe640 1 -- 172.21.15.113:0/1698521659 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6f7c08c10 con 0x55c6f7d16bd0 2024-04-22T07:15:27.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc7e37fe640 10 client.15801 handle_command_reply: tid=0 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.576+0000 7fc80d432740 1 client.15801 shutdown 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 msgr2=0x55c6f7c99e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c99e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02c770 tx=0x7fc7fc0093f0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 shutdown_connections 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55c6f7d16bd0 0x55c6f7d18fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6f7c9fa00 0x55c6f7c99e40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c6f7c96e80 0x55c6f7c99900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 --2- 172.21.15.113:0/1698521659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c6f7c964d0 0x55c6f7c993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc7e37fe640 0 client.15801 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 >> 172.21.15.113:0/1698521659 conn(0x55c6f7c90ba0 msgr2=0x55c6f7c97b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 shutdown_connections 2024-04-22T07:15:27.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 1 -- 172.21.15.113:0/1698521659 wait complete. 2024-04-22T07:15:27.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80d432740 20 client.15801 trim_cache size 0 max 16384 2024-04-22T07:15:27.584 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:15:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc7ec0a3ed0 msgr2=0x7fc7ec0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc7ec0a3ed0 0x7fc7ec0a6390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc032690 tx=0x7fc7fc0323c0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 msgr2=0x7fc80419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 0x7fc80419e970 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800e9f0 tx=0x7fc7f800edf0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 shutdown_connections 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc7ec0a3ed0 0x7fc7ec0a6390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc80410c510 0x7fc8041985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc804103af0 0x7fc80419e970 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 --2- 172.21.15.113:0/2874493271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8041031d0 0x7fc80419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 >> 172.21.15.113:0/2874493271 conn(0x7fc804077f50 msgr2=0x7fc8040fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 shutdown_connections 2024-04-22T07:15:27.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:27.577+0000 7fc80a848640 1 -- 172.21.15.113:0/2874493271 wait complete. 2024-04-22T07:15:28.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:28 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4209270321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:28.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:28 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4209270321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:28.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:28 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4209270321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:28.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:15:28.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:15:28.587 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:15:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2696132392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 msgr2=0x7fdfb0101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2696132392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb0101a20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9802bc80 comp rx=0 tx=0).stop 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2696132392 shutdown_connections 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2696132392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 0x7fdfb010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2696132392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb0101a20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2696132392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb0105e20 0x7fdfb0106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2696132392 >> 172.21.15.113:0/2696132392 conn(0x7fdfb00773e0 msgr2=0x7fdfb00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2696132392 shutdown_connections 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.702+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2696132392 wait complete. 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 Processor -- start 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 -- start start 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 0x7fdfb019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb0105e20 0x7fdfb0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0111330 con 0x7fdfb01015c0 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01111b0 con 0x7fdfb0101f60 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfb4e3f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01114b0 con 0x7fdfb0105e20 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfae59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.703+0000 7fdfae59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57798/0 (socket says 172.21.15.113:57798) 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 -- 172.21.15.113:0/2537407429 learned_addr learned my addr 172.21.15.113:0/2537407429 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb0105e20 0x7fdfb0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 0x7fdfb019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 -- 172.21.15.113:0/2537407429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb0105e20 msgr2=0x7fdfb0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb0105e20 0x7fdfb0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 -- 172.21.15.113:0/2537407429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 msgr2=0x7fdfb019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 0x7fdfb019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 -- 172.21.15.113:0/2537407429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb0199030 con 0x7fdfb01015c0 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 0x7fdfb019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfae59c640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb019e6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000b4a0 tx=0x7fdfa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfa77fe640 1 -- 172.21.15.113:0/2537407429 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0014070 con 0x7fdfb01015c0 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfa77fe640 1 -- 172.21.15.113:0/2537407429 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdfa0010ae0 con 0x7fdfb01015c0 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb0199320 con 0x7fdfb01015c0 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfa77fe640 1 -- 172.21.15.113:0/2537407429 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0017ce0 con 0x7fdfb01015c0 2024-04-22T07:15:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.704+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01a5630 con 0x7fdfb01015c0 2024-04-22T07:15:28.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.706+0000 7fdfa77fe640 1 -- 172.21.15.113:0/2537407429 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fdfa0017070 con 0x7fdfb01015c0 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 Processor -- start 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfa77fe640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdf9c0a3ed0 0x7fdf9c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdf9c0a3ed0 0x7fdf9c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfa77fe640 1 -- 172.21.15.113:0/2537407429 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdfa00d0080 con 0x7fdfb01015c0 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 -- start start 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b468e0 0x556d30b46cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b47290 0x556d30b4f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 0x556d30b52260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556d30aaaf00 con 0x556d30b4fe10 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556d30a48590 con 0x556d30b47290 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.707+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556d30aa6ec0 con 0x556d30b468e0 2024-04-22T07:15:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 0x556d30b52260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 0x556d30b52260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57808/0 (socket says 172.21.15.113:57808) 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 -- 172.21.15.113:0/1886007312 learned_addr learned my addr 172.21.15.113:0/1886007312 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdf9c0a3ed0 0x7fdf9c0a6390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8005fd0 tx=0x7fdfa8009600 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b47290 0x556d30b4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfae59c640 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b468e0 0x556d30b46cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 -- 172.21.15.113:0/1886007312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b468e0 msgr2=0x556d30b46cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b468e0 0x556d30b46cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 -- 172.21.15.113:0/1886007312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b47290 msgr2=0x556d30b4f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b47290 0x556d30b4f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 -- 172.21.15.113:0/1886007312 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d308857e0 con 0x556d30b4fe10 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfae59c640 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b468e0 0x556d30b46cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 0x556d30b52260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8012770 tx=0x7fdfa8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1cf5a27012e2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfa4ff9640 1 -- 172.21.15.113:0/1886007312 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8023070 con 0x556d30b4fe10 2024-04-22T07:15:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfa4ff9640 1 -- 172.21.15.113:0/1886007312 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdfa8011bf0 con 0x556d30b4fe10 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.708+0000 7fdfa4ff9640 1 -- 172.21.15.113:0/1886007312 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa801fd20 con 0x556d30b4fe10 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1886007312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 msgr2=0x556d30b52260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 0x556d30b52260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8012770 tx=0x7fdfa8013000 comp rx=0 tx=0).stop 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1886007312 shutdown_connections 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b4fe10 0x556d30b52260 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b47290 0x556d30b4f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1886007312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b468e0 0x556d30b46cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1886007312 >> 172.21.15.113:0/1886007312 conn(0x556d30b40fb0 msgr2=0x556d30b41390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1886007312 shutdown_connections 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1886007312 wait complete. 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 -- start start 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.709+0000 7fdfb7a2d740 1 Processor -- start 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 0x556d30b4d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b4fe10 0x556d30b4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556d30b52d80 con 0x556d30b47290 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556d30a64580 con 0x556d30b468e0 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556d30b52f00 con 0x556d30b4fe10 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 10 client.0 _ll_register_callbacks cb 0x556d30a6a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 0x556d30b4d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 0x556d30b4d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57818/0 (socket says 172.21.15.113:57818) 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 -- 172.21.15.113:0/2913515348 learned_addr learned my addr 172.21.15.113:0/2913515348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfae59c640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b4fe10 0x556d30b4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 -- 172.21.15.113:0/2913515348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b4fe10 msgr2=0x556d30b4d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b4fe10 0x556d30b4d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 -- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 msgr2=0x556d30b4cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b4cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 -- 172.21.15.113:0/2913515348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d308e5f70 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 0x556d30b4d2c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802c1c0 tx=0x7fdf9802cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b8912259a21799 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdf95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdf95ffb640 1 -- 172.21.15.113:0/2913515348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98005230 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdf95ffb640 1 -- 172.21.15.113:0/2913515348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdf980053d0 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.710+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa8011bf0 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdf95ffb640 1 -- 172.21.15.113:0/2913515348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdf98004490 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 10 client.15813 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556d3077d170 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdf95ffb640 1 -- 172.21.15.113:0/2913515348 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdfa8011bf0 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdf95ffb640 1 -- 172.21.15.113:0/2913515348 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdf98003950 con 0x556d30b47290 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 10 client.15813 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:28.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 20 client.15813 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b602d0 0x556d30b626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 4 client.15813 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d30ab8920 con 0x556d30b602d0 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.711+0000 7fdfae59c640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b602d0 0x556d30b626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdfae59c640 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b602d0 0x556d30b626c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdaf61db33c8bdc6 server_cookie=cfffdd308e29275d in_seq=0 out_seq=0 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdf95ffb640 10 client.15813 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdf95ffb640 1 -- 172.21.15.113:0/2913515348 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d30ab8920 con 0x556d30b602d0 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdf95ffb640 10 client.15813 handle_command_reply: tid=0 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdfb7a2d740 1 client.15813 shutdown 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 msgr2=0x556d30b4d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.712+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 0x556d30b4d2c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdf9802c1c0 tx=0x7fdf9802cdc0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 shutdown_connections 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b4fe10 0x556d30b4d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b602d0 0x556d30b626c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b47290 0x556d30b4d2c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/2913515348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b4cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdf95ffb640 0 client.15813 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 >> 172.21.15.113:0/2913515348 conn(0x556d30b40fb0 msgr2=0x556d30b42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 shutdown_connections 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/2913515348 wait complete. 2024-04-22T07:15:28.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.713+0000 7fdfb7a2d740 20 client.15813 trim_cache size 0 max 16384 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 Processor -- start 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 -- start start 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b40fb0 0x556d30b5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 0x556d30b4d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b67ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01b0620 con 0x556d30b468e0 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01adef0 con 0x556d30b40fb0 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01abd40 con 0x556d30b47290 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 0x556d30b4d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 0x556d30b4d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57820/0 (socket says 172.21.15.113:57820) 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 -- 172.21.15.113:0/1957720235 learned_addr learned my addr 172.21.15.113:0/1957720235 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfae59c640 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b40fb0 0x556d30b5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b67ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 -- 172.21.15.113:0/1957720235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 msgr2=0x556d30b67ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b67ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 -- 172.21.15.113:0/1957720235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b40fb0 msgr2=0x556d30b5bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b40fb0 0x556d30b5bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfa7fff640 1 -- 172.21.15.113:0/1957720235 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d30b4b2d0 con 0x556d30b468e0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.730+0000 7fdfaed9d640 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b67ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 0x556d30b4d450 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004650 tx=0x7fdf9802bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f06b6e6478a29ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdf97fff640 1 -- 172.21.15.113:0/1957720235 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9803f440 con 0x556d30b468e0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdf97fff640 1 -- 172.21.15.113:0/1957720235 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdf98003ac0 con 0x556d30b468e0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdf97fff640 1 -- 172.21.15.113:0/1957720235 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98054440 con 0x556d30b468e0 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1957720235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 msgr2=0x556d30b4d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 0x556d30b4d450 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004650 tx=0x7fdf9802bc80 comp rx=0 tx=0).stop 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1957720235 shutdown_connections 2024-04-22T07:15:28.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b67ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b468e0 0x556d30b4d450 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/1957720235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b40fb0 0x556d30b5bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1957720235 >> 172.21.15.113:0/1957720235 conn(0x556d30b4b9c0 msgr2=0x556d30b4cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1957720235 shutdown_connections 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.731+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/1957720235 wait complete. 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 -- start start 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 Processor -- start 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 0x556d30b54b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01ac3b0 con 0x556d30b40fb0 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01ad820 con 0x556d30b468e0 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01aca80 con 0x556d30b47290 2024-04-22T07:15:28.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfb7a2d740 10 client.0 _ll_register_callbacks cb 0x7fdf98005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 0x556d30b54b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 0x556d30b54b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57828/0 (socket says 172.21.15.113:57828) 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 -- 172.21.15.113:0/3582207168 learned_addr learned my addr 172.21.15.113:0/3582207168 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 -- 172.21.15.113:0/3582207168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 msgr2=0x556d30b555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 -- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 msgr2=0x556d30b550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 -- 172.21.15.113:0/3582207168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30bc6770 con 0x556d30b40fb0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdfae59c640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 0x556d30b54b60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0098380 tx=0x7fdfa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da608fcb0981e22b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdf95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdf95ffb640 1 -- 172.21.15.113:0/3582207168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00b9070 con 0x556d30b40fb0 2024-04-22T07:15:28.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.732+0000 7fdf95ffb640 1 -- 172.21.15.113:0/3582207168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fdfa00b5250 con 0x556d30b40fb0 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf98003ac0 con 0x556d30b40fb0 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdf95ffb640 1 -- 172.21.15.113:0/3582207168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdfa001e040 con 0x556d30b40fb0 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 10 client.15819 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556d30bc7a50 con 0x556d30b40fb0 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdf95ffb640 1 -- 172.21.15.113:0/3582207168 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdf98003ac0 con 0x556d30b40fb0 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdf95ffb640 1 -- 172.21.15.113:0/3582207168 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdfa00a4790 con 0x556d30b40fb0 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 10 client.15819 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 20 client.15819 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b4fe10 0x556d30bca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 4 client.15819 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d30ab2790 con 0x556d30b4fe10 2024-04-22T07:15:28.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.733+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b4fe10 0x556d30bca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.734+0000 7fdf95ffb640 10 client.15819 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.734+0000 7fdfa7fff640 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b4fe10 0x556d30bca200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7f308ea89fb1cb server_cookie=86eb52eceb779572 in_seq=0 out_seq=0 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdf95ffb640 1 -- 172.21.15.113:0/3582207168 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d30ab2790 con 0x556d30b4fe10 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdf95ffb640 10 client.15819 handle_command_reply: tid=0 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 client.15819 shutdown 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 msgr2=0x556d30b54b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 0x556d30b54b60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0098380 tx=0x7fdfa00046c0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 shutdown_connections 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556d30b47290 0x556d30b555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d30b468e0 0x556d30b550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556d30b4fe10 0x556d30bca200 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 --2- 172.21.15.113:0/3582207168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556d30b40fb0 0x556d30b54b60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 >> 172.21.15.113:0/3582207168 conn(0x556d30b4b9c0 msgr2=0x556d30b4d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdf95ffb640 0 client.15819 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 shutdown_connections 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 1 -- 172.21.15.113:0/3582207168 wait complete. 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.735+0000 7fdfb7a2d740 20 client.15819 trim_cache size 0 max 16384 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "38837255-4c8c-4fbf-a173-349ce07919ee", 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:15:28.740 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdf9c0a3ed0 msgr2=0x7fdf9c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdf9c0a3ed0 0x7fdf9c0a6390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8005fd0 tx=0x7fdfa8009600 comp rx=0 tx=0).stop 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 msgr2=0x7fdfb019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb019e6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000b4a0 tx=0x7fdfa000b960 comp rx=0 tx=0).stop 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 shutdown_connections 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdf9c0a3ed0 0x7fdf9c0a6390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb0105e20 0x7fdfb0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfb0101f60 0x7fdfb019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 --2- 172.21.15.113:0/2537407429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfb01015c0 0x7fdfb019e6b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 >> 172.21.15.113:0/2537407429 conn(0x7fdfb00773e0 msgr2=0x7fdfb0100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 shutdown_connections 2024-04-22T07:15:28.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.736+0000 7fdfb4e3f640 1 -- 172.21.15.113:0/2537407429 wait complete. 2024-04-22T07:15:28.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:15:28.745 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:15:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 -- 172.21.15.113:0/418907115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0101090 msgr2=0x7f61e0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/418907115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0101090 0x7f61e0101470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009900 tx=0x7f61d802bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 -- 172.21.15.113:0/418907115 shutdown_connections 2024-04-22T07:15:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/418907115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 0x7f61e010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/418907115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101a40 0x7f61e0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/418907115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0101090 0x7f61e0101470 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 -- 172.21.15.113:0/418907115 >> 172.21.15.113:0/418907115 conn(0x7f61e00771d0 msgr2=0x7f61e00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 -- 172.21.15.113:0/418907115 shutdown_connections 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.847+0000 7f61e4ceb640 1 -- 172.21.15.113:0/418907115 wait complete. 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 Processor -- start 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 -- start start 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101090 0x7f61e019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 0x7f61e019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 0x7f61e0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0112300 con 0x7f61e0198680 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0112180 con 0x7f61e0101090 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61e4ceb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61e0112480 con 0x7f61e0106160 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 0x7f61e0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 0x7f61e0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57832/0 (socket says 172.21.15.113:57832) 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 -- 172.21.15.113:0/569699516 learned_addr learned my addr 172.21.15.113:0/569699516 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 0x7f61e019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61de59c640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101090 0x7f61e019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 -- 172.21.15.113:0/569699516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 msgr2=0x7f61e019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 0x7f61e019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 -- 172.21.15.113:0/569699516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101090 msgr2=0x7f61e019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101090 0x7f61e019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 -- 172.21.15.113:0/569699516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e0199280 con 0x7f61e0198680 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 0x7f61e019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61de59c640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101090 0x7f61e019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61ded9d640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 0x7f61e0198b00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f61d000c970 tx=0x7f61d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.848+0000 7f61d77fe640 1 -- 172.21.15.113:0/569699516 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0007bb0 con 0x7f61e0198680 2024-04-22T07:15:28.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.849+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01a5360 con 0x7f61e0198680 2024-04-22T07:15:28.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.849+0000 7f61d77fe640 1 -- 172.21.15.113:0/569699516 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61d0007d50 con 0x7f61e0198680 2024-04-22T07:15:28.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.849+0000 7f61d77fe640 1 -- 172.21.15.113:0/569699516 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000f450 con 0x7f61e0198680 2024-04-22T07:15:28.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.849+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a57c0 con 0x7f61e0198680 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61d77fe640 1 -- 172.21.15.113:0/569699516 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f61d0004f20 con 0x7f61e0198680 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 Processor -- start 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 -- start start 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b30690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 0x559456b3d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b3de90 0x559456b40280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559456a951e0 con 0x559456b30bd0 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559456a32390 con 0x559456b302b0 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.851+0000 7f61e79ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559456a911a0 con 0x559456b3de90 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61d77fe640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61cc0a3ed0 0x7f61cc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 0x559456b3d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 0x559456b3d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57836/0 (socket says 172.21.15.113:57836) 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 -- 172.21.15.113:0/2439806337 learned_addr learned my addr 172.21.15.113:0/2439806337 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61ded9d640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61cc0a3ed0 0x7f61cc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b30690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61d77fe640 1 -- 172.21.15.113:0/569699516 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f61d00d0820 con 0x7f61e0198680 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b3de90 0x559456b40280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 -- 172.21.15.113:0/2439806337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b3de90 msgr2=0x559456b40280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b3de90 0x559456b40280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 -- 172.21.15.113:0/2439806337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 msgr2=0x559456b30690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b30690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 -- 172.21.15.113:0/2439806337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594568717d0 con 0x559456b30bd0 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61ded9d640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61cc0a3ed0 0x7f61cc0a6390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61d00cddc0 tx=0x7f61d0098040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61de59c640 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 0x559456b3d950 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000c00 tx=0x7f61d802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3407b4c2fbf4335 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61d4ff9640 1 -- 172.21.15.113:0/2439806337 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d802cad0 con 0x559456b30bd0 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61d4ff9640 1 -- 172.21.15.113:0/2439806337 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61d802cc70 con 0x559456b30bd0 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.852+0000 7f61d4ff9640 1 -- 172.21.15.113:0/2439806337 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80048e0 con 0x559456b30bd0 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 -- 172.21.15.113:0/2439806337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 msgr2=0x559456b3d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 0x559456b3d950 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000c00 tx=0x7f61d802c650 comp rx=0 tx=0).stop 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 -- 172.21.15.113:0/2439806337 shutdown_connections 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b3de90 0x559456b40280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b30bd0 0x559456b3d950 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2439806337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b30690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 -- 172.21.15.113:0/2439806337 >> 172.21.15.113:0/2439806337 conn(0x559456b2acb0 msgr2=0x559456b2b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 -- 172.21.15.113:0/2439806337 shutdown_connections 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 -- 172.21.15.113:0/2439806337 wait complete. 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 -- start start 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.853+0000 7f61e79ba740 1 Processor -- start 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b3ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b3b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b3b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559456b40bf0 con 0x559456b3de90 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559456a4e2f0 con 0x559456b302b0 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559456b40d70 con 0x559456b30bd0 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 10 client.0 _ll_register_callbacks cb 0x559456a5b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b3b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b3b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57840/0 (socket says 172.21.15.113:57840) 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3486656513 learned_addr learned my addr 172.21.15.113:0/3486656513 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61de59c640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b3b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b3ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3486656513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 msgr2=0x559456b3b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b3b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 msgr2=0x559456b3ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b3ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3486656513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594567a73f0 con 0x559456b3de90 2024-04-22T07:15:28.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b3b8d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f61c8013ce0 tx=0x7f61c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e771bc501bc310 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61c5ffb640 1 -- 172.21.15.113:0/3486656513 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8012050 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61c5ffb640 1 -- 172.21.15.113:0/3486656513 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f61c80121f0 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.854+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d802cc70 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61c5ffb640 1 -- 172.21.15.113:0/3486656513 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f61c8004b20 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 10 client.15831 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5594567670f0 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61c5ffb640 1 -- 172.21.15.113:0/3486656513 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61d802cc70 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61c5ffb640 1 -- 172.21.15.113:0/3486656513 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f61c800f040 con 0x559456b3de90 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 10 client.15831 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 20 client.15831 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456b4de10 0x559456b50260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 4 client.15831 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:28.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559456aa2c00 con 0x559456b4de10 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.855+0000 7f61de59c640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456b4de10 0x559456b50260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61de59c640 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456b4de10 0x559456b50260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897dae4e8e88e6c4 server_cookie=19107cb22e93be76 in_seq=0 out_seq=0 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61c5ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61c5ffb640 1 -- 172.21.15.113:0/3486656513 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559456aa2c00 con 0x559456b4de10 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61c5ffb640 10 client.15831 handle_command_reply: tid=0 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 client.15831 shutdown 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 msgr2=0x559456b3b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b3b8d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f61c8013ce0 tx=0x7f61c8004ab0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 shutdown_connections 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456b4de10 0x559456b50260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b3b8d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61c5ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b3b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3486656513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b302b0 0x559456b3ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 >> 172.21.15.113:0/3486656513 conn(0x559456b2acb0 msgr2=0x559456b2ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 shutdown_connections 2024-04-22T07:15:28.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 1 -- 172.21.15.113:0/3486656513 wait complete. 2024-04-22T07:15:28.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.856+0000 7f61e79ba740 20 client.15831 trim_cache size 0 max 16384 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.873+0000 7f61e79ba740 1 Processor -- start 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 -- start start 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b302b0 0x559456b49b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b30bd0 0x559456b3a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b55b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ae480 con 0x559456b3de90 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01abd50 con 0x559456b30bd0 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61e79ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a9ba0 con 0x559456b302b0 2024-04-22T07:15:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b55b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b55b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57848/0 (socket says 172.21.15.113:57848) 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3338684731 learned_addr learned my addr 172.21.15.113:0/3338684731 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3338684731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b302b0 msgr2=0x559456b49b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b302b0 0x559456b49b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3338684731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b30bd0 msgr2=0x559456b3a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b30bd0 0x559456b3a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 -- 172.21.15.113:0/3338684731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559456b30a00 con 0x559456b3de90 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b55b30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f61c80204d0 tx=0x7f61c8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a04104fd98665ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61c7fff640 1 -- 172.21.15.113:0/3338684731 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8013ae0 con 0x559456b3de90 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61c7fff640 1 -- 172.21.15.113:0/3338684731 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61c80136f0 con 0x559456b3de90 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.874+0000 7f61c7fff640 1 -- 172.21.15.113:0/3338684731 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c80054f0 con 0x559456b3de90 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- 172.21.15.113:0/3338684731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 msgr2=0x559456b55b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b55b30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f61c80204d0 tx=0x7f61c8009f90 comp rx=0 tx=0).stop 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- 172.21.15.113:0/3338684731 shutdown_connections 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b3de90 0x559456b55b30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b30bd0 0x559456b3a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- 172.21.15.113:0/3338684731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b302b0 0x559456b49b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- 172.21.15.113:0/3338684731 >> 172.21.15.113:0/3338684731 conn(0x559456b2acb0 msgr2=0x559456b29d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- 172.21.15.113:0/3338684731 shutdown_connections 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- 172.21.15.113:0/3338684731 wait complete. 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- start start 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 Processor -- start 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 0x559456b37bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b3de90 0x559456b38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aa210 con 0x559456b302b0 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.875+0000 7f61e79ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ab680 con 0x559456b3de90 2024-04-22T07:15:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61e79ba740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aa8e0 con 0x559456b30bd0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61e79ba740 10 client.0 _ll_register_callbacks cb 0x7f61c80134d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 0x559456b37bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 0x559456b37bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57852/0 (socket says 172.21.15.113:57852) 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 -- 172.21.15.113:0/2860135733 learned_addr learned my addr 172.21.15.113:0/2860135733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61de59c640 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b38130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 -- 172.21.15.113:0/2860135733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 msgr2=0x559456b38130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b38130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 -- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b3de90 msgr2=0x559456b38670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b3de90 0x559456b38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 -- 172.21.15.113:0/2860135733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559456bb3470 con 0x559456b302b0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61ddd9b640 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 0x559456b37bf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f61c8004af0 tx=0x7f61c8013bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93abb82f8ee5ef3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61c5ffb640 1 -- 172.21.15.113:0/2860135733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8013330 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61c5ffb640 1 -- 172.21.15.113:0/2860135733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f61c8005270 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c80136f0 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61c5ffb640 1 -- 172.21.15.113:0/2860135733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f61c8005470 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61e79ba740 10 client.15837 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559456bb4780 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.876+0000 7f61c5ffb640 1 -- 172.21.15.113:0/2860135733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61c80136f0 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61c5ffb640 1 -- 172.21.15.113:0/2860135733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f61c8031020 con 0x559456b302b0 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61e79ba740 10 client.15837 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61e79ba740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61e79ba740 20 client.15837 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456bb4d30 0x559456bb7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61e79ba740 4 client.15837 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559456a9ca70 con 0x559456bb4d30 2024-04-22T07:15:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61de59c640 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456bb4d30 0x559456bb7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61c5ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.877+0000 7f61de59c640 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456bb4d30 0x559456bb7120 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4129699707021416 server_cookie=29a74cab26985874 in_seq=0 out_seq=0 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61c5ffb640 1 -- 172.21.15.113:0/2860135733 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559456a9ca70 con 0x559456bb4d30 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61c5ffb640 10 client.15837 handle_command_reply: tid=0 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 client.15837 shutdown 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 msgr2=0x559456b37bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 0x559456b37bf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f61c8004af0 tx=0x7f61c8013bf0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 shutdown_connections 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559456b3de90 0x559456b38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559456b30bd0 0x559456b38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x559456bb4d30 0x559456bb7120 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 --2- 172.21.15.113:0/2860135733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559456b302b0 0x559456b37bf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 >> 172.21.15.113:0/2860135733 conn(0x559456b2acb0 msgr2=0x559456b3ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61c5ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 shutdown_connections 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 1 -- 172.21.15.113:0/2860135733 wait complete. 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e79ba740 20 client.15837 trim_cache size 0 max 16384 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stdout: "38837255-4c8c-4fbf-a173-349ce07919ee": { 2024-04-22T07:15:28.883 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:15:28.884 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "38837255-4c8c-4fbf-a173-349ce07919ee", 2024-04-22T07:15:28.884 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:15:28.884 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:28.884 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:28.884 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61cc0a3ed0 msgr2=0x7f61cc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61cc0a3ed0 0x7f61cc0a6390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61d00cddc0 tx=0x7f61d0098040 comp rx=0 tx=0).stop 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 msgr2=0x7f61e0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 0x7f61e0198b00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f61d000c970 tx=0x7f61d000ce30 comp rx=0 tx=0).stop 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 shutdown_connections 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61cc0a3ed0 0x7f61cc0a6390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61e0198680 0x7f61e0198b00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61e0106160 0x7f61e019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 --2- 172.21.15.113:0/569699516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e0101090 0x7f61e019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 >> 172.21.15.113:0/569699516 conn(0x7f61e00771d0 msgr2=0x7f61e00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 shutdown_connections 2024-04-22T07:15:28.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:28.878+0000 7f61e4ceb640 1 -- 172.21.15.113:0/569699516 wait complete. 2024-04-22T07:15:28.887 INFO:tasks.cephfs.filesystem:scrub status for tag:38837255-4c8c-4fbf-a173-349ce07919ee - {'path': '/', 'tag': '38837255-4c8c-4fbf-a173-349ce07919ee', 'options': 'recursive,force'} 2024-04-22T07:15:29.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:29 smithi133 ceph-mon[31198]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:29.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:29 smithi154 ceph-mon[31862]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:29.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:29 smithi113 ceph-mon[27933]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:30 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:15:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:30 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:15:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:30 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:15:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:30 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:15:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:30 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:15:30.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:30 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:15:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:30 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:15:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:30 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:15:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:30 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:15:30.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:30 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:15:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:30 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:15:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:30 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:15:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:30 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:15:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:30 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:15:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:30 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:15:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:31 smithi133 ceph-mon[31198]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-22T07:15:31.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:31 smithi154 ceph-mon[31862]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-22T07:15:31.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:31 smithi113 ceph-mon[27933]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-22T07:15:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:33 smithi133 ceph-mon[31198]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:33.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:33 smithi154 ceph-mon[31862]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:33.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:33 smithi113 ceph-mon[27933]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-22T07:15:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:35 smithi133 ceph-mon[31198]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-22T07:15:35.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:35 smithi154 ceph-mon[31862]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-22T07:15:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:35 smithi113 ceph-mon[27933]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-22T07:15:37.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:37 smithi133 ceph-mon[31198]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:37.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:37 smithi154 ceph-mon[31862]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:37.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:37 smithi113 ceph-mon[27933]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:38.863 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:38.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3710389279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8105e30 msgr2=0x7f8db8106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3710389279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8105e30 0x7f8db8106210 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009cd0 tx=0x7f8db402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3710389279 shutdown_connections 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3710389279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db8101b60 0x7f8db810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3710389279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8db81011a0 0x7f8db8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3710389279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8105e30 0x7f8db8106210 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3710389279 >> 172.21.15.113:0/3710389279 conn(0x7f8db8077e30 msgr2=0x7f8db8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3710389279 shutdown_connections 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3710389279 wait complete. 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 Processor -- start 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.981+0000 7f8dc04ba640 1 -- start start 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db81011a0 0x7f8db819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:38.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 0x7f8db819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:38.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8db8105e30 0x7f8db8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:38.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8111200 con 0x7f8db8101b60 2024-04-22T07:15:38.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8111080 con 0x7f8db8105e30 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8111380 con 0x7f8db81011a0 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 0x7f8db819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 0x7f8db819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60352/0 (socket says 172.21.15.113:60352) 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 -- 172.21.15.113:0/3447507437 learned_addr learned my addr 172.21.15.113:0/3447507437 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbea57640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8db8105e30 0x7f8db8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbe256640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db81011a0 0x7f8db819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 -- 172.21.15.113:0/3447507437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db81011a0 msgr2=0x7f8db819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db81011a0 0x7f8db819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 -- 172.21.15.113:0/3447507437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8db8105e30 msgr2=0x7f8db8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8db8105e30 0x7f8db8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 -- 172.21.15.113:0/3447507437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db8198f50 con 0x7f8db8101b60 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbe256640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db81011a0 0x7f8db819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dbda55640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 0x7f8db819eb10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8da800d950 tx=0x7f8da800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:38.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8017070 con 0x7f8db8101b60 2024-04-22T07:15:38.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8da8009c30 con 0x7f8db8101b60 2024-04-22T07:15:38.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.982+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db8199240 con 0x7f8db8101b60 2024-04-22T07:15:38.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.983+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da800f6b0 con 0x7f8db8101b60 2024-04-22T07:15:38.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.983+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db81a5550 con 0x7f8db8101b60 2024-04-22T07:15:38.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.983+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db8102570 con 0x7f8db8101b60 2024-04-22T07:15:38.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.985+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f8da80109c0 con 0x7f8db8101b60 2024-04-22T07:15:38.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.985+0000 7f8daf7fe640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8da40a3ed0 0x7f8da40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:38.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.986+0000 7f8dbe256640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8da40a3ed0 0x7f8da40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:38.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.986+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f8da80d0700 con 0x7f8db8101b60 2024-04-22T07:15:38.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.986+0000 7f8dbe256640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8da40a3ed0 0x7f8da40a6390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006010 tx=0x7f8db4005c50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:38.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:38.990+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8da8009de0 con 0x7f8db8101b60 2024-04-22T07:15:39.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.206+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8db8199bf0 con 0x7f8db8101b60 2024-04-22T07:15:39.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.206+0000 7f8daf7fe640 1 -- 172.21.15.113:0/3447507437 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f8db8199bf0 con 0x7f8db8101b60 2024-04-22T07:15:39.207 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:39.208 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8da40a3ed0 msgr2=0x7f8da40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8da40a3ed0 0x7f8da40a6390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006010 tx=0x7f8db4005c50 comp rx=0 tx=0).stop 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 msgr2=0x7f8db819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 0x7f8db819eb10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8da800d950 tx=0x7f8da800de10 comp rx=0 tx=0).stop 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 shutdown_connections 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8da40a3ed0 0x7f8da40a6390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8db8105e30 0x7f8db8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8db8101b60 0x7f8db819eb10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 --2- 172.21.15.113:0/3447507437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8db81011a0 0x7f8db819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 >> 172.21.15.113:0/3447507437 conn(0x7f8db8077e30 msgr2=0x7f8db8069440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 shutdown_connections 2024-04-22T07:15:39.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.208+0000 7f8dc04ba640 1 -- 172.21.15.113:0/3447507437 wait complete. 2024-04-22T07:15:39.218 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id e298081e --release --await' 2024-04-22T07:15:39.219 DEBUG:teuthology.orchestra.run.smithi113:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24506 quiesce db --set-id e298081e --release --await 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 -- 172.21.15.113:0/3212145452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc102b40 msgr2=0x7f23fc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 --2- 172.21.15.113:0/3212145452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc102b40 0x7f23fc106b80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009900 tx=0x7f23ec02bc30 comp rx=0 tx=0).stop 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 -- 172.21.15.113:0/3212145452 shutdown_connections 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 --2- 172.21.15.113:0/3212145452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23fc107260 0x7f23fc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 --2- 172.21.15.113:0/3212145452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc102b40 0x7f23fc106b80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 --2- 172.21.15.113:0/3212145452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc102190 0x7f23fc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 -- 172.21.15.113:0/3212145452 >> 172.21.15.113:0/3212145452 conn(0x7f23fc0793a0 msgr2=0x7f23fc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 -- 172.21.15.113:0/3212145452 shutdown_connections 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 -- 172.21.15.113:0/3212145452 wait complete. 2024-04-22T07:15:39.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 Processor -- start 2024-04-22T07:15:39.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.329+0000 7f24023f4640 1 -- start start 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc102190 0x7f23fc19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23fc107260 0x7f23fc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 0x7f23fc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1122e0 con 0x7f23fc198640 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc112160 con 0x7f23fc102190 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc112460 con 0x7f23fc107260 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 0x7f23fc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 0x7f23fc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60384/0 (socket says 172.21.15.113:60384) 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 -- 172.21.15.113:0/4046099925 learned_addr learned my addr 172.21.15.113:0/4046099925 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23fc107260 0x7f23fc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f23fbfff640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc102190 0x7f23fc19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 -- 172.21.15.113:0/4046099925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23fc107260 msgr2=0x7f23fc19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23fc107260 0x7f23fc19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 -- 172.21.15.113:0/4046099925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc102190 msgr2=0x7f23fc19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc102190 0x7f23fc19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 -- 172.21.15.113:0/4046099925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc199240 con 0x7f23fc198640 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f2400991640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 0x7f23fc198ac0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f23f400cce0 tx=0x7f23f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f23f97fa640 1 -- 172.21.15.113:0/4046099925 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400f450 con 0x7f23fc198640 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1a5320 con 0x7f23fc198640 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f23f97fa640 1 -- 172.21.15.113:0/4046099925 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23f400fac0 con 0x7f23fc198640 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f23f97fa640 1 -- 172.21.15.113:0/4046099925 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4018be0 con 0x7f23fc198640 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.330+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a56d0 con 0x7f23fc198640 2024-04-22T07:15:39.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.332+0000 7f23f97fa640 1 -- 172.21.15.113:0/4046099925 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f23f400fc70 con 0x7f23fc198640 2024-04-22T07:15:39.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 Processor -- start 2024-04-22T07:15:39.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 -- start start 2024-04-22T07:15:39.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1310 0x56387e2d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 0x56387e2dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56387e236500 con 0x56387e2da810 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56387e1dc6a0 con 0x56387e2d1c30 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2404feb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56387e2324c0 con 0x56387e2d1310 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23f97fa640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23e40a3ed0 0x7f23e40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 0x56387e2dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 0x56387e2dcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60394/0 (socket says 172.21.15.113:60394) 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1952366292 learned_addr learned my addr 172.21.15.113:0/1952366292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f2400991640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23e40a3ed0 0x7f23e40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fbfff640 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1310 0x56387e2d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23f97fa640 1 -- 172.21.15.113:0/4046099925 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f23f40d1120 con 0x7f23fc198640 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1952366292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1310 msgr2=0x56387e2d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1310 0x56387e2d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1952366292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 msgr2=0x56387e2da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.333+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1952366292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387e1fb5e0 con 0x56387e2da810 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2400991640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23e40a3ed0 0x7f23e40a6390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23f40ce6c0 tx=0x7f23f400af30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 0x56387e2dcc00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009230 tx=0x7f23ec005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdee4c6dd85135 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f23e27fc640 1 -- 172.21.15.113:0/1952366292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec004230 con 0x56387e2da810 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f23e27fc640 1 -- 172.21.15.113:0/1952366292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23ec0043d0 con 0x56387e2da810 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f23e27fc640 1 -- 172.21.15.113:0/1952366292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec034710 con 0x56387e2da810 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 -- 172.21.15.113:0/1952366292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 msgr2=0x56387e2dcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 0x56387e2dcc00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009230 tx=0x7f23ec005e50 comp rx=0 tx=0).stop 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 -- 172.21.15.113:0/1952366292 shutdown_connections 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2da810 0x56387e2dcc00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 --2- 172.21.15.113:0/1952366292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1310 0x56387e2d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 -- 172.21.15.113:0/1952366292 >> 172.21.15.113:0/1952366292 conn(0x56387e2cba40 msgr2=0x56387e2cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.334+0000 7f2404feb740 1 -- 172.21.15.113:0/1952366292 shutdown_connections 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 -- 172.21.15.113:0/1952366292 wait complete. 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 -- start start 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 Processor -- start 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56387e2dd5a0 con 0x56387e2d1310 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56387e1efa60 con 0x56387e2d1c30 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56387e2dd720 con 0x56387e2da810 2024-04-22T07:15:39.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f2404feb740 10 client.0 _ll_register_callbacks cb 0x56387df48110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2da2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60404/0 (socket says 172.21.15.113:60404) 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.335+0000 7f23fb7fe640 1 -- 172.21.15.113:0/3287599503 learned_addr learned my addr 172.21.15.113:0/3287599503 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fbfff640 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fb7fe640 1 -- 172.21.15.113:0/3287599503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 msgr2=0x56387e2d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fb7fe640 1 -- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 msgr2=0x56387e2d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2d7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fb7fe640 1 -- 172.21.15.113:0/3287599503 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387df48920 con 0x56387e2d1310 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2da2d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f23ec02c170 tx=0x7f23ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfb46afd5d36b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23db7fe640 1 -- 172.21.15.113:0/3287599503 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec041430 con 0x56387e2d1310 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23db7fe640 1 -- 172.21.15.113:0/3287599503 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f23ec0415d0 con 0x56387e2d1310 2024-04-22T07:15:39.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ec0043d0 con 0x56387e2d1310 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23db7fe640 1 -- 172.21.15.113:0/3287599503 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f23ec05b050 con 0x56387e2d1310 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 10 client.15855 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56387e014d30 con 0x56387e2d1310 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23db7fe640 1 -- 172.21.15.113:0/3287599503 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23ec0043d0 con 0x56387e2d1310 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f23db7fe640 1 -- 172.21.15.113:0/3287599503 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f23ec04d5e0 con 0x56387e2d1310 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 10 client.15855 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 10 client.15855 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 20 client.15855 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e2ea830 0x56387e2ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 4 client.15855 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.336+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56387e209a10 con 0x56387e2ea830 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.337+0000 7f23fbfff640 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e2ea830 0x56387e2ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.337+0000 7f23db7fe640 10 client.15855 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.337+0000 7f23fbfff640 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e2ea830 0x56387e2ecc20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e07b86701ce6eb server_cookie=f1e0f1cf06c69d14 in_seq=0 out_seq=0 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f23db7fe640 1 -- 172.21.15.113:0/3287599503 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56387e209a10 con 0x56387e2ea830 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f23db7fe640 10 client.15855 handle_command_reply: tid=0 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 client.15855 shutdown 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 msgr2=0x56387e2da2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2da2d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f23ec02c170 tx=0x7f23ec009260 comp rx=0 tx=0).stop 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 shutdown_connections 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1c30 0x56387e2d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e2ea830 0x56387e2ecc20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 --2- 172.21.15.113:0/3287599503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2da2d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f23db7fe640 0 client.15855 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 >> 172.21.15.113:0/3287599503 conn(0x56387e2cba40 msgr2=0x56387e2cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 shutdown_connections 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 1 -- 172.21.15.113:0/3287599503 wait complete. 2024-04-22T07:15:39.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.338+0000 7f2404feb740 20 client.15855 trim_cache size 0 max 16384 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 Processor -- start 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 -- start start 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1c30 0x56387e2cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2da810 0x56387e2f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1b0430 con 0x56387e2d1310 2024-04-22T07:15:39.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1add00 con 0x56387e2da810 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f2404feb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1abb50 con 0x56387e2d1c30 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2e64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60412/0 (socket says 172.21.15.113:60412) 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.355+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1154752839 learned_addr learned my addr 172.21.15.113:0/1154752839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fbfff640 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1c30 0x56387e2cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1154752839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1c30 msgr2=0x56387e2cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1c30 0x56387e2cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1154752839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2da810 msgr2=0x56387e2f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2da810 0x56387e2f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fb7fe640 1 -- 172.21.15.113:0/1154752839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387e2d6630 con 0x56387e2d1310 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fbfff640 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1c30 0x56387e2cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2e64c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f23ec000c00 tx=0x7f23ec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613b9e129d20d4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23e1ffb640 1 -- 172.21.15.113:0/1154752839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec045030 con 0x56387e2d1310 2024-04-22T07:15:39.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23e1ffb640 1 -- 172.21.15.113:0/1154752839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23ec052070 con 0x56387e2d1310 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f23e1ffb640 1 -- 172.21.15.113:0/1154752839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec05f690 con 0x56387e2d1310 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f2404feb740 1 -- 172.21.15.113:0/1154752839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 msgr2=0x56387e2e64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f2404feb740 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2e64c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f23ec000c00 tx=0x7f23ec005d50 comp rx=0 tx=0).stop 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f2404feb740 1 -- 172.21.15.113:0/1154752839 shutdown_connections 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f2404feb740 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2da810 0x56387e2f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f2404feb740 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2d1c30 0x56387e2cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.356+0000 7f2404feb740 1 --2- 172.21.15.113:0/1154752839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1310 0x56387e2e64c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- 172.21.15.113:0/1154752839 >> 172.21.15.113:0/1154752839 conn(0x56387e2cba40 msgr2=0x56387e2ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- 172.21.15.113:0/1154752839 shutdown_connections 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- 172.21.15.113:0/1154752839 wait complete. 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- start start 2024-04-22T07:15:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 Processor -- start 2024-04-22T07:15:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1310 0x56387e2df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 0x56387e2dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ac1c0 con 0x56387e2d1c30 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ad630 con 0x56387e2d1310 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ac890 con 0x56387e2da810 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f2404feb740 10 client.0 _ll_register_callbacks cb 0x7f23f000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f23fbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 0x56387e2dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f23fbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 0x56387e2dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60422/0 (socket says 172.21.15.113:60422) 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.357+0000 7f23fbfff640 1 -- 172.21.15.113:0/1361814954 learned_addr learned my addr 172.21.15.113:0/1361814954 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1310 0x56387e2df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fbfff640 1 -- 172.21.15.113:0/1361814954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 msgr2=0x56387e2e0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fbfff640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2e0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fbfff640 1 -- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1310 msgr2=0x56387e2df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fbfff640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1310 0x56387e2df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fbfff640 1 -- 172.21.15.113:0/1361814954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387e2d1a60 con 0x56387e2d1c30 2024-04-22T07:15:39.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23fbfff640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 0x56387e2dfee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f23f000b370 tx=0x7f23f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd62cddc4e00762 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23db7fe640 1 -- 172.21.15.113:0/1361814954 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f001d880 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23db7fe640 1 -- 172.21.15.113:0/1361814954 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f23f001c3d0 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ec052070 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23db7fe640 1 -- 172.21.15.113:0/1361814954 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f23f002c050 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f2404feb740 10 client.15861 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56387e351cf0 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23db7fe640 1 -- 172.21.15.113:0/1361814954 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f23ec052070 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f23db7fe640 1 -- 172.21.15.113:0/1361814954 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f23f0010640 con 0x56387e2d1c30 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f2404feb740 10 client.15861 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f2404feb740 10 client.15861 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.358+0000 7f2404feb740 20 client.15861 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.359+0000 7f2404feb740 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e3524a0 0x56387e3548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.359+0000 7f2404feb740 4 client.15861 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e298081e", "release": true, "await": true}] 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.359+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e298081e", "release": true, "await": true}) -- 0x56387e243f20 con 0x56387e3524a0 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.359+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e3524a0 0x56387e3548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.359+0000 7f23fb7fe640 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e3524a0 0x56387e3548c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643a58e3545b89ab server_cookie=1da3a8c3d0ab74b6 in_seq=0 out_seq=0 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.359+0000 7f23db7fe640 10 client.15861 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f23db7fe640 1 -- 172.21.15.113:0/1361814954 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x56387e243f20 con 0x56387e3524a0 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f23db7fe640 10 client.15861 handle_command_reply: tid=0 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 client.15861 shutdown 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 msgr2=0x56387e2dfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 0x56387e2dfee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f23f000b370 tx=0x7f23f0011810 comp rx=0 tx=0).stop 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 shutdown_connections 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56387e2da810 0x56387e2e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56387e3524a0 0x56387e3548c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56387e2d1c30 0x56387e2dfee0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 --2- 172.21.15.113:0/1361814954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56387e2d1310 0x56387e2df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f23db7fe640 0 client.15861 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 >> 172.21.15.113:0/1361814954 conn(0x56387e2cba40 msgr2=0x56387e2ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 shutdown_connections 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 1 -- 172.21.15.113:0/1361814954 wait complete. 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f2404feb740 20 client.15861 trim_cache size 0 max 16384 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stdout: "epoch": 22, 2024-04-22T07:15:39.366 INFO:teuthology.orchestra.run.smithi113.stdout: "leader": 24506, 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "set_version": 4, 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "sets": { 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "e298081e": { 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "version": 4, 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "age_ref": 0.0, 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RELEASED", 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "timeout": 60.0, 2024-04-22T07:15:39.367 INFO:teuthology.orchestra.run.smithi113.stdout: "expiration": 78.8, 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: "members": { 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: "file:/": { 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: "excluded": false, 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RELEASED", 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23e40a3ed0 msgr2=0x7f23e40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f24023f4640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23e40a3ed0 0x7f23e40a6390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23f40ce6c0 tx=0x7f23f400af30 comp rx=0 tx=0).stop 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 msgr2=0x7f23fc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.362+0000 7f24023f4640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 0x7f23fc198ac0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f23f400cce0 tx=0x7f23f400a620 comp rx=0 tx=0).stop 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 shutdown_connections 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23e40a3ed0 0x7f23e40a6390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23fc198640 0x7f23fc198ac0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23fc107260 0x7f23fc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 --2- 172.21.15.113:0/4046099925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23fc102190 0x7f23fc19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 >> 172.21.15.113:0/4046099925 conn(0x7f23fc0793a0 msgr2=0x7f23fc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 shutdown_connections 2024-04-22T07:15:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.363+0000 7f24023f4640 1 -- 172.21.15.113:0/4046099925 wait complete. 2024-04-22T07:15:39.372 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: e298081e, seconds elapsed: 19.9 2024-04-22T07:15:39.372 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-04-22T07:15:39.372 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:39.420 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-mon[31198]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:39.420 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f18324bd700 time 2024-04-22T07:15:39.362052+0000 2024-04-22T07:15:39.420 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.420 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.420 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1844af63dd] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: clone() 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.362+0000 7f18324bd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f18324bd700 time 2024-04-22T07:15:39.362052+0000 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.422 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1844af63dd] 2024-04-22T07:15:39.422 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.422 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-mon[31862]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: clone() 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.424 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: *** Caught signal (Aborted) ** 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: in thread 7f18324bd700 thread_name:quiesce.agt 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f184387ed20] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: gsignal() 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: abort() 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1844af6437] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.425 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 13: clone() 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.363+0000 7f18324bd700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: in thread 7f18324bd700 thread_name:quiesce.agt 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f184387ed20] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: gsignal() 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: abort() 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1844af6437] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.426 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 13: clone() 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: -1> 2024-04-22T07:15:39.362+0000 7f18324bd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f18324bd700 time 2024-04-22T07:15:39.362052+0000 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1844af63dd] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.427 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: clone() 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 0> 2024-04-22T07:15:39.363+0000 7f18324bd700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: in thread 7f18324bd700 thread_name:quiesce.agt 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f184387ed20] 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: gsignal() 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: abort() 2024-04-22T07:15:39.428 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1844af6437] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 13: clone() 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: -1> 2024-04-22T07:15:39.362+0000 7f18324bd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f18324bd700 time 2024-04-22T07:15:39.362052+0000 2024-04-22T07:15:39.429 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1844af63dd] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: clone() 2024-04-22T07:15:39.430 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 0> 2024-04-22T07:15:39.363+0000 7f18324bd700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: in thread 7f18324bd700 thread_name:quiesce.agt 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f184387ed20] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2: gsignal() 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 3: abort() 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1844af6437] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1844af65a3] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x557aed8b8e9c] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x557aed8b8f9f] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x557aed7e70d3] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x557aed7e703c] 2024-04-22T07:15:39.431 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x557aeda66d3f] 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x557aedacf560] 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f18438741ca] 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 13: clone() 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[61589]: 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:39 smithi133 podman[64772]: 2024-04-22 07:15:39.42296606 +0000 UTC m=+0.024131764 container died 9a81b14fc52ea01479e3415df909a31c27a38eb2223e9a8c500513adaa6a5d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-22T07:15:39.432 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe41c6b3700 time 2024-04-22T07:15:39.362107+0000 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe42ecec3dd] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: clone() 2024-04-22T07:15:39.433 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.361+0000 7fe41c6b3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe41c6b3700 time 2024-04-22T07:15:39.362107+0000 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe42ecec3dd] 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.434 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: clone() 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: *** Caught signal (Aborted) ** 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: in thread 7fe41c6b3700 thread_name:quiesce.agt 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe42da74d20] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: gsignal() 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: abort() 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe42ecec437] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.435 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 13: clone() 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.363+0000 7fe41c6b3700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: in thread 7fe41c6b3700 thread_name:quiesce.agt 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe42da74d20] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: gsignal() 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: abort() 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe42ecec437] 2024-04-22T07:15:39.436 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 13: clone() 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: -1> 2024-04-22T07:15:39.361+0000 7fe41c6b3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe41c6b3700 time 2024-04-22T07:15:39.362107+0000 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.437 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe42ecec3dd] 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.438 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: clone() 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 0> 2024-04-22T07:15:39.363+0000 7fe41c6b3700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: in thread 7fe41c6b3700 thread_name:quiesce.agt 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe42da74d20] 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: gsignal() 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: abort() 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe42ecec437] 2024-04-22T07:15:39.439 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 13: clone() 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: -1> 2024-04-22T07:15:39.361+0000 7fe41c6b3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe41c6b3700 time 2024-04-22T07:15:39.362107+0000 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.440 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe42ecec3dd] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: clone() 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 0> 2024-04-22T07:15:39.363+0000 7fe41c6b3700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:15:39.441 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: in thread 7fe41c6b3700 thread_name:quiesce.agt 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe42da74d20] 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2: gsignal() 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 3: abort() 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe42ecec437] 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe42ecec5a3] 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x5609b8b6de9c] 2024-04-22T07:15:39.442 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x5609b8b6df9f] 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x5609b8a9c0d3] 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x5609b8a9c03c] 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x5609b8d1bd3f] 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x5609b8d84560] 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe42da6a1ca] 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 13: clone() 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[63591]: 2024-04-22T07:15:39.443 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:39 smithi154 podman[66564]: 2024-04-22 07:15:39.424351637 +0000 UTC m=+0.023099135 container died 2cb0ad707540d628b04dac19a91b06cb60740e4f119678cd55fa01cf4beb7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:15:39.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 -- 172.21.15.113:0/2979276276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 msgr2=0x7f77f410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 --2- 172.21.15.113:0/2979276276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f410dd50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009ca0 tx=0x7f77ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 -- 172.21.15.113:0/2979276276 shutdown_connections 2024-04-22T07:15:39.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 --2- 172.21.15.113:0/2979276276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f410dd50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 --2- 172.21.15.113:0/2979276276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f4101a60 0x7f77f4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 --2- 172.21.15.113:0/2979276276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f41010b0 0x7f77f4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 -- 172.21.15.113:0/2979276276 >> 172.21.15.113:0/2979276276 conn(0x7f77f40771d0 msgr2=0x7f77f40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 -- 172.21.15.113:0/2979276276 shutdown_connections 2024-04-22T07:15:39.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.473+0000 7f77fb806640 1 -- 172.21.15.113:0/2979276276 wait complete. 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 Processor -- start 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 -- start start 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f41010b0 0x7f77f419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f41985e0 0x7f77f4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4112320 con 0x7f77f4106180 2024-04-22T07:15:39.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41121a0 con 0x7f77f41010b0 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77fb806640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41124a0 con 0x7f77f41985e0 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60454/0 (socket says 172.21.15.113:60454) 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 -- 172.21.15.113:0/1983819836 learned_addr learned my addr 172.21.15.113:0/1983819836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f9da3640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f41985e0 0x7f77f4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f95a2640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f41010b0 0x7f77f419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 -- 172.21.15.113:0/1983819836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f41985e0 msgr2=0x7f77f4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f41985e0 0x7f77f4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 -- 172.21.15.113:0/1983819836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f41010b0 msgr2=0x7f77f419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f41010b0 0x7f77f419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 -- 172.21.15.113:0/1983819836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f4199100 con 0x7f77f4106180 2024-04-22T07:15:39.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.474+0000 7f77f8da1640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f419e970 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d740 tx=0x7f77f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.475+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0017070 con 0x7f77f4106180 2024-04-22T07:15:39.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.475+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f77f000de50 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.475+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0004d90 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.475+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f41993f0 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.475+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41a55f0 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.476+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f4069230 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.477+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f77f0004f30 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.477+0000 7f77ea7fc640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f77e00a3ed0 0x7f77e00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.478+0000 7f77f95a2640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f77e00a3ed0 0x7f77e00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.478+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f77f00d07a0 con 0x7f77f4106180 2024-04-22T07:15:39.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.478+0000 7f77f95a2640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f77e00a3ed0 0x7f77e00a6390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77e400bbe0 tx=0x7f77e400b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.481+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f009c5e0 con 0x7f77f4106180 2024-04-22T07:15:39.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:39 smithi113 ceph-mon[27933]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:39.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.696+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77f4199b10 con 0x7f77f4106180 2024-04-22T07:15:39.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.697+0000 7f77ea7fc640 1 -- 172.21.15.113:0/1983819836 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11875 (secure 0 0 0) 0x7f77f4199b10 con 0x7f77f4106180 2024-04-22T07:15:39.696 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:14:47.974590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24473,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6839/3404626603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3404626603},{"type":"v1","addr":"172.21.15.154:6839","nonce":3404626603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/65907218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":65907218},{"type":"v1","addr":"172.21.15.133:6837","nonce":65907218}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24469,24473]},"id":1}]} 2024-04-22T07:15:39.696 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f77e00a3ed0 msgr2=0x7f77e00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f77e00a3ed0 0x7f77e00a6390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77e400bbe0 tx=0x7f77e400b390 comp rx=0 tx=0).stop 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 msgr2=0x7f77f419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f419e970 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f77f000d740 tx=0x7f77f000dc00 comp rx=0 tx=0).stop 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 shutdown_connections 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f77e00a3ed0 0x7f77e00a6390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77f41985e0 0x7f77f4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77f4106180 0x7f77f419e970 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 --2- 172.21.15.113:0/1983819836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f41010b0 0x7f77f419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 >> 172.21.15.113:0/1983819836 conn(0x7f77f40771d0 msgr2=0x7f77f40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 shutdown_connections 2024-04-22T07:15:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.699+0000 7f77fb806640 1 -- 172.21.15.113:0/1983819836 wait complete. 2024-04-22T07:15:39.709 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 78.85 --await' 2024-04-22T07:15:39.709 DEBUG:teuthology.orchestra.run.smithi113:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24506 quiesce db / --timeout 60 --expiration 78.85 --await 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/2268516979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f406a8e0 msgr2=0x7ff3f406acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 --2- 172.21.15.113:0/2268516979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f406a8e0 0x7ff3f406acc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e402bc50 comp rx=0 tx=0).stop 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/2268516979 shutdown_connections 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 --2- 172.21.15.113:0/2268516979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3f4107300 0x7ff3f410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 --2- 172.21.15.113:0/2268516979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3f406b290 0x7ff3f4106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 --2- 172.21.15.113:0/2268516979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f406a8e0 0x7ff3f406acc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/2268516979 >> 172.21.15.113:0/2268516979 conn(0x7ff3f4077f20 msgr2=0x7ff3f4078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/2268516979 shutdown_connections 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/2268516979 wait complete. 2024-04-22T07:15:39.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.817+0000 7ff3fa9a4640 1 Processor -- start 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 -- start start 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3f406a8e0 0x7ff3f419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3f4107300 0x7ff3f419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f4198760 0x7ff3f4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4112400 con 0x7ff3f4198760 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4112280 con 0x7ff3f406a8e0 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3fa9a4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4112580 con 0x7ff3f4107300 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f4198760 0x7ff3f4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f4198760 0x7ff3f4198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60486/0 (socket says 172.21.15.113:60486) 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 -- 172.21.15.113:0/3243859701 learned_addr learned my addr 172.21.15.113:0/3243859701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3f406a8e0 0x7ff3f419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3f4107300 0x7ff3f419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 -- 172.21.15.113:0/3243859701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3f4107300 msgr2=0x7ff3f419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3f4107300 0x7ff3f419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 -- 172.21.15.113:0/3243859701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3f406a8e0 msgr2=0x7ff3f419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3f406a8e0 0x7ff3f419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 -- 172.21.15.113:0/3243859701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f4199360 con 0x7ff3f4198760 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.818+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3f4198760 0x7ff3f4198be0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b970 tx=0x7ff3ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.819+0000 7ff3f17fa640 1 -- 172.21.15.113:0/3243859701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec019070 con 0x7ff3f4198760 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.819+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/3243859701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f41a5440 con 0x7ff3f4198760 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.819+0000 7ff3f17fa640 1 -- 172.21.15.113:0/3243859701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff3ec00cb40 con 0x7ff3f4198760 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.819+0000 7ff3f17fa640 1 -- 172.21.15.113:0/3243859701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec007610 con 0x7ff3f4198760 2024-04-22T07:15:39.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.819+0000 7ff3fa9a4640 1 -- 172.21.15.113:0/3243859701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41a58d0 con 0x7ff3f4198760 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f17fa640 1 -- 172.21.15.113:0/3243859701 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7ff3ec01c070 con 0x7ff3f4198760 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 Processor -- start 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 -- start start 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f17fa640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff3dc0a3ed0 0x7ff3dc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b12e0 0x5586c63b9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b9ec0 0x5586c63bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5586c63110b0 con 0x5586c63b12e0 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586c62c2680 con 0x5586c63b09c0 2024-04-22T07:15:39.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3fd590740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5586c62cec60 con 0x5586c63b9ec0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f17fa640 1 -- 172.21.15.113:0/3243859701 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff3ec0cff20 con 0x7ff3f4198760 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff3dc0a3ed0 0x7ff3dc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:36986/0 (socket says 172.21.15.113:36986) 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.821+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2823878211 learned_addr learned my addr 172.21.15.113:0/2823878211 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b9ec0 0x5586c63bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b12e0 0x5586c63b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2823878211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b9ec0 msgr2=0x5586c63bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b9ec0 0x5586c63bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2823878211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b12e0 msgr2=0x5586c63b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b12e0 0x5586c63b9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2823878211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c60f3c90 con 0x5586c63b09c0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63b0da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e402c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=265fcb719ccc8df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/3243859701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff3dc0a3ed0 0x7ff3dc0a6390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5586c63b0490 tx=0x7ff3e8006db0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3da7fc640 1 -- 172.21.15.113:0/2823878211 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e402cc30 con 0x5586c63b09c0 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3da7fc640 1 -- 172.21.15.113:0/2823878211 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff3e4005040 con 0x5586c63b09c0 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.822+0000 7ff3da7fc640 1 -- 172.21.15.113:0/2823878211 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4004a20 con 0x5586c63b09c0 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 -- 172.21.15.113:0/2823878211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 msgr2=0x5586c63b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63b0da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e402c790 comp rx=0 tx=0).stop 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 -- 172.21.15.113:0/2823878211 shutdown_connections 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b9ec0 0x5586c63bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b12e0 0x5586c63b9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2823878211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63b0da0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 -- 172.21.15.113:0/2823878211 >> 172.21.15.113:0/2823878211 conn(0x5586c63ab0f0 msgr2=0x5586c63ab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 -- 172.21.15.113:0/2823878211 shutdown_connections 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 -- 172.21.15.113:0/2823878211 wait complete. 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 -- start start 2024-04-22T07:15:39.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.823+0000 7ff3fd590740 1 Processor -- start 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 0x5586c63b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b9ec0 0x5586c63b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5586c6314400 con 0x5586c63b09c0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586c62d0840 con 0x5586c63b9ec0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5586c6319720 con 0x5586c63b12e0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 10 client.0 _ll_register_callbacks cb 0x5586c6321ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 0x5586c63b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 0x5586c63b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60500/0 (socket says 172.21.15.113:60500) 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2567044093 learned_addr learned my addr 172.21.15.113:0/2567044093 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b9ec0 0x5586c63b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2567044093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 msgr2=0x5586c63b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b9ec0 msgr2=0x5586c63b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b9ec0 0x5586c63b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 -- 172.21.15.113:0/2567044093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c6154420 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 0x5586c63b6f20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005ab0 tx=0x7ff3e40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12c7502d6bdd354 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3d37fe640 1 -- 172.21.15.113:0/2567044093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4004bb0 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3d37fe640 1 -- 172.21.15.113:0/2567044093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff3e4004d50 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.824+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e4005040 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3d37fe640 1 -- 172.21.15.113:0/2567044093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4034440 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 10 client.15879 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5586c5fe7580 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3d37fe640 1 -- 172.21.15.113:0/2567044093 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff3e4005040 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3d37fe640 1 -- 172.21.15.113:0/2567044093 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff3e404d030 con 0x5586c63b09c0 2024-04-22T07:15:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 10 client.15879 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 10 client.15879 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 20 client.15879 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c63c9a10 0x5586c63cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 4 client.15879 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586c6322b10 con 0x5586c63c9a10 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.825+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c63c9a10 0x5586c63cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3d37fe640 10 client.15879 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c63c9a10 0x5586c63cbe00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfc3c19d3ad09ad server_cookie=bffadb1fdca7895d in_seq=0 out_seq=0 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3d37fe640 1 -- 172.21.15.113:0/2567044093 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586c6322b10 con 0x5586c63c9a10 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3d37fe640 10 client.15879 handle_command_reply: tid=0 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 client.15879 shutdown 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 msgr2=0x5586c63b6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 0x5586c63b6f20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005ab0 tx=0x7ff3e40054a0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 shutdown_connections 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b9ec0 0x5586c63b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c63c9a10 0x5586c63cbe00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3fd590740 1 --2- 172.21.15.113:0/2567044093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b09c0 0x5586c63b6f20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.826+0000 7ff3d37fe640 0 client.15879 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.827+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 >> 172.21.15.113:0/2567044093 conn(0x5586c63ab0f0 msgr2=0x5586c63ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.827+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 shutdown_connections 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.827+0000 7ff3fd590740 1 -- 172.21.15.113:0/2567044093 wait complete. 2024-04-22T07:15:39.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.827+0000 7ff3fd590740 20 client.15879 trim_cache size 0 max 16384 2024-04-22T07:15:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 Processor -- start 2024-04-22T07:15:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 -- start start 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b09c0 0x5586c63b9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b12e0 0x5586c63aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41adf00 con 0x5586c63b9ec0 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41abd70 con 0x5586c63b12e0 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3fd590740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ad830 con 0x5586c63b09c0 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60516/0 (socket says 172.21.15.113:60516) 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 -- 172.21.15.113:0/1531812708 learned_addr learned my addr 172.21.15.113:0/1531812708 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b12e0 0x5586c63aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b09c0 0x5586c63b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 -- 172.21.15.113:0/1531812708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b09c0 msgr2=0x5586c63b9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b09c0 0x5586c63b9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 -- 172.21.15.113:0/1531812708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b12e0 msgr2=0x5586c63aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b12e0 0x5586c63aa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 -- 172.21.15.113:0/1531812708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c63b5ed0 con 0x5586c63b9ec0 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63b77c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b940 tx=0x7ff3ec0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a35d2273e1962de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3d9ffb640 1 -- 172.21.15.113:0/1531812708 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0a0170 con 0x5586c63b9ec0 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3d9ffb640 1 -- 172.21.15.113:0/1531812708 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff3ec0a4a30 con 0x5586c63b9ec0 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.844+0000 7ff3d9ffb640 1 -- 172.21.15.113:0/1531812708 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec097bc0 con 0x5586c63b9ec0 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 -- 172.21.15.113:0/1531812708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 msgr2=0x5586c63b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63b77c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b940 tx=0x7ff3ec0bc040 comp rx=0 tx=0).stop 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 -- 172.21.15.113:0/1531812708 shutdown_connections 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63b77c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b12e0 0x5586c63aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 --2- 172.21.15.113:0/1531812708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b09c0 0x5586c63b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 -- 172.21.15.113:0/1531812708 >> 172.21.15.113:0/1531812708 conn(0x5586c63ab0f0 msgr2=0x5586c63ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 -- 172.21.15.113:0/1531812708 shutdown_connections 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 -- 172.21.15.113:0/1531812708 wait complete. 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.845+0000 7ff3fd590740 1 -- start start 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 Processor -- start 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41aca90 con 0x5586c63b9ec0 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ac3c0 con 0x5586c63b09c0 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41ad160 con 0x5586c63b12e0 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3fd590740 10 client.0 _ll_register_callbacks cb 0x7ff3e800cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60522/0 (socket says 172.21.15.113:60522) 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 -- 172.21.15.113:0/743762653 learned_addr learned my addr 172.21.15.113:0/743762653 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 -- 172.21.15.113:0/743762653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 msgr2=0x5586c63be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 -- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 msgr2=0x5586c63c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 -- 172.21.15.113:0/743762653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c63b1110 con 0x5586c63b9ec0 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f37fe640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3f8f41640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63be820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0a4500 tx=0x7ff3ec09bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653047e0f4be9a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.846+0000 7ff3d37fe640 1 -- 172.21.15.113:0/743762653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0b9020 con 0x5586c63b9ec0 2024-04-22T07:15:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3d37fe640 1 -- 172.21.15.113:0/743762653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7ff3ec0b5230 con 0x5586c63b9ec0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3ec0a4a30 con 0x5586c63b9ec0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3d37fe640 1 -- 172.21.15.113:0/743762653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff3ec0a45a0 con 0x5586c63b9ec0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 10 client.15885 fetch_fsmap learned FSMap version 22 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5586c6430950 con 0x5586c63b9ec0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3d37fe640 1 -- 172.21.15.113:0/743762653 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff3ec0a4a30 con 0x5586c63b9ec0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3d37fe640 1 -- 172.21.15.113:0/743762653 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff3ec0bc0b0 con 0x5586c63b9ec0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 10 client.15885 fetch_fsmap finished waiting for FSMap version 22 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 10 client.15885 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 20 client.15885 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c64310b0 0x5586c64334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 4 client.15885 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.85, "await": true}] 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.85, "await": true}) -- 0x5586c631c980 con 0x5586c64310b0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.847+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c64310b0 0x5586c64334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:39.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.848+0000 7ff3d37fe640 10 client.15885 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:39.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:39.848+0000 7ff3f3fff640 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c64310b0 0x5586c64334d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24063533970563a3 server_cookie=beab91be692f6d56 in_seq=0 out_seq=0 2024-04-22T07:15:40.351 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:40 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3447507437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:40.351 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:40 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1983819836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:40.351 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:40 smithi133 podman[64772]: 2024-04-22 07:15:40.09816242 +0000 UTC m=+0.699328121 container remove 9a81b14fc52ea01479e3415df909a31c27a38eb2223e9a8c500513adaa6a5d89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416) 2024-04-22T07:15:40.352 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:40 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-22T07:15:40.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:40 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3447507437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:40.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:40 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1983819836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:40.566 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:40 smithi154 podman[66564]: 2024-04-22 07:15:40.10725229 +0000 UTC m=+0.705999776 container remove 2cb0ad707540d628b04dac19a91b06cb60740e4f119678cd55fa01cf4beb7d3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:15:40.566 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:40 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-04-22T07:15:40.566 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:40 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-22T07:15:40.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:40 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3447507437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:40.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:40 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1983819836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:15:40.814 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:40 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-22T07:15:41.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:41 smithi133 ceph-mon[31198]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:41.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:41 smithi154 ceph-mon[31862]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:41.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:41 smithi113 ceph-mon[27933]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:43.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:43 smithi133 ceph-mon[31198]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:43.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:43 smithi154 ceph-mon[31862]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:43.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:43 smithi113 ceph-mon[27933]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:45 smithi133 ceph-mon[31198]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:45.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:45 smithi154 ceph-mon[31862]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:45 smithi113 ceph-mon[27933]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:15:47.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:47 smithi133 ceph-mon[31198]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:47.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:47 smithi154 ceph-mon[31862]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:47.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:47 smithi113 ceph-mon[27933]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-22T07:15:49.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:49 smithi133 ceph-mon[31198]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:15:49.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:49 smithi154 ceph-mon[31862]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:15:49.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:49 smithi113 ceph-mon[27933]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:15:50.821 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:50 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-04-22T07:15:50.821 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:50 smithi154 systemd[1]: Stopped Ceph mds.i for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:15:50.821 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:50 smithi154 systemd[1]: Starting Ceph mds.i for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:15:50.822 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:50 smithi154 podman[66658]: 2024-04-22 07:15:50.707819064 +0000 UTC m=+0.018379031 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:15:50.822 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:50 smithi154 podman[66658]: 2024-04-22 07:15:50.824587265 +0000 UTC m=+0.135147203 container create d376c651472949bbbe92e52948b5ce8c9e2255b28b44c480cdab77abc96fa5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True) 2024-04-22T07:15:50.828 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:50 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-04-22T07:15:50.829 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:50 smithi133 systemd[1]: Stopped Ceph mds.h for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:15:50.829 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:50 smithi133 systemd[1]: Starting Ceph mds.h for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:15:50.829 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:50 smithi133 podman[64865]: 2024-04-22 07:15:50.710880336 +0000 UTC m=+0.018886167 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:15:51.091 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:50 smithi133 podman[64865]: 2024-04-22 07:15:50.831882874 +0000 UTC m=+0.139888754 container create 3155cda1a7b142c8bd6801d0e4c952f219479ac21c7f24b463c1424326c9b388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-22T07:15:51.091 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:51 smithi133 podman[64865]: 2024-04-22 07:15:51.06765034 +0000 UTC m=+0.375656161 container init 3155cda1a7b142c8bd6801d0e4c952f219479ac21c7f24b463c1424326c9b388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:15:51.091 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:51 smithi133 podman[64865]: 2024-04-22 07:15:51.070668856 +0000 UTC m=+0.378674678 container start 3155cda1a7b142c8bd6801d0e4c952f219479ac21c7f24b463c1424326c9b388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-22T07:15:51.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3d37fe640 1 -- 172.21.15.113:0/743762653 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: -125 ) ==== 8+0+589 (crc 0 0 0) 0x5586c631c980 con 0x5586c64310b0 2024-04-22T07:15:51.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3d37fe640 10 client.15885 handle_command_reply: tid=0 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 client.15885 shutdown 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 msgr2=0x5586c63be820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63be820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0a4500 tx=0x7ff3ec09bf30 comp rx=0 tx=0).stop 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 shutdown_connections 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5586c64310b0 0x5586c64334d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586c63b9ec0 0x5586c63be820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586c63b12e0 0x5586c63be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3d37fe640 0 client.15885 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 --2- 172.21.15.113:0/743762653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586c63b09c0 0x5586c63c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 >> 172.21.15.113:0/743762653 conn(0x5586c63ab0f0 msgr2=0x5586c63acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 shutdown_connections 2024-04-22T07:15:51.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.100+0000 7ff3fd590740 1 -- 172.21.15.113:0/743762653 wait complete. 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:51.101+0000 7ff3fd590740 20 client.15885 trim_cache size 0 max 16384 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "epoch": 23, 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "leader": 24506, 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "set_version": 6, 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "sets": { 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "42b0cd47": { 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "version": 6, 2024-04-22T07:15:51.100 INFO:teuthology.orchestra.run.smithi113.stdout: "age_ref": 0.0, 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "CANCELED", 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "timeout": 60.0, 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "expiration": 78.8, 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "members": { 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "file:/": { 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "excluded": true, 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "QUIESCING", 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 11.2 2024-04-22T07:15:51.101 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:51.102 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:51.102 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:51.102 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:51.102 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:15:51.102 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:51.102 INFO:teuthology.orchestra.run.smithi113.stderr:Error ECANCELED: 2024-04-22T07:15:51.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-04-22T07:15:51.104 WARNING:tasks.quiescer.fs.[cephfs]:Quiesce set got cancelled (cap = 10).Won't raise an error since this could be a failover, will wait for the next quiesce attempt 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 podman[66658]: 2024-04-22 07:15:51.08552052 +0000 UTC m=+0.396080469 container init d376c651472949bbbe92e52948b5ce8c9e2255b28b44c480cdab77abc96fa5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 podman[66658]: 2024-04-22 07:15:51.08840652 +0000 UTC m=+0.398966466 container start d376c651472949bbbe92e52948b5ce8c9e2255b28b44c480cdab77abc96fa5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux ) 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mds[66677]: -- 172.21.15.154:0/3731405017 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558a1ace64e0 con 0x558a1acbcc00 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mds[66677]: -- 172.21.15.154:0/3731405017 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a1ace9a40 con 0x558a1acbcc00 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[66673]: starting mds.i at 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 bash[66658]: d376c651472949bbbe92e52948b5ce8c9e2255b28b44c480cdab77abc96fa5c5 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:15:51 smithi154 systemd[1]: Started Ceph mds.i for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:15:51.359 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: daemon mds.h restarted 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:boot 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: fsmap cephfs:2/3 {0=b=up:active,2=i=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: daemon mds.i restarted 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:boot 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: Standby daemon mds.f assigned to filesystem cephfs as rank 2 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: fsmap cephfs:2/3 {0=b=up:active,1=c=up:replay} 2 up:standby-replay 8 up:standby, 1 failed 2024-04-22T07:15:51.360 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:15:51.361 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:51 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=f=up:replay} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:51.420 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mds[64885]: -- 172.21.15.133:0/2209467121 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646496dba40 con 0x5646496ae800 2024-04-22T07:15:51.420 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[64880]: starting mds.h at 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:51 smithi133 bash[64865]: 3155cda1a7b142c8bd6801d0e4c952f219479ac21c7f24b463c1424326c9b388 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:15:51 smithi133 systemd[1]: Started Ceph mds.h for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: daemon mds.h restarted 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:boot 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: fsmap cephfs:2/3 {0=b=up:active,2=i=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: daemon mds.i restarted 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T07:15:51.421 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:boot 2024-04-22T07:15:51.422 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: Standby daemon mds.f assigned to filesystem cephfs as rank 2 2024-04-22T07:15:51.422 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: fsmap cephfs:2/3 {0=b=up:active,1=c=up:replay} 2 up:standby-replay 8 up:standby, 1 failed 2024-04-22T07:15:51.422 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:15:51.422 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:51 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=f=up:replay} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:51.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-22T07:15:51.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: daemon mds.h restarted 2024-04-22T07:15:51.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:boot 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: fsmap cephfs:2/3 {0=b=up:active,2=i=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: daemon mds.i restarted 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:boot 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: Standby daemon mds.f assigned to filesystem cephfs as rank 2 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: fsmap cephfs:2/3 {0=b=up:active,1=c=up:replay} 2 up:standby-replay 8 up:standby, 1 failed 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:15:51.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:51 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=f=up:replay} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:53 smithi133 ceph-mon[31198]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:15:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:resolve 2024-04-22T07:15:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:resolve 2024-04-22T07:15:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:53 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=f=up:resolve} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:53 smithi154 ceph-mon[31862]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:15:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:resolve 2024-04-22T07:15:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:resolve 2024-04-22T07:15:53.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:53 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=f=up:resolve} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:53 smithi113 ceph-mon[27933]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-22T07:15:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:resolve 2024-04-22T07:15:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:resolve 2024-04-22T07:15:53.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:53 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=f=up:resolve} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.15579 172.21.15.113:0/567144824 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.15342 172.21.15.113:0/612866186 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.24916 172.21.15.113:0/2373633208 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.24922 172.21.15.113:0/3083165914 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: reconnect by client.24922 172.21.15.113:0/3083165914 after 0.000999973 2024-04-22T07:15:54.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:reconnect 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:reconnect 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:reconnect,2=f=up:reconnect} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:rejoin 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:rejoin 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:54 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:rejoin,2=f=up:rejoin} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.15579 172.21.15.113:0/567144824 after 0.000999973 2024-04-22T07:15:54.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.15342 172.21.15.113:0/612866186 after 0.000999973 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999973 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999973 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.24916 172.21.15.113:0/2373633208 after 0.000999973 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.24922 172.21.15.113:0/3083165914 after 0.000999973 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: reconnect by client.24922 172.21.15.113:0/3083165914 after 0.000999973 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:reconnect 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:reconnect 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:reconnect,2=f=up:reconnect} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:rejoin 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:rejoin 2024-04-22T07:15:54.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:54 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:rejoin,2=f=up:rejoin} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.15579 172.21.15.113:0/567144824 after 0.000999973 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.15342 172.21.15.113:0/612866186 after 0.000999973 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999973 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999973 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.24916 172.21.15.113:0/2373633208 after 0.000999973 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.24922 172.21.15.113:0/3083165914 after 0.000999973 2024-04-22T07:15:54.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: reconnect by client.24922 172.21.15.113:0/3083165914 after 0.000999973 2024-04-22T07:15:54.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:reconnect 2024-04-22T07:15:54.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:reconnect 2024-04-22T07:15:54.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:reconnect,2=f=up:reconnect} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:54.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:rejoin 2024-04-22T07:15:54.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:rejoin 2024-04-22T07:15:54.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:54 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:rejoin,2=f=up:rejoin} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: Cluster is now healthy 2024-04-22T07:15:55.485 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:active 2024-04-22T07:15:55.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:active 2024-04-22T07:15:55.486 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:55 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:15:55.375+0000 7f5d572bd700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:15:55.375+0000 7f5d572bd700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:15:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: Cluster is now healthy 2024-04-22T07:15:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:active 2024-04-22T07:15:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:active 2024-04-22T07:15:55.565 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:55 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:55.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: Cluster is now healthy 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] up:active 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] up:active 2024-04-22T07:15:55.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:55 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 1 up:standby-replay 8 up:standby 2024-04-22T07:15:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:56 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:15:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:15:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:15:56.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:56 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:15:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:15:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:15:56.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:56 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:15:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:15:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:15:56.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:15:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:57 smithi133 ceph-mon[31198]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 639 B/s wr, 26 op/s 2024-04-22T07:15:57.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:57 smithi154 ceph-mon[31862]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 639 B/s wr, 26 op/s 2024-04-22T07:15:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:57 smithi113 ceph-mon[27933]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 639 B/s wr, 26 op/s 2024-04-22T07:15:58.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:15:58.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 -- 172.21.15.113:0/2678944123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201072b0 msgr2=0x7f232010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 --2- 172.21.15.113:0/2678944123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201072b0 0x7f232010ee30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2318009ca0 tx=0x7f231802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 -- 172.21.15.113:0/2678944123 shutdown_connections 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 --2- 172.21.15.113:0/2678944123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201072b0 0x7f232010ee30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 --2- 172.21.15.113:0/2678944123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23201010a0 0x7f2320106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 --2- 172.21.15.113:0/2678944123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23201006f0 0x7f2320100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 -- 172.21.15.113:0/2678944123 >> 172.21.15.113:0/2678944123 conn(0x7f2320077360 msgr2=0x7f23200fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 -- 172.21.15.113:0/2678944123 shutdown_connections 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.019+0000 7f2327167640 1 -- 172.21.15.113:0/2678944123 wait complete. 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 Processor -- start 2024-04-22T07:15:59.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 -- start start 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23201006f0 0x7f2320101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23201072b0 0x7f2320101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 0x7f23201a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2320112300 con 0x7f23201020e0 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2320112180 con 0x7f23201072b0 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2327167640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2320112480 con 0x7f23201006f0 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 0x7f23201a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 0x7f23201a6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52636/0 (socket says 172.21.15.113:52636) 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 -- 172.21.15.113:0/1386957701 learned_addr learned my addr 172.21.15.113:0/1386957701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2324f03640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23201006f0 0x7f2320101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f231ffff640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23201072b0 0x7f2320101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 -- 172.21.15.113:0/1386957701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23201006f0 msgr2=0x7f2320101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23201006f0 0x7f2320101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 -- 172.21.15.113:0/1386957701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23201072b0 msgr2=0x7f2320101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23201072b0 0x7f2320101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.020+0000 7f2325704640 1 -- 172.21.15.113:0/1386957701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23201a70a0 con 0x7f23201020e0 2024-04-22T07:15:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.021+0000 7f2325704640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 0x7f23201a6a30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f231802c700 tx=0x7f2318035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.021+0000 7f231dffb640 1 -- 172.21.15.113:0/1386957701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318039040 con 0x7f23201020e0 2024-04-22T07:15:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.021+0000 7f231dffb640 1 -- 172.21.15.113:0/1386957701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2318044070 con 0x7f23201020e0 2024-04-22T07:15:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.021+0000 7f231dffb640 1 -- 172.21.15.113:0/1386957701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23180043e0 con 0x7f23201020e0 2024-04-22T07:15:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.021+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23201a7330 con 0x7f23201020e0 2024-04-22T07:15:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.021+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23201a77d0 con 0x7f23201020e0 2024-04-22T07:15:59.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.023+0000 7f2329e37740 1 Processor -- start 2024-04-22T07:15:59.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f231dffb640 1 -- 172.21.15.113:0/1386957701 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f2318004580 con 0x7f23201020e0 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 -- start start 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3bb580 0x563d5e3bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 0x563d5e3c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563d5e3201e0 con 0x563d5e3bbea0 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563d5e2bd390 con 0x563d5e3c4a80 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2329e37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563d5e31c1a0 con 0x563d5e3bb580 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 0x563d5e3c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 0x563d5e3c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52638/0 (socket says 172.21.15.113:52638) 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 -- 172.21.15.113:0/4173864024 learned_addr learned my addr 172.21.15.113:0/4173864024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f231ffff640 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f231dffb640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23100a3ed0 0x7f23100a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f231ffff640 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3bb580 0x563d5e3bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 -- 172.21.15.113:0/4173864024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3bb580 msgr2=0x563d5e3bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3bb580 0x563d5e3bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 -- 172.21.15.113:0/4173864024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 msgr2=0x563d5e3c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f2324f03640 1 -- 172.21.15.113:0/4173864024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5e0fc7d0 con 0x563d5e3bbea0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.024+0000 7f231ffff640 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3bb580 0x563d5e3bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f2325704640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23100a3ed0 0x7f23100a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f231dffb640 1 -- 172.21.15.113:0/1386957701 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f23180f3010 con 0x7f23201020e0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f2324f03640 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 0x563d5e3c4540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f231400b760 tx=0x7f231400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17403c001745b180 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f230affd640 1 -- 172.21.15.113:0/4173864024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314003dd0 con 0x563d5e3bbea0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f2325704640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23100a3ed0 0x7f23100a6390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23180f0610 tx=0x7f23180b4290 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f230affd640 1 -- 172.21.15.113:0/4173864024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2314015070 con 0x563d5e3bbea0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.025+0000 7f230affd640 1 -- 172.21.15.113:0/4173864024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314004e50 con 0x563d5e3bbea0 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 -- 172.21.15.113:0/4173864024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 msgr2=0x563d5e3c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 0x563d5e3c4540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f231400b760 tx=0x7f231400bc20 comp rx=0 tx=0).stop 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 -- 172.21.15.113:0/4173864024 shutdown_connections 2024-04-22T07:15:59.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bbea0 0x563d5e3c4540 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 --2- 172.21.15.113:0/4173864024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3bb580 0x563d5e3bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 -- 172.21.15.113:0/4173864024 >> 172.21.15.113:0/4173864024 conn(0x563d5e3b5cb0 msgr2=0x563d5e3b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 -- 172.21.15.113:0/4173864024 shutdown_connections 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 -- 172.21.15.113:0/4173864024 wait complete. 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 -- start start 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.026+0000 7f2329e37740 1 Processor -- start 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563d5e3c7870 con 0x563d5e3bb580 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52652/0 (socket says 172.21.15.113:52652) 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 -- 172.21.15.113:0/3453014821 learned_addr learned my addr 172.21.15.113:0/3453014821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563d5e2d92f0 con 0x563d5e3c2630 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563d5e3c79f0 con 0x563d5e3c4a80 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2329e37740 10 client.0 _ll_register_callbacks cb 0x563d5e2e6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2324f03640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 -- 172.21.15.113:0/3453014821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 msgr2=0x563d5e3c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 -- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 msgr2=0x563d5e3d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 -- 172.21.15.113:0/3453014821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5e0323f0 con 0x563d5e3bb580 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f2324f03640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f231ffff640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c1bb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f230c004590 tx=0x7f230c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5751d4151d2f6d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.027+0000 7f22fffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f22fffff640 1 -- 172.21.15.113:0/3453014821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c005220 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f22fffff640 1 -- 172.21.15.113:0/3453014821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f230c0053c0 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2314015070 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f22fffff640 1 -- 172.21.15.113:0/3453014821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f230c022050 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 10 client.15909 fetch_fsmap learned FSMap version 31 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d5dff20f0 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f22fffff640 1 -- 172.21.15.113:0/3453014821 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2314015070 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f22fffff640 1 -- 172.21.15.113:0/3453014821 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f230c012870 con 0x563d5e3bb580 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 10 client.15909 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 20 client.15909 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e3d4fd0 0x563d5e3d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 4 client.15909 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:59.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.028+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d5e32dc00 con 0x563d5e3d4fd0 2024-04-22T07:15:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.029+0000 7f2324f03640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e3d4fd0 0x563d5e3d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.029+0000 7f22fffff640 10 client.15909 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.029+0000 7f2324f03640 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e3d4fd0 0x563d5e3d7450 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b03ca43ffbc607 server_cookie=a9506ab997de80fc in_seq=0 out_seq=0 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f22fffff640 1 -- 172.21.15.113:0/3453014821 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d5e32dc00 con 0x563d5e3d4fd0 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f22fffff640 10 client.15909 handle_command_reply: tid=0 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 client.15909 shutdown 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 msgr2=0x563d5e3c1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c1bb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f230c004590 tx=0x7f230c0035f0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 shutdown_connections 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e3d4fd0 0x563d5e3d7450 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 --2- 172.21.15.113:0/3453014821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c1bb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 >> 172.21.15.113:0/3453014821 conn(0x563d5e3b5cb0 msgr2=0x563d5e3b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f22fffff640 0 client.15909 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 shutdown_connections 2024-04-22T07:15:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 1 -- 172.21.15.113:0/3453014821 wait complete. 2024-04-22T07:15:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.030+0000 7f2329e37740 20 client.15909 trim_cache size 0 max 16384 2024-04-22T07:15:59.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.063+0000 7f2329e37740 1 Processor -- start 2024-04-22T07:15:59.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.063+0000 7f2329e37740 1 -- start start 2024-04-22T07:15:59.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2329e37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2329e37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c2630 0x563d5e3dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2329e37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2329e37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23201b2780 con 0x563d5e3bb580 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2329e37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23201b0050 con 0x563d5e3c4a80 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2329e37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23201adea0 con 0x563d5e3c2630 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3b7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52654/0 (socket says 172.21.15.113:52654) 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 -- 172.21.15.113:0/3838531406 learned_addr learned my addr 172.21.15.113:0/3838531406 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f2324f03640 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c2630 0x563d5e3dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 -- 172.21.15.113:0/3838531406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c2630 msgr2=0x563d5e3dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c2630 0x563d5e3dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 -- 172.21.15.113:0/3838531406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 msgr2=0x563d5e3dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3dd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 -- 172.21.15.113:0/3838531406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5e3c0100 con 0x563d5e3bb580 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f231ffff640 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3b7900 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f230c003bf0 tx=0x7f230c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317f778709035818 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.064+0000 7f230a7fc640 1 -- 172.21.15.113:0/3838531406 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c0127c0 con 0x563d5e3bb580 2024-04-22T07:15:59.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f230a7fc640 1 -- 172.21.15.113:0/3838531406 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f230c02f070 con 0x563d5e3bb580 2024-04-22T07:15:59.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f230a7fc640 1 -- 172.21.15.113:0/3838531406 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c026670 con 0x563d5e3bb580 2024-04-22T07:15:59.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 -- 172.21.15.113:0/3838531406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 msgr2=0x563d5e3b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3b7900 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f230c003bf0 tx=0x7f230c004280 comp rx=0 tx=0).stop 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 -- 172.21.15.113:0/3838531406 shutdown_connections 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c4a80 0x563d5e3dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c2630 0x563d5e3dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 --2- 172.21.15.113:0/3838531406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3b7900 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 -- 172.21.15.113:0/3838531406 >> 172.21.15.113:0/3838531406 conn(0x563d5e3b5cb0 msgr2=0x563d5e3b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 -- 172.21.15.113:0/3838531406 shutdown_connections 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.065+0000 7f2329e37740 1 -- 172.21.15.113:0/3838531406 wait complete. 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 -- start start 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 Processor -- start 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f23201ae510 con 0x563d5e3bb580 2024-04-22T07:15:59.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23201af980 con 0x563d5e3c2630 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23201aebe0 con 0x563d5e3c4a80 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f2329e37740 10 client.0 _ll_register_callbacks cb 0x7f230c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f231ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f231ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52658/0 (socket says 172.21.15.113:52658) 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.066+0000 7f231ffff640 1 -- 172.21.15.113:0/3755666136 learned_addr learned my addr 172.21.15.113:0/3755666136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f2324f03640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 -- 172.21.15.113:0/3755666136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 msgr2=0x563d5e3c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 -- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 msgr2=0x563d5e3c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 -- 172.21.15.113:0/3755666136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5e43a6c0 con 0x563d5e3bb580 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f231ffff640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c8780 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f230c004330 tx=0x7f230c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af65baf4c215e947 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f22fffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f22fffff640 1 -- 172.21.15.113:0/3755666136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c01d410 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f22fffff640 1 -- 172.21.15.113:0/3755666136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f230c01d5b0 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230c02f070 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f22fffff640 1 -- 172.21.15.113:0/3755666136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f230c046050 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f2329e37740 10 client.15915 fetch_fsmap learned FSMap version 31 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.067+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d5e43b9b0 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f22fffff640 1 -- 172.21.15.113:0/3755666136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f230c02f070 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f22fffff640 1 -- 172.21.15.113:0/3755666136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f230c027840 con 0x563d5e3bb580 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2329e37740 10 client.15915 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2329e37740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2329e37740 20 client.15915 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2329e37740 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e43bfb0 0x563d5e43e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2329e37740 4 client.15915 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d5e327a70 con 0x563d5e43bfb0 2024-04-22T07:15:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.068+0000 7f2324f03640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e43bfb0 0x563d5e43e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2324f03640 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e43bfb0 0x563d5e43e3d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b399e049a84fc7f server_cookie=2a47924ad88d5959 in_seq=0 out_seq=0 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f22fffff640 10 client.15915 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f22fffff640 1 -- 172.21.15.113:0/3755666136 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d5e327a70 con 0x563d5e43bfb0 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f22fffff640 10 client.15915 handle_command_reply: tid=0 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 client.15915 shutdown 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 msgr2=0x563d5e3c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c8780 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f230c004330 tx=0x7f230c01f040 comp rx=0 tx=0).stop 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 shutdown_connections 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d5e3c4a80 0x563d5e3c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5e3c2630 0x563d5e3c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563d5e43bfb0 0x563d5e43e3d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 --2- 172.21.15.113:0/3755666136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d5e3bb580 0x563d5e3c8780 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 >> 172.21.15.113:0/3755666136 conn(0x563d5e3b5cb0 msgr2=0x563d5e3b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f22fffff640 0 client.15915 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:15:59.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 shutdown_connections 2024-04-22T07:15:59.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 1 -- 172.21.15.113:0/3755666136 wait complete. 2024-04-22T07:15:59.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.069+0000 7f2329e37740 20 client.15915 trim_cache size 0 max 16384 2024-04-22T07:15:59.073 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:15:59.073 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:15:59.073 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:15:59.073 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23100a3ed0 msgr2=0x7f23100a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23100a3ed0 0x7f23100a6390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23180f0610 tx=0x7f23180b4290 comp rx=0 tx=0).stop 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 msgr2=0x7f23201a6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 0x7f23201a6a30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f231802c700 tx=0x7f2318035d80 comp rx=0 tx=0).stop 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 shutdown_connections 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f23100a3ed0 0x7f23100a6390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23201020e0 0x7f23201a6a30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23201072b0 0x7f2320101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 --2- 172.21.15.113:0/1386957701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23201006f0 0x7f2320101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 >> 172.21.15.113:0/1386957701 conn(0x7f2320077360 msgr2=0x7f2320105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 shutdown_connections 2024-04-22T07:15:59.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.070+0000 7f2327167640 1 -- 172.21.15.113:0/1386957701 wait complete. 2024-04-22T07:15:59.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:15:59.082 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:15:59.205 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:15:59 smithi113 ceph-mon[27933]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 639 B/s wr, 26 op/s 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 -- 172.21.15.113:0/2476857622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80109470 msgr2=0x7fec80111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 --2- 172.21.15.113:0/2476857622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80109470 0x7fec80111040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fec78009ca0 tx=0x7fec7802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 -- 172.21.15.113:0/2476857622 shutdown_connections 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 --2- 172.21.15.113:0/2476857622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec80109470 0x7fec80111040 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 --2- 172.21.15.113:0/2476857622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80104d50 0x7fec80108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 --2- 172.21.15.113:0/2476857622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec801043a0 0x7fec80104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 -- 172.21.15.113:0/2476857622 >> 172.21.15.113:0/2476857622 conn(0x7fec800ffc00 msgr2=0x7fec80102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 -- 172.21.15.113:0/2476857622 shutdown_connections 2024-04-22T07:15:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.214+0000 7fec87ea0640 1 -- 172.21.15.113:0/2476857622 wait complete. 2024-04-22T07:15:59.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 Processor -- start 2024-04-22T07:15:59.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 -- start start 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec801043a0 0x7fec801a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80109470 0x7fec801a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 0x7fec8019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec80114510 con 0x7fec8019a810 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fec80114390 con 0x7fec80109470 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec87ea0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fec80114690 con 0x7fec801043a0 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec8643d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 0x7fec8019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec8643d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 0x7fec8019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52676/0 (socket says 172.21.15.113:52676) 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec8643d640 1 -- 172.21.15.113:0/1984402257 learned_addr learned my addr 172.21.15.113:0/1984402257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec8543b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80109470 0x7fec801a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.215+0000 7fec85c3c640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec801043a0 0x7fec801a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8643d640 1 -- 172.21.15.113:0/1984402257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec801043a0 msgr2=0x7fec801a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8643d640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec801043a0 0x7fec801a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8643d640 1 -- 172.21.15.113:0/1984402257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80109470 msgr2=0x7fec801a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8643d640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80109470 0x7fec801a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8643d640 1 -- 172.21.15.113:0/1984402257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8019b3a0 con 0x7fec8019a810 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8543b640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80109470 0x7fec801a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec8643d640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 0x7fec8019acc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fec78037040 tx=0x7fec780323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78039040 con 0x7fec8019a810 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fec78044070 con 0x7fec8019a810 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8019b630 con 0x7fec8019a810 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec780042a0 con 0x7fec8019a810 2024-04-22T07:15:59.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.216+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec801a77e0 con 0x7fec8019a810 2024-04-22T07:15:59.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.217+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec80105870 con 0x7fec8019a810 2024-04-22T07:15:59.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.219+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fec78004440 con 0x7fec8019a810 2024-04-22T07:15:59.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.220+0000 7fec76ffd640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fec6c0a3ed0 0x7fec6c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.220+0000 7fec85c3c640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fec6c0a3ed0 0x7fec6c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.220+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fec780f34b0 con 0x7fec8019a810 2024-04-22T07:15:59.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.220+0000 7fec85c3c640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fec6c0a3ed0 0x7fec6c0a6390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009bd0 tx=0x7fec7c005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.224+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec780bf9a0 con 0x7fec8019a810 2024-04-22T07:15:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:15:59 smithi133 ceph-mon[31198]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 639 B/s wr, 26 op/s 2024-04-22T07:15:59.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:15:59 smithi154 ceph-mon[31862]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 639 B/s wr, 26 op/s 2024-04-22T07:15:59.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.594+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec80104d50 con 0x7fec8019a810 2024-04-22T07:15:59.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.595+0000 7fec76ffd640 1 -- 172.21.15.113:0/1984402257 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11884 (secure 0 0 0) 0x7fec780bf2a0 con 0x7fec8019a810 2024-04-22T07:15:59.593 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:15:59.593 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:15:55.192202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24445,"mds_2":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":26,"state":"up:active","state_seq":54,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":24,"state":"up:active","state_seq":53,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24433,24445,24506]},"id":1}]} 2024-04-22T07:15:59.593 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fec6c0a3ed0 msgr2=0x7fec6c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fec6c0a3ed0 0x7fec6c0a6390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009bd0 tx=0x7fec7c005f90 comp rx=0 tx=0).stop 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 msgr2=0x7fec8019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 0x7fec8019acc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fec78037040 tx=0x7fec780323d0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 shutdown_connections 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fec6c0a3ed0 0x7fec6c0a6390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8019a810 0x7fec8019acc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec80109470 0x7fec801a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 --2- 172.21.15.113:0/1984402257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec801043a0 0x7fec801a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 >> 172.21.15.113:0/1984402257 conn(0x7fec800ffc00 msgr2=0x7fec80101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 shutdown_connections 2024-04-22T07:15:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.598+0000 7fec87ea0640 1 -- 172.21.15.113:0/1984402257 wait complete. 2024-04-22T07:15:59.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:15:59.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 -- 172.21.15.113:0/2198699425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 msgr2=0x7f6ddc10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 --2- 172.21.15.113:0/2198699425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc10e9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009ca0 tx=0x7f6dd402c180 comp rx=0 tx=0).stop 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 -- 172.21.15.113:0/2198699425 shutdown_connections 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 --2- 172.21.15.113:0/2198699425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc10e9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 --2- 172.21.15.113:0/2198699425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103970 0x7f6ddc10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 --2- 172.21.15.113:0/2198699425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103050 0x7f6ddc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 -- 172.21.15.113:0/2198699425 >> 172.21.15.113:0/2198699425 conn(0x7f6ddc077fb0 msgr2=0x7f6ddc0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 -- 172.21.15.113:0/2198699425 shutdown_connections 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.746+0000 7f6de106a640 1 -- 172.21.15.113:0/2198699425 wait complete. 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 Processor -- start 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 -- start start 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103050 0x7f6ddc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103970 0x7f6ddc19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1122e0 con 0x7f6ddc10c570 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc112160 con 0x7f6ddc103050 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6de106a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc112460 con 0x7f6ddc103970 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.747+0000 7f6ddb59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52690/0 (socket says 172.21.15.113:52690) 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 -- 172.21.15.113:0/2921375619 learned_addr learned my addr 172.21.15.113:0/2921375619 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103050 0x7f6ddc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103970 0x7f6ddc19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 -- 172.21.15.113:0/2921375619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103970 msgr2=0x7f6ddc19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103970 0x7f6ddc19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 -- 172.21.15.113:0/2921375619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103050 msgr2=0x7f6ddc19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103050 0x7f6ddc19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 -- 172.21.15.113:0/2921375619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc198da0 con 0x7f6ddc10c570 2024-04-22T07:15:59.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103970 0x7f6ddc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103050 0x7f6ddc19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6ddb59e640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc198620 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6dd402c790 tx=0x7f6dd4035ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6dd8d99640 1 -- 172.21.15.113:0/2921375619 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4039040 con 0x7f6ddc10c570 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6dd8d99640 1 -- 172.21.15.113:0/2921375619 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6dd4044070 con 0x7f6ddc10c570 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6dd8d99640 1 -- 172.21.15.113:0/2921375619 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4032900 con 0x7f6ddc10c570 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc199030 con 0x7f6ddc10c570 2024-04-22T07:15:59.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.748+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc1a52e0 con 0x7f6ddc10c570 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.751+0000 7f6de3c5e740 1 Processor -- start 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.751+0000 7f6dd8d99640 1 -- 172.21.15.113:0/2921375619 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f6dd4032aa0 con 0x7f6ddc10c570 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 -- start start 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 0x55a5867be8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 0x55a5867c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867c7a00 0x55a5867c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a5867231f0 con 0x55a5867bee80 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a5866c04b0 con 0x55a5867c7a00 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a58671f1b0 con 0x55a5867be4d0 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 0x55a5867be8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6ddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 0x55a5867c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 0x55a5867be8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:38380/0 (socket says 172.21.15.113:38380) 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6dd3fff640 1 -- 172.21.15.113:0/2363806673 learned_addr learned my addr 172.21.15.113:0/2363806673 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6ddad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 0x55a5867c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52696/0 (socket says 172.21.15.113:52696) 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.752+0000 7f6dd8d99640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6dc80a3ed0 0x7f6dc80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867c7a00 0x55a5867c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6dd8d99640 1 -- 172.21.15.113:0/2921375619 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f6dd40f2d60 con 0x7f6ddc10c570 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddb59e640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6dc80a3ed0 0x7f6dc80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddad9d640 1 -- 172.21.15.113:0/2363806673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 msgr2=0x55a5867be8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 0x55a5867be8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddad9d640 1 -- 172.21.15.113:0/2363806673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867c7a00 msgr2=0x55a5867c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867c7a00 0x55a5867c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddad9d640 1 -- 172.21.15.113:0/2363806673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5866b21b0 con 0x55a5867bee80 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 0x55a5867be8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddb59e640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6dc80a3ed0 0x7f6dc80a6390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40f0360 tx=0x7f6dd40895f0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 0x55a5867c7430 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00bc80 tx=0x7f6dcc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2311df94492944 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6dd17fa640 1 -- 172.21.15.113:0/2363806673 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc015070 con 0x55a5867bee80 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6dd17fa640 1 -- 172.21.15.113:0/2363806673 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6dcc011910 con 0x55a5867bee80 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.753+0000 7f6dd17fa640 1 -- 172.21.15.113:0/2363806673 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc010930 con 0x55a5867bee80 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 -- 172.21.15.113:0/2363806673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 msgr2=0x55a5867c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 0x55a5867c7430 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00bc80 tx=0x7f6dcc00f730 comp rx=0 tx=0).stop 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 -- 172.21.15.113:0/2363806673 shutdown_connections 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867c7a00 0x55a5867c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867bee80 0x55a5867c7430 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/2363806673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867be4d0 0x55a5867be8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 -- 172.21.15.113:0/2363806673 >> 172.21.15.113:0/2363806673 conn(0x55a5867b8ba0 msgr2=0x55a5867b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 -- 172.21.15.113:0/2363806673 shutdown_connections 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.754+0000 7f6de3c5e740 1 -- 172.21.15.113:0/2363806673 wait complete. 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 -- start start 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 Processor -- start 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52704/0 (socket says 172.21.15.113:52704) 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6dd3fff640 1 -- 172.21.15.113:0/483833695 learned_addr learned my addr 172.21.15.113:0/483833695 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a5867ca970 con 0x55a5867c7a00 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a5866dc500 con 0x55a5867be4d0 2024-04-22T07:15:59.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a5867caaf0 con 0x55a5867bee80 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.755+0000 7f6de3c5e740 10 client.0 _ll_register_callbacks cb 0x55a5866aab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 -- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 msgr2=0x55a5867c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 -- 172.21.15.113:0/483833695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 msgr2=0x55a5867c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5864353f0 con 0x55a5867c7a00 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c4cf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400bfe0 tx=0x7f6dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad59904570ba0398 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dc27fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dc27fc640 1 -- 172.21.15.113:0/483833695 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4024060 con 0x55a5867c7a00 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dc27fc640 1 -- 172.21.15.113:0/483833695 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f6dc401e310 con 0x55a5867c7a00 2024-04-22T07:15:59.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dcc011910 con 0x55a5867c7a00 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dc27fc640 1 -- 172.21.15.113:0/483833695 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6dc40115f0 con 0x55a5867c7a00 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6de3c5e740 10 client.15933 fetch_fsmap learned FSMap version 31 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a5864ff7d0 con 0x55a5867c7a00 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.756+0000 7f6dc27fc640 1 -- 172.21.15.113:0/483833695 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6dcc011910 con 0x55a5867c7a00 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6dc27fc640 1 -- 172.21.15.113:0/483833695 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f6dc401c930 con 0x55a5867c7a00 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6de3c5e740 10 client.15933 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6de3c5e740 10 client.15933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6de3c5e740 20 client.15933 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a5867da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6de3c5e740 4 client.15933 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 --> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5866c2930 con 0x55a5867d7ce0 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a5867da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6dc27fc640 10 client.15933 ms_handle_connect on v2:172.21.15.154:6834/217490889 2024-04-22T07:15:59.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.757+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a5867da0d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f67b2acfc6869f17 server_cookie=ddfc2681329ada20 in_seq=0 out_seq=0 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6dc27fc640 1 -- 172.21.15.113:0/483833695 <== mds.2 v2:172.21.15.154:6834/217490889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5866c2930 con 0x55a5867d7ce0 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6dc27fc640 10 client.15933 handle_command_reply: tid=0 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 client.15933 shutdown 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 msgr2=0x55a5867c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c4cf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400bfe0 tx=0x7f6dc40046c0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 shutdown_connections 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a5867da0d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6dc27fc640 0 client.15933 ms_handle_reset on v2:172.21.15.154:6834/217490889 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c4cf0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/483833695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.758+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 >> 172.21.15.113:0/483833695 conn(0x55a5867b8ba0 msgr2=0x55a5867ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.759+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 shutdown_connections 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.759+0000 7f6de3c5e740 1 -- 172.21.15.113:0/483833695 wait complete. 2024-04-22T07:15:59.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.759+0000 7f6de3c5e740 20 client.15933 trim_cache size 0 max 16384 2024-04-22T07:15:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.772+0000 7f6de3c5e740 1 Processor -- start 2024-04-22T07:15:59.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.772+0000 7f6de3c5e740 1 -- start start 2024-04-22T07:15:59.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 0x55a5867d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867bee80 0x55a5867b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867c7a00 0x55a5867c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1ae3b0 con 0x55a5867be4d0 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1abc80 con 0x55a5867bee80 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1a9ad0 con 0x55a5867c7a00 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 0x55a5867d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 0x55a5867d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52716/0 (socket says 172.21.15.113:52716) 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 -- 172.21.15.113:0/1555636515 learned_addr learned my addr 172.21.15.113:0/1555636515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867bee80 0x55a5867b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867c7a00 0x55a5867c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 -- 172.21.15.113:0/1555636515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867c7a00 msgr2=0x55a5867c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867c7a00 0x55a5867c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 -- 172.21.15.113:0/1555636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867bee80 msgr2=0x55a5867b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867bee80 0x55a5867b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.773+0000 7f6dd3fff640 1 -- 172.21.15.113:0/1555636515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5867bd7c0 con 0x55a5867be4d0 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.774+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 0x55a5867d3860 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4022040 tx=0x7f6dc4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44853077f64eb792 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.774+0000 7f6dd0ff9640 1 -- 172.21.15.113:0/1555636515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4011240 con 0x55a5867be4d0 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.774+0000 7f6dd0ff9640 1 -- 172.21.15.113:0/1555636515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6dc401e110 con 0x55a5867be4d0 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.774+0000 7f6dd0ff9640 1 -- 172.21.15.113:0/1555636515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc401c550 con 0x55a5867be4d0 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1555636515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 msgr2=0x55a5867d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 0x55a5867d3860 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4022040 tx=0x7f6dc4005850 comp rx=0 tx=0).stop 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1555636515 shutdown_connections 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867c7a00 0x55a5867c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867bee80 0x55a5867b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1555636515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867be4d0 0x55a5867d3860 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1555636515 >> 172.21.15.113:0/1555636515 conn(0x55a5867b8ba0 msgr2=0x55a5867b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1555636515 shutdown_connections 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1555636515 wait complete. 2024-04-22T07:15:59.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 -- start start 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 Processor -- start 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.775+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6de3c5e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1aa140 con 0x55a5867c7a00 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1ab5b0 con 0x55a5867be4d0 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6de3c5e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1aa810 con 0x55a5867bee80 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6de3c5e740 10 client.0 _ll_register_callbacks cb 0x7f6dc401e930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dd3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52730/0 (socket says 172.21.15.113:52730) 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dd3fff640 1 -- 172.21.15.113:0/1243136240 learned_addr learned my addr 172.21.15.113:0/1243136240 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 -- 172.21.15.113:0/1243136240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 msgr2=0x55a5867cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 -- 172.21.15.113:0/1243136240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 msgr2=0x55a5867c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 -- 172.21.15.113:0/1243136240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5867c18d0 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6ddad9d640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867d02b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a5867b9c10 tx=0x7f6dcc00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4efdc27eb00904a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dc27fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.776+0000 7f6dc27fc640 1 -- 172.21.15.113:0/1243136240 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc02e060 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6dc27fc640 1 -- 172.21.15.113:0/1243136240 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f6dcc005450 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc401e110 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6dc27fc640 1 -- 172.21.15.113:0/1243136240 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc015040 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 10 client.25294 fetch_fsmap learned FSMap version 31 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a58683dd80 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6dc27fc640 1 -- 172.21.15.113:0/1243136240 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6dc401e110 con 0x55a5867bee80 2024-04-22T07:15:59.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6dc27fc640 1 -- 172.21.15.113:0/1243136240 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f6dcc004820 con 0x55a5867bee80 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 10 client.25294 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 10 client.25294 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 20 client.25294 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a586840240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 4 client.25294 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 --> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a586730c10 con 0x55a5867d7ce0 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.777+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a586840240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6dc27fc640 10 client.25294 ms_handle_connect on v2:172.21.15.154:6834/217490889 2024-04-22T07:15:59.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6dd3fff640 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a586840240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d11bc0a0a59b36e0 server_cookie=76802d27582a05b7 in_seq=0 out_seq=0 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6dc27fc640 1 -- 172.21.15.113:0/1243136240 <== mds.2 v2:172.21.15.154:6834/217490889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a586730c10 con 0x55a5867d7ce0 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6dc27fc640 10 client.25294 handle_command_reply: tid=0 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 client.25294 shutdown 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 msgr2=0x55a5867d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867d02b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a5867b9c10 tx=0x7f6dcc00bef0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 shutdown_connections 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55a5867d7ce0 0x55a586840240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5867c7a00 0x55a5867c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a5867bee80 0x55a5867d02b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 --2- 172.21.15.113:0/1243136240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5867be4d0 0x55a5867cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 >> 172.21.15.113:0/1243136240 conn(0x55a5867b8ba0 msgr2=0x55a5867b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6dc27fc640 0 client.25294 ms_handle_reset on v2:172.21.15.154:6834/217490889 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 shutdown_connections 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 1 -- 172.21.15.113:0/1243136240 wait complete. 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.778+0000 7f6de3c5e740 20 client.25294 trim_cache size 0 max 16384 2024-04-22T07:15:59.786 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:15:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6dc80a3ed0 msgr2=0x7f6dc80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6dc80a3ed0 0x7f6dc80a6390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40f0360 tx=0x7f6dd40895f0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 msgr2=0x7f6ddc198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc198620 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6dd402c790 tx=0x7f6dd4035ec0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 shutdown_connections 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6dc80a3ed0 0x7f6dc80a6390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ddc10c570 0x7f6ddc198620 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ddc103970 0x7f6ddc19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 --2- 172.21.15.113:0/2921375619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ddc103050 0x7f6ddc19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 >> 172.21.15.113:0/2921375619 conn(0x7f6ddc077fb0 msgr2=0x7f6ddc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 shutdown_connections 2024-04-22T07:15:59.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.779+0000 7f6de106a640 1 -- 172.21.15.113:0/2921375619 wait complete. 2024-04-22T07:15:59.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:15:59.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.921+0000 7f67dece4640 1 -- 172.21.15.113:0/2075194608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 msgr2=0x7f67d810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.921+0000 7f67dece4640 1 --2- 172.21.15.113:0/2075194608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d810dd30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009ca0 tx=0x7f67d002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 -- 172.21.15.113:0/2075194608 shutdown_connections 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 --2- 172.21.15.113:0/2075194608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d810dd30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 --2- 172.21.15.113:0/2075194608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d8101a40 0x7f67d8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 --2- 172.21.15.113:0/2075194608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d8101090 0x7f67d8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 -- 172.21.15.113:0/2075194608 >> 172.21.15.113:0/2075194608 conn(0x7f67d8077f50 msgr2=0x7f67d8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 -- 172.21.15.113:0/2075194608 shutdown_connections 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 -- 172.21.15.113:0/2075194608 wait complete. 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 Processor -- start 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 -- start start 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d8101090 0x7f67d819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.922+0000 7f67dece4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d81985e0 0x7f67d8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dece4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8112300 con 0x7f67d8106160 2024-04-22T07:15:59.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dece4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8112180 con 0x7f67d8101090 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52748/0 (socket says 172.21.15.113:52748) 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 -- 172.21.15.113:0/1850279953 learned_addr learned my addr 172.21.15.113:0/1850279953 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8112480 con 0x7f67d81985e0 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dd281640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d81985e0 0x7f67d8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dca80640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d8101090 0x7f67d819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 -- 172.21.15.113:0/1850279953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d81985e0 msgr2=0x7f67d8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d81985e0 0x7f67d8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 -- 172.21.15.113:0/1850279953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d8101090 msgr2=0x7f67d819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d8101090 0x7f67d819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 -- 172.21.15.113:0/1850279953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d8199100 con 0x7f67d8106160 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dd281640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d81985e0 0x7f67d8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d819e970 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f67c4005fd0 tx=0x7f67c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d5ffb640 1 -- 172.21.15.113:0/1850279953 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4017070 con 0x7f67d8106160 2024-04-22T07:15:59.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d5ffb640 1 -- 172.21.15.113:0/1850279953 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f67c4009c30 con 0x7f67d8106160 2024-04-22T07:15:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67d5ffb640 1 -- 172.21.15.113:0/1850279953 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4003de0 con 0x7f67d8106160 2024-04-22T07:15:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.923+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81993f0 con 0x7f67d8106160 2024-04-22T07:15:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.924+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81a55f0 con 0x7f67d8106160 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.926+0000 7f67d5ffb640 1 -- 172.21.15.113:0/1850279953 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f67c4003f80 con 0x7f67d8106160 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.927+0000 7f67d5ffb640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f67c80a3f50 0x7f67c80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.927+0000 7f67d5ffb640 1 -- 172.21.15.113:0/1850279953 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f67c40d0630 con 0x7f67d8106160 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.927+0000 7f67dca80640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f67c80a3f50 0x7f67c80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.927+0000 7f67dca80640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f67c80a3f50 0x7f67c80a6410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009bd0 tx=0x7f67cc005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.927+0000 7f67e19b1740 1 Processor -- start 2024-04-22T07:15:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.927+0000 7f67e19b1740 1 -- start start 2024-04-22T07:15:59.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a33e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a347430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a349e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a347430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67e19b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56032a2a31f0 con 0x56032a33ee80 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a347430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52762/0 (socket says 172.21.15.113:52762) 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 -- 172.21.15.113:0/2499884746 learned_addr learned my addr 172.21.15.113:0/2499884746 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56032a2404b0 con 0x56032a347a00 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67dd281640 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a349e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67dca80640 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a33e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56032a29f1b0 con 0x56032a33e4d0 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 -- 172.21.15.113:0/2499884746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 msgr2=0x56032a33e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a33e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 -- 172.21.15.113:0/2499884746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 msgr2=0x56032a349e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.928+0000 7f67d7fff640 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a349e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.929+0000 7f67d7fff640 1 -- 172.21.15.113:0/2499884746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032a2321b0 con 0x56032a33ee80 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.929+0000 7f67d7fff640 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a347430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f67c4098770 tx=0x7f67c409cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0634a1a6bb2fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.929+0000 7f67c2ffd640 1 -- 172.21.15.113:0/2499884746 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40a06c0 con 0x56032a33ee80 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.929+0000 7f67c2ffd640 1 -- 172.21.15.113:0/2499884746 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f67c40a4f80 con 0x56032a33ee80 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.929+0000 7f67c2ffd640 1 -- 172.21.15.113:0/2499884746 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40cddd0 con 0x56032a33ee80 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 msgr2=0x56032a347430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a347430 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f67c4098770 tx=0x7f67c409cad0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 shutdown_connections 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a349e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a347430 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 --2- 172.21.15.113:0/2499884746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a33e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 >> 172.21.15.113:0/2499884746 conn(0x56032a338ba0 msgr2=0x56032a338f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 shutdown_connections 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 -- 172.21.15.113:0/2499884746 wait complete. 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 -- start start 2024-04-22T07:15:59.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.930+0000 7f67e19b1740 1 Processor -- start 2024-04-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 0x56032a351b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33ee80 0x56032a352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a34bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56032a34a970 con 0x56032a33e4d0 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56032a25c500 con 0x56032a347a00 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56032a34aaf0 con 0x56032a33ee80 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67e19b1740 10 client.0 _ll_register_callbacks cb 0x56032a22ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 0x56032a351b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 0x56032a351b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52766/0 (socket says 172.21.15.113:52766) 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 -- 172.21.15.113:0/1374647791 learned_addr learned my addr 172.21.15.113:0/1374647791 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33ee80 0x56032a352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dd281640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a34bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 -- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33ee80 msgr2=0x56032a352090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33ee80 0x56032a352090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 -- 172.21.15.113:0/1374647791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 msgr2=0x56032a34bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a34bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 -- 172.21.15.113:0/1374647791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560329fb53f0 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67dca80640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 0x56032a351b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f67cc003a10 tx=0x7f67cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d1d35ab5d1c72a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.931+0000 7f67b7fff640 1 -- 172.21.15.113:0/1374647791 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc028440 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67b7fff640 1 -- 172.21.15.113:0/1374647791 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f67cc0273e0 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c40a4f80 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67b7fff640 1 -- 172.21.15.113:0/1374647791 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67cc0285e0 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 10 client.15951 fetch_fsmap learned FSMap version 31 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56032a07f7d0 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67b7fff640 1 -- 172.21.15.113:0/1374647791 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f67c40a4f80 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67b7fff640 1 -- 172.21.15.113:0/1374647791 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f67cc026070 con 0x56032a33e4d0 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 10 client.15951 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 10 client.15951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 20 client.15951 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a35c0a0 0x56032a35e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 4 client.15951 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 --> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56032a242930 con 0x56032a35c0a0 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.932+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a35c0a0 0x56032a35e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.933+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a35c0a0 0x56032a35e490 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6fd9299a753f769 server_cookie=99f4fd2ede62a60d in_seq=0 out_seq=0 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.933+0000 7f67b7fff640 10 client.15951 ms_handle_connect on v2:172.21.15.154:6836/1615298521 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67b7fff640 1 -- 172.21.15.113:0/1374647791 <== mds.1 v2:172.21.15.154:6836/1615298521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56032a242930 con 0x56032a35c0a0 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67b7fff640 10 client.15951 handle_command_reply: tid=0 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 client.15951 shutdown 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 msgr2=0x56032a351b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 0x56032a351b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f67cc003a10 tx=0x7f67cc004730 comp rx=0 tx=0).stop 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 shutdown_connections 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a35c0a0 0x56032a35e490 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a34bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33ee80 0x56032a352090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67b7fff640 0 client.15951 ms_handle_reset on v2:172.21.15.154:6836/1615298521 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1374647791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33e4d0 0x56032a351b50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 >> 172.21.15.113:0/1374647791 conn(0x56032a338ba0 msgr2=0x56032a33a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 shutdown_connections 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 1 -- 172.21.15.113:0/1374647791 wait complete. 2024-04-22T07:15:59.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.934+0000 7f67e19b1740 20 client.15951 trim_cache size 0 max 16384 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 Processor -- start 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 -- start start 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a351db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a33ee80 0x56032a363c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 0x56032a364140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81ae4c0 con 0x56032a347a00 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81abd90 con 0x56032a33ee80 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67e19b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81a9be0 con 0x56032a33e4d0 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67dd281640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 0x56032a364140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67dd281640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 0x56032a364140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52776/0 (socket says 172.21.15.113:52776) 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67dd281640 1 -- 172.21.15.113:0/1051437560 learned_addr learned my addr 172.21.15.113:0/1051437560 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67d7fff640 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a33ee80 0x56032a363c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.958+0000 7f67dca80640 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a351db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67dd281640 1 -- 172.21.15.113:0/1051437560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 msgr2=0x56032a351db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67dd281640 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a351db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67dd281640 1 -- 172.21.15.113:0/1051437560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a33ee80 msgr2=0x56032a363c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67dd281640 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a33ee80 0x56032a363c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67dd281640 1 -- 172.21.15.113:0/1051437560 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56032a347790 con 0x56032a347a00 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67dd281640 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 0x56032a364140 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f67d0037040 tx=0x7f67d0032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e6260547fe3ead server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67c27fc640 1 -- 172.21.15.113:0/1051437560 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0039040 con 0x56032a347a00 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67c27fc640 1 -- 172.21.15.113:0/1051437560 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f67d0003e40 con 0x56032a347a00 2024-04-22T07:15:59.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.959+0000 7f67c27fc640 1 -- 172.21.15.113:0/1051437560 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0004800 con 0x56032a347a00 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 -- 172.21.15.113:0/1051437560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 msgr2=0x56032a364140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 0x56032a364140 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f67d0037040 tx=0x7f67d0032340 comp rx=0 tx=0).stop 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 -- 172.21.15.113:0/1051437560 shutdown_connections 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a347a00 0x56032a364140 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a33ee80 0x56032a363c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 --2- 172.21.15.113:0/1051437560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a351db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 -- 172.21.15.113:0/1051437560 >> 172.21.15.113:0/1051437560 conn(0x56032a338ba0 msgr2=0x56032a3516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 -- 172.21.15.113:0/1051437560 shutdown_connections 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 -- 172.21.15.113:0/1051437560 wait complete. 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 -- start start 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.960+0000 7f67e19b1740 1 Processor -- start 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a3582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a355680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a355bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81aa250 con 0x56032a33ee80 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81ab6c0 con 0x56032a347a00 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81aa920 con 0x56032a33e4d0 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67e19b1740 10 client.0 _ll_register_callbacks cb 0x7f67cc0036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a355680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a355680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52780/0 (socket says 172.21.15.113:52780) 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 -- 172.21.15.113:0/3202567454 learned_addr learned my addr 172.21.15.113:0/3202567454 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67dca80640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a3582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67dd281640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a355bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 -- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 msgr2=0x56032a3582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a3582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 -- 172.21.15.113:0/3202567454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 msgr2=0x56032a355bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a355bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67d7fff640 1 -- 172.21.15.113:0/3202567454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032a356200 con 0x56032a33ee80 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.961+0000 7f67dd281640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a355bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67d7fff640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a355680 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cdbb0 tx=0x7f67c400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9068cadb61f286 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67b7fff640 1 -- 172.21.15.113:0/3202567454 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40985b0 con 0x56032a33ee80 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67b7fff640 1 -- 172.21.15.113:0/3202567454 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f67c4098980 con 0x56032a33ee80 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56032a34b610 con 0x56032a33ee80 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67b7fff640 1 -- 172.21.15.113:0/3202567454 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67c40a4f80 con 0x56032a33ee80 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67e19b1740 10 client.15957 fetch_fsmap learned FSMap version 31 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.962+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56032a3c22d0 con 0x56032a33ee80 2024-04-22T07:15:59.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67b7fff640 1 -- 172.21.15.113:0/3202567454 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x56032a34b610 con 0x56032a33ee80 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67b7fff640 1 -- 172.21.15.113:0/3202567454 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f67c40a0160 con 0x56032a33ee80 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67e19b1740 10 client.15957 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67e19b1740 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67e19b1740 20 client.15957 populate_metadata read hostname 'smithi113' 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67e19b1740 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a3c26d0 0x56032a3c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67e19b1740 4 client.15957 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 --> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56032a2b0c10 con 0x56032a3c26d0 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.963+0000 7f67dca80640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a3c26d0 0x56032a3c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67b7fff640 10 client.15957 ms_handle_connect on v2:172.21.15.154:6836/1615298521 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67dca80640 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a3c26d0 0x56032a3c4ac0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=510daf68eeb8c917 server_cookie=7e36b7d9fbe77cb9 in_seq=0 out_seq=0 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67b7fff640 1 -- 172.21.15.113:0/3202567454 <== mds.1 v2:172.21.15.154:6836/1615298521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56032a2b0c10 con 0x56032a3c26d0 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67b7fff640 10 client.15957 handle_command_reply: tid=0 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67e19b1740 1 client.15957 shutdown 2024-04-22T07:15:59.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 msgr2=0x56032a355680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.964+0000 7f67e19b1740 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a355680 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cdbb0 tx=0x7f67c400bf60 comp rx=0 tx=0).stop 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 shutdown_connections 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x56032a3c26d0 0x56032a3c4ac0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56032a347a00 0x56032a355bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56032a33ee80 0x56032a355680 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67b7fff640 0 client.15957 ms_handle_reset on v2:172.21.15.154:6836/1615298521 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 --2- 172.21.15.113:0/3202567454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56032a33e4d0 0x56032a3582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 >> 172.21.15.113:0/3202567454 conn(0x56032a338ba0 msgr2=0x56032a3437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 shutdown_connections 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 1 -- 172.21.15.113:0/3202567454 wait complete. 2024-04-22T07:15:59.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67e19b1740 20 client.15957 trim_cache size 0 max 16384 2024-04-22T07:15:59.968 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:15:59.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f67c80a3f50 msgr2=0x7f67c80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.965+0000 7f67dece4640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f67c80a3f50 0x7f67c80a6410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009bd0 tx=0x7f67cc005f90 comp rx=0 tx=0).stop 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 msgr2=0x7f67d819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d819e970 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f67c4005fd0 tx=0x7f67c400dde0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 shutdown_connections 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f67c80a3f50 0x7f67c80a6410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67d81985e0 0x7f67d8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67d8106160 0x7f67d819e970 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 --2- 172.21.15.113:0/1850279953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d8101090 0x7f67d819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 >> 172.21.15.113:0/1850279953 conn(0x7f67d8077f50 msgr2=0x7f67d80fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 shutdown_connections 2024-04-22T07:15:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:15:59.966+0000 7f67dece4640 1 -- 172.21.15.113:0/1850279953 wait complete. 2024-04-22T07:15:59.975 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 -- 172.21.15.113:0/822327691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c106160 msgr2=0x7f5f3c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 --2- 172.21.15.113:0/822327691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c106160 0x7f5f3c10dd30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009ca0 tx=0x7f5f3402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 -- 172.21.15.113:0/822327691 shutdown_connections 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 --2- 172.21.15.113:0/822327691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c106160 0x7f5f3c10dd30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 --2- 172.21.15.113:0/822327691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c101a40 0x7f5f3c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 --2- 172.21.15.113:0/822327691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c101090 0x7f5f3c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 -- 172.21.15.113:0/822327691 >> 172.21.15.113:0/822327691 conn(0x7f5f3c077f50 msgr2=0x7f5f3c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 -- 172.21.15.113:0/822327691 shutdown_connections 2024-04-22T07:16:00.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.081+0000 7f5f40a06640 1 -- 172.21.15.113:0/822327691 wait complete. 2024-04-22T07:16:00.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 Processor -- start 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 -- start start 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 0x7f5f3c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c106160 0x7f5f3c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c19a0c0 0x7f5f3c1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1111e0 con 0x7f5f3c101090 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c111060 con 0x7f5f3c106160 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f40a06640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c111360 con 0x7f5f3c19a0c0 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 0x7f5f3c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 0x7f5f3c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52800/0 (socket says 172.21.15.113:52800) 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f3a59c640 1 -- 172.21.15.113:0/3476165012 learned_addr learned my addr 172.21.15.113:0/3476165012 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c106160 0x7f5f3c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.082+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c19a0c0 0x7f5f3c1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f3a59c640 1 -- 172.21.15.113:0/3476165012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c19a0c0 msgr2=0x7f5f3c1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c19a0c0 0x7f5f3c1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f3a59c640 1 -- 172.21.15.113:0/3476165012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c106160 msgr2=0x7f5f3c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c106160 0x7f5f3c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f3a59c640 1 -- 172.21.15.113:0/3476165012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c1a4e40 con 0x7f5f3c101090 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 0x7f5f3c1995a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c890 tx=0x7f5f2c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f2b7fe640 1 -- 172.21.15.113:0/3476165012 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c004220 con 0x7f5f3c101090 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f2b7fe640 1 -- 172.21.15.113:0/3476165012 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5f2c0043c0 con 0x7f5f3c101090 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c1a5130 con 0x7f5f3c101090 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f2b7fe640 1 -- 172.21.15.113:0/3476165012 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0049c0 con 0x7f5f3c101090 2024-04-22T07:16:00.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.083+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c1a5680 con 0x7f5f3c101090 2024-04-22T07:16:00.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.085+0000 7f5f2b7fe640 1 -- 172.21.15.113:0/3476165012 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f5f2c005140 con 0x7f5f3c101090 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.086+0000 7f5f2b7fe640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5f240a3f50 0x7f5f240a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.086+0000 7f5f2b7fe640 1 -- 172.21.15.113:0/3476165012 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f5f2c0d08c0 con 0x7f5f3c101090 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.086+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5f240a3f50 0x7f5f240a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.086+0000 7f5f435f4740 1 Processor -- start 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 -- start start 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5f240a3f50 0x7f5f240a6410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005e00 tx=0x7f5f30005c50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55ac9fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 0x55a55aca8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca8e10 0x55a55acab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a55ac001f0 con 0x55a55aca0290 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a55ab9d4b0 con 0x55a55aca8e10 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f435f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a55abfc1b0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 0x55a55aca8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 0x55a55aca8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52802/0 (socket says 172.21.15.113:52802) 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 -- 172.21.15.113:0/3829899203 learned_addr learned my addr 172.21.15.113:0/3829899203 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca8e10 0x55a55acab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55ac9fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 -- 172.21.15.113:0/3829899203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 msgr2=0x55a55ac9fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55ac9fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 -- 172.21.15.113:0/3829899203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca8e10 msgr2=0x55a55acab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca8e10 0x55a55acab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f39d9b640 1 -- 172.21.15.113:0/3829899203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55ab8f1b0 con 0x55a55aca0290 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55ac9fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.087+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca8e10 0x55a55acab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.088+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 0x55a55aca8840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000fdb0 tx=0x7f5f30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c2b812387c0c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.088+0000 7f5f28ff9640 1 -- 172.21.15.113:0/3829899203 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30022070 con 0x55a55aca0290 2024-04-22T07:16:00.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.088+0000 7f5f28ff9640 1 -- 172.21.15.113:0/3829899203 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5f3001e920 con 0x55a55aca0290 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f28ff9640 1 -- 172.21.15.113:0/3829899203 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30010a00 con 0x55a55aca0290 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 -- 172.21.15.113:0/3829899203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 msgr2=0x55a55aca8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 0x55a55aca8840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000fdb0 tx=0x7f5f30012000 comp rx=0 tx=0).stop 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 -- 172.21.15.113:0/3829899203 shutdown_connections 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca8e10 0x55a55acab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca0290 0x55a55aca8840 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 --2- 172.21.15.113:0/3829899203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55ac9fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 -- 172.21.15.113:0/3829899203 >> 172.21.15.113:0/3829899203 conn(0x55a55ac99fb0 msgr2=0x55a55ac9a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:00.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 -- 172.21.15.113:0/3829899203 shutdown_connections 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 -- 172.21.15.113:0/3829899203 wait complete. 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.089+0000 7f5f435f4740 1 -- start start 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 Processor -- start 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55aca5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acb5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a55acabdd0 con 0x55a55aca8e10 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a55abb9500 con 0x55a55aca61d0 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a55acabf50 con 0x55a55ac9f8e0 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f39d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55aca5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f39d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55aca5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52808/0 (socket says 172.21.15.113:52808) 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f39d9b640 1 -- 172.21.15.113:0/4204494079 learned_addr learned my addr 172.21.15.113:0/4204494079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:00.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acb5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 10 client.0 _ll_register_callbacks cb 0x55a55ab87b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3a59c640 1 -- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 msgr2=0x55a55acb5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acb5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3a59c640 1 -- 172.21.15.113:0/4204494079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 msgr2=0x55a55aca5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55aca5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f3a59c640 1 -- 172.21.15.113:0/4204494079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55a9123f0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55aca5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.090+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3001e920 con 0x55a55ac9f8e0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0cdd50 tx=0x7f5f2c09cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6382f060c4dc8bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f21ffb640 10 client.25321 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f21ffb640 1 -- 172.21.15.113:0/4204494079 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01d040 con 0x55a55ac9f8e0 2024-04-22T07:16:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f21ffb640 1 -- 172.21.15.113:0/4204494079 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f5f2c085ae0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f21ffb640 1 -- 172.21.15.113:0/4204494079 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f2c085d00 con 0x55a55ac9f8e0 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f435f4740 10 client.25321 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55a55a9dc7d0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.091+0000 7f5f21ffb640 1 -- 172.21.15.113:0/4204494079 <== mon.1 v2:172.21.15.154:3300/0 4 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f5f2c0bb280 con 0x55a55ac9f8e0 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f435f4740 10 client.25321 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f435f4740 10 client.25321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f435f4740 20 client.25321 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55acb92f0 0x55a55acbb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f435f4740 4 client.25321 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a55ab9f930 con 0x55a55acb92f0 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55acb92f0 0x55a55acbb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f21ffb640 10 client.25321 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:00.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.092+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55acb92f0 0x55a55acbb720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e7efa4c4ae75e2 server_cookie=613c6b8d0681e9be in_seq=0 out_seq=0 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f21ffb640 1 -- 172.21.15.113:0/4204494079 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a55ab9f930 con 0x55a55acb92f0 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f21ffb640 10 client.25321 handle_command_reply: tid=0 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 client.25321 shutdown 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 msgr2=0x55a55aca8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0cdd50 tx=0x7f5f2c09cd60 comp rx=0 tx=0).stop 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 shutdown_connections 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acb5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55acb92f0 0x55a55acbb720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55aca5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4204494079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 >> 172.21.15.113:0/4204494079 conn(0x55a55ac99fb0 msgr2=0x55a55ac9bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f21ffb640 0 client.25321 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 shutdown_connections 2024-04-22T07:16:00.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 1 -- 172.21.15.113:0/4204494079 wait complete. 2024-04-22T07:16:00.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.093+0000 7f5f435f4740 20 client.25321 trim_cache size 0 max 16384 2024-04-22T07:16:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 Processor -- start 2024-04-22T07:16:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 -- start start 2024-04-22T07:16:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acc0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55acc1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1ae430 con 0x55a55aca8e10 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1abd00 con 0x55a55aca61d0 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.118+0000 7f5f435f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1a9b50 con 0x55a55ac9f8e0 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55acc1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55acc1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52814/0 (socket says 172.21.15.113:52814) 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 -- 172.21.15.113:0/4071252448 learned_addr learned my addr 172.21.15.113:0/4071252448 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acc0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 -- 172.21.15.113:0/4071252448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 msgr2=0x55a55aca8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 -- 172.21.15.113:0/4071252448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 msgr2=0x55a55acc0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acc0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 -- 172.21.15.113:0/4071252448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55ac9f4c0 con 0x55a55aca8e10 2024-04-22T07:16:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acc0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:00.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55acc1530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009ca0 tx=0x7f5f34009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df77c7d0e3ffe3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:00.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f23fff640 1 -- 172.21.15.113:0/4071252448 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34045030 con 0x55a55aca8e10 2024-04-22T07:16:00.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f23fff640 1 -- 172.21.15.113:0/4071252448 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5f34032c60 con 0x55a55aca8e10 2024-04-22T07:16:00.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.119+0000 7f5f23fff640 1 -- 172.21.15.113:0/4071252448 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34039040 con 0x55a55aca8e10 2024-04-22T07:16:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 -- 172.21.15.113:0/4071252448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 msgr2=0x55a55acc1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55acc1530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009ca0 tx=0x7f5f34009360 comp rx=0 tx=0).stop 2024-04-22T07:16:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 -- 172.21.15.113:0/4071252448 shutdown_connections 2024-04-22T07:16:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55aca8e10 0x55a55acc1530 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55acc0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 --2- 172.21.15.113:0/4071252448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ac9f8e0 0x55a55aca8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 -- 172.21.15.113:0/4071252448 >> 172.21.15.113:0/4071252448 conn(0x55a55ac99fb0 msgr2=0x55a55aca0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 -- 172.21.15.113:0/4071252448 shutdown_connections 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.120+0000 7f5f435f4740 1 -- 172.21.15.113:0/4071252448 wait complete. 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 -- start start 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 Processor -- start 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 0x55a55acb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55aca3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55aca8e10 0x55a55aca3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1aa1c0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1ab630 con 0x55a55aca61d0 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c1aa890 con 0x55a55aca8e10 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f435f4740 10 client.0 _ll_register_callbacks cb 0x7f5f2c09bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 0x55a55acb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 0x55a55acb1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52830/0 (socket says 172.21.15.113:52830) 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 -- 172.21.15.113:0/375220856 learned_addr learned my addr 172.21.15.113:0/375220856 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55aca3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3ad9d640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55aca8e10 0x55a55aca3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 -- 172.21.15.113:0/375220856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55aca8e10 msgr2=0x55a55aca3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55aca8e10 0x55a55aca3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 -- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 msgr2=0x55a55aca3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55aca3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.121+0000 7f5f3a59c640 1 -- 172.21.15.113:0/375220856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55aca1ed0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f3a59c640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 0x55a55acb1760 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c098a00 tx=0x7f5f2c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1c1423146e155a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f21ffb640 1 -- 172.21.15.113:0/375220856 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01d070 con 0x55a55ac9f8e0 2024-04-22T07:16:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f21ffb640 1 -- 172.21.15.113:0/375220856 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f5f2c0984f0 con 0x55a55ac9f8e0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f34032c60 con 0x55a55ac9f8e0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f21ffb640 1 -- 172.21.15.113:0/375220856 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f2c0a5210 con 0x55a55ac9f8e0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 10 client.15975 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a55ad1ec70 con 0x55a55ac9f8e0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f21ffb640 1 -- 172.21.15.113:0/375220856 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5f34032c60 con 0x55a55ac9f8e0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f21ffb640 1 -- 172.21.15.113:0/375220856 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f5f2c0ce060 con 0x55a55ac9f8e0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 10 client.15975 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 20 client.15975 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55ad1f340 0x55a55ad21750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 4 client.15975 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.122+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a55ac0dc10 con 0x55a55ad1f340 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55ad1f340 0x55a55ad21750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f39d9b640 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55ad1f340 0x55a55ad21750 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c9d03349e191d2 server_cookie=c188eef903e1d5fc in_seq=0 out_seq=0 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f21ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f21ffb640 1 -- 172.21.15.113:0/375220856 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a55ac0dc10 con 0x55a55ad1f340 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f21ffb640 10 client.15975 handle_command_reply: tid=0 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 client.15975 shutdown 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 msgr2=0x55a55acb1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 0x55a55acb1760 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c098a00 tx=0x7f5f2c004570 comp rx=0 tx=0).stop 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 shutdown_connections 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55aca8e10 0x55a55aca3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a55aca61d0 0x55a55aca3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55a55ad1f340 0x55a55ad21750 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f21ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 --2- 172.21.15.113:0/375220856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a55ac9f8e0 0x55a55acb1760 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.123+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 >> 172.21.15.113:0/375220856 conn(0x55a55ac99fb0 msgr2=0x55a55ac9afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 shutdown_connections 2024-04-22T07:16:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f435f4740 1 -- 172.21.15.113:0/375220856 wait complete. 2024-04-22T07:16:00.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f435f4740 20 client.15975 trim_cache size 0 max 16384 2024-04-22T07:16:00.129 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5f240a3f50 msgr2=0x7f5f240a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5f240a3f50 0x7f5f240a6410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005e00 tx=0x7f5f30005c50 comp rx=0 tx=0).stop 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 msgr2=0x7f5f3c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 0x7f5f3c1995a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c890 tx=0x7f5f2c00cd50 comp rx=0 tx=0).stop 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 shutdown_connections 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5f240a3f50 0x7f5f240a6410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c19a0c0 0x7f5f3c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c106160 0x7f5f3c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 --2- 172.21.15.113:0/3476165012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3c101090 0x7f5f3c1995a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 >> 172.21.15.113:0/3476165012 conn(0x7f5f3c077f50 msgr2=0x7f5f3c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 shutdown_connections 2024-04-22T07:16:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:00.124+0000 7f5f40a06640 1 -- 172.21.15.113:0/3476165012 wait complete. 2024-04-22T07:16:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:00 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1984402257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:16:00.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:00 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1984402257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:16:00.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:00 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1984402257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:16:01.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:16:01.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:16:01.135 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:16:01.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 -- 172.21.15.113:0/3158652144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d4106160 msgr2=0x7f94d410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/3158652144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d4106160 0x7f94d410dd30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009ca0 tx=0x7f94cc02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 -- 172.21.15.113:0/3158652144 shutdown_connections 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/3158652144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d4106160 0x7f94d410dd30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/3158652144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d4101a40 0x7f94d4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/3158652144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94d4101090 0x7f94d4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 -- 172.21.15.113:0/3158652144 >> 172.21.15.113:0/3158652144 conn(0x7f94d4077f50 msgr2=0x7f94d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 -- 172.21.15.113:0/3158652144 shutdown_connections 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.249+0000 7f94d8d9d640 1 -- 172.21.15.113:0/3158652144 wait complete. 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 Processor -- start 2024-04-22T07:16:01.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 -- start start 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94d4101090 0x7f94d41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d4106160 0x7f94d4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 0x7f94d419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4111260 con 0x7f94d419bf60 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41110e0 con 0x7f94d4101090 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d8d9d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41113e0 con 0x7f94d4106160 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 0x7f94d419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 0x7f94d419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52844/0 (socket says 172.21.15.113:52844) 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d2d9d640 1 -- 172.21.15.113:0/1460106851 learned_addr learned my addr 172.21.15.113:0/1460106851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d259c640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94d4101090 0x7f94d41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.250+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d4106160 0x7f94d4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d2d9d640 1 -- 172.21.15.113:0/1460106851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d4106160 msgr2=0x7f94d4197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d4106160 0x7f94d4197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d2d9d640 1 -- 172.21.15.113:0/1460106851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94d4101090 msgr2=0x7f94d41976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94d4101090 0x7f94d41976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d2d9d640 1 -- 172.21.15.113:0/1460106851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d419ca50 con 0x7f94d419bf60 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 0x7f94d419c410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f94cc02c700 tx=0x7f94cc035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94c37fe640 1 -- 172.21.15.113:0/1460106851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc039040 con 0x7f94d419bf60 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d419cce0 con 0x7f94d419bf60 2024-04-22T07:16:01.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94c37fe640 1 -- 172.21.15.113:0/1460106851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94cc044070 con 0x7f94d419bf60 2024-04-22T07:16:01.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94c37fe640 1 -- 172.21.15.113:0/1460106851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004540 con 0x7f94d419bf60 2024-04-22T07:16:01.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.251+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a4500 con 0x7f94d419bf60 2024-04-22T07:16:01.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.254+0000 7f94c37fe640 1 -- 172.21.15.113:0/1460106851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7f94cc0046e0 con 0x7f94d419bf60 2024-04-22T07:16:01.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.254+0000 7f94db98f740 1 Processor -- start 2024-04-22T07:16:01.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.255+0000 7f94db98f740 1 -- start start 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.255+0000 7f94db98f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e225cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.255+0000 7f94c37fe640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94bc0a3ed0 0x7f94bc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.255+0000 7f94d259c640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94bc0a3ed0 0x7f94bc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.255+0000 7f94d1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e225cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.255+0000 7f94d1d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e225cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52860/0 (socket says 172.21.15.113:52860) 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 -- 172.21.15.113:0/3789338039 learned_addr learned my addr 172.21.15.113:0/3789338039 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94db98f740 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94db98f740 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e231260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94c37fe640 1 -- 172.21.15.113:0/1460106851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f94cc0f32c0 con 0x7f94d419bf60 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56527e189f00 con 0x56527e2258e0 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56527e127590 con 0x56527e226290 2024-04-22T07:16:01.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56527e185ec0 con 0x56527e22ee10 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e231260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 -- 172.21.15.113:0/3789338039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 msgr2=0x56527e231260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e231260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 -- 172.21.15.113:0/3789338039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 msgr2=0x56527e22e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 -- 172.21.15.113:0/3789338039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527df647e0 con 0x56527e2258e0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d259c640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94bc0a3ed0 0x7f94bc0a6390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94c400abe0 tx=0x7f94c400a2d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e225cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f94c800bd30 tx=0x7f94c800b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ab735833960b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.256+0000 7f94c0ff9640 1 -- 172.21.15.113:0/3789338039 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c801c070 con 0x56527e2258e0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.257+0000 7f94c0ff9640 1 -- 172.21.15.113:0/3789338039 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94c8017400 con 0x56527e2258e0 2024-04-22T07:16:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.257+0000 7f94c0ff9640 1 -- 172.21.15.113:0/3789338039 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c80145e0 con 0x56527e2258e0 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.257+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 msgr2=0x56527e225cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.257+0000 7f94db98f740 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e225cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f94c800bd30 tx=0x7f94c800b5d0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 shutdown_connections 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e231260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 --2- 172.21.15.113:0/3789338039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e225cc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 >> 172.21.15.113:0/3789338039 conn(0x56527e21ffb0 msgr2=0x56527e220390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 shutdown_connections 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 -- 172.21.15.113:0/3789338039 wait complete. 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 -- start start 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.258+0000 7f94db98f740 1 Processor -- start 2024-04-22T07:16:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e22bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e22c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56527e231d80 con 0x56527e2258e0 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56527e143580 con 0x56527e226290 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56527e231f00 con 0x56527e22ee10 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 10 client.0 _ll_register_callbacks cb 0x56527e149650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e22bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e22bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52872/0 (socket says 172.21.15.113:52872) 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 -- 172.21.15.113:0/3382830341 learned_addr learned my addr 172.21.15.113:0/3382830341 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e22c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 -- 172.21.15.113:0/3382830341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 msgr2=0x56527e22c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e22c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 -- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 msgr2=0x56527e22c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 -- 172.21.15.113:0/3382830341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527dfc4f70 con 0x56527e2258e0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94d259c640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e22bdf0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f94c4005e10 tx=0x7f94c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987f27aa26101ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3382830341 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4016070 con 0x56527e2258e0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3382830341 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f94c400fe40 con 0x56527e2258e0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.259+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c8017400 con 0x56527e2258e0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3382830341 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f94c4027480 con 0x56527e2258e0 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 10 client.15987 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:01.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56527de5c170 con 0x56527e2258e0 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3382830341 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94c8017400 con 0x56527e2258e0 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3382830341 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f94c401c540 con 0x56527e2258e0 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 10 client.15987 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 20 client.15987 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e23f3b0 0x56527e241830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 4 client.15987 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56527e197920 con 0x56527e23f3b0 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.260+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e23f3b0 0x56527e241830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.261+0000 7f94b9ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.261+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e23f3b0 0x56527e241830 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f188997ab8c487 server_cookie=81ad9d5153b4089 in_seq=0 out_seq=0 2024-04-22T07:16:01.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3382830341 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56527e197920 con 0x56527e23f3b0 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94b9ffb640 10 client.15987 handle_command_reply: tid=0 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 client.15987 shutdown 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 msgr2=0x56527e22bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e22bdf0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f94c4005e10 tx=0x7f94c4004730 comp rx=0 tx=0).stop 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 shutdown_connections 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e22ee10 0x56527e22c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e22c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e23f3b0 0x56527e241830 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 --2- 172.21.15.113:0/3382830341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e2258e0 0x56527e22bdf0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 >> 172.21.15.113:0/3382830341 conn(0x56527e21ffb0 msgr2=0x56527e221d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94b9ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 shutdown_connections 2024-04-22T07:16:01.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 1 -- 172.21.15.113:0/3382830341 wait complete. 2024-04-22T07:16:01.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.262+0000 7f94db98f740 20 client.15987 trim_cache size 0 max 16384 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 Processor -- start 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 -- start start 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 0x56527e22e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e23ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e247100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ae270 con 0x56527e22ee10 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41abb40 con 0x56527e226290 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94db98f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41a9990 con 0x56527e2258e0 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e247100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e23ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 0x56527e22e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e247100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52876/0 (socket says 172.21.15.113:52876) 2024-04-22T07:16:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e23ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:34786/0 (socket says 172.21.15.113:34786) 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d2d9d640 1 -- 172.21.15.113:0/448410031 learned_addr learned my addr 172.21.15.113:0/448410031 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 0x56527e22e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:38572/0 (socket says 172.21.15.113:38572) 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 -- 172.21.15.113:0/448410031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 msgr2=0x56527e23ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e23ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 -- 172.21.15.113:0/448410031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 msgr2=0x56527e247100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e247100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.295+0000 7f94d259c640 1 -- 172.21.15.113:0/448410031 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527e23df50 con 0x56527e2258e0 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.296+0000 7f94d259c640 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 0x56527e22e520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f94c400f850 tx=0x7f94c4004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36df46441d7fd73d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.296+0000 7f94bbfff640 1 -- 172.21.15.113:0/448410031 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400fe20 con 0x56527e2258e0 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.296+0000 7f94bbfff640 1 -- 172.21.15.113:0/448410031 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94c403a030 con 0x56527e2258e0 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.296+0000 7f94bbfff640 1 -- 172.21.15.113:0/448410031 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c401c740 con 0x56527e2258e0 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 -- 172.21.15.113:0/448410031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 msgr2=0x56527e22e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 0x56527e22e520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f94c400f850 tx=0x7f94c4004660 comp rx=0 tx=0).stop 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 -- 172.21.15.113:0/448410031 shutdown_connections 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e247100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e226290 0x56527e23ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 --2- 172.21.15.113:0/448410031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e2258e0 0x56527e22e520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 -- 172.21.15.113:0/448410031 >> 172.21.15.113:0/448410031 conn(0x56527e21ffb0 msgr2=0x56527e22a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 -- 172.21.15.113:0/448410031 shutdown_connections 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 -- 172.21.15.113:0/448410031 wait complete. 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 -- start start 2024-04-22T07:16:01.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.297+0000 7f94db98f740 1 Processor -- start 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e2258e0 0x56527e237210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e226290 0x56527e232720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e232c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41aa000 con 0x56527e22ee10 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41ab470 con 0x56527e2258e0 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e2258e0 0x56527e237210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e2258e0 0x56527e237210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:34800/0 (socket says 172.21.15.113:34800) 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d259c640 1 -- 172.21.15.113:0/3516679059 learned_addr learned my addr 172.21.15.113:0/3516679059 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41aa6d0 con 0x56527e226290 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e232c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d1d9b640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e226290 0x56527e232720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94db98f740 10 client.0 _ll_register_callbacks cb 0x7f94c400a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 -- 172.21.15.113:0/3516679059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e226290 msgr2=0x56527e232720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e226290 0x56527e232720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 -- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e2258e0 msgr2=0x56527e237210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e2258e0 0x56527e237210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 -- 172.21.15.113:0/3516679059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527e2331a0 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94d2d9d640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e232c60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0bbac0 tx=0x7f94cc0beca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9f5c4c9c4c754c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.298+0000 7f94b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3516679059 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0bf0f0 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3516679059 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f94cc0c3150 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c403a030 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3516679059 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f94cc0c34a0 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 10 client.15993 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56527e2a56e0 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3516679059 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94c403a030 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3516679059 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f94cc03c030 con 0x56527e22ee10 2024-04-22T07:16:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 10 client.15993 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 20 client.15993 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.299+0000 7f94db98f740 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e2a5eb0 0x56527e2a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.300+0000 7f94db98f740 4 client.15993 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.300+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56527e191790 con 0x56527e2a5eb0 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.300+0000 7f94d259c640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e2a5eb0 0x56527e2a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.300+0000 7f94d259c640 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e2a5eb0 0x56527e2a82d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde76f82d2f914da server_cookie=f6dc5ca79f204e45 in_seq=0 out_seq=0 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.300+0000 7f94b9ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.301+0000 7f94b9ffb640 1 -- 172.21.15.113:0/3516679059 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56527e191790 con 0x56527e2a5eb0 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.301+0000 7f94b9ffb640 10 client.15993 handle_command_reply: tid=0 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.301+0000 7f94db98f740 1 client.15993 shutdown 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 msgr2=0x56527e232c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e232c60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0bbac0 tx=0x7f94cc0beca0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 shutdown_connections 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x56527e2a5eb0 0x56527e2a82d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56527e22ee10 0x56527e232c60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94b9ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56527e226290 0x56527e232720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 --2- 172.21.15.113:0/3516679059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56527e2258e0 0x56527e237210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 >> 172.21.15.113:0/3516679059 conn(0x56527e21ffb0 msgr2=0x56527e21e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 shutdown_connections 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 1 -- 172.21.15.113:0/3516679059 wait complete. 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.302+0000 7f94db98f740 20 client.15993 trim_cache size 0 max 16384 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "a661c639-e07a-4780-b8a5-79bf2e7d582c", 2024-04-22T07:16:01.304 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:16:01.305 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:16:01.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94bc0a3ed0 msgr2=0x7f94bc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94bc0a3ed0 0x7f94bc0a6390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94c400abe0 tx=0x7f94c400a2d0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 msgr2=0x7f94d419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 0x7f94d419c410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f94cc02c700 tx=0x7f94cc035d80 comp rx=0 tx=0).stop 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 shutdown_connections 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94bc0a3ed0 0x7f94bc0a6390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d419bf60 0x7f94d419c410 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94d4106160 0x7f94d4197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 --2- 172.21.15.113:0/1460106851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94d4101090 0x7f94d41976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 >> 172.21.15.113:0/1460106851 conn(0x7f94d4077f50 msgr2=0x7f94d40fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 shutdown_connections 2024-04-22T07:16:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.303+0000 7f94d8d9d640 1 -- 172.21.15.113:0/1460106851 wait complete. 2024-04-22T07:16:01.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:16:01.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 -- 172.21.15.113:0/1936095001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101a30 msgr2=0x7fc9f8105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 --2- 172.21.15.113:0/1936095001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101a30 0x7fc9f8105a70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e402bc30 comp rx=0 tx=0).stop 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 -- 172.21.15.113:0/1936095001 shutdown_connections 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 --2- 172.21.15.113:0/1936095001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8106150 0x7fc9f810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 --2- 172.21.15.113:0/1936095001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101a30 0x7fc9f8105a70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 --2- 172.21.15.113:0/1936095001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f8101080 0x7fc9f8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 -- 172.21.15.113:0/1936095001 >> 172.21.15.113:0/1936095001 conn(0x7fc9f8078000 msgr2=0x7fc9f8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 -- 172.21.15.113:0/1936095001 shutdown_connections 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.421+0000 7fc9feb52640 1 -- 172.21.15.113:0/1936095001 wait complete. 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 Processor -- start 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 -- start start 2024-04-22T07:16:01.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 0x7fc9f819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8106150 0x7fc9f819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f8198600 0x7fc9f8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81111d0 con 0x7fc9f8101080 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8111050 con 0x7fc9f8106150 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9feb52640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8111350 con 0x7fc9f8198600 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 0x7fc9f819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 0x7fc9f819e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52908/0 (socket says 172.21.15.113:52908) 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8106150 0x7fc9f819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/542822817 learned_addr learned my addr 172.21.15.113:0/542822817 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f8198600 0x7fc9f8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/542822817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f8198600 msgr2=0x7fc9f8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f8198600 0x7fc9f8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/542822817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8106150 msgr2=0x7fc9f819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8106150 0x7fc9f819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.422+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/542822817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f8199120 con 0x7fc9f8101080 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.423+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 0x7fc9f819e450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00e9f0 tx=0x7fc9ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.423+0000 7fc9f5ffb640 1 -- 172.21.15.113:0/542822817 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec00cd80 con 0x7fc9f8101080 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.423+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f8199410 con 0x7fc9f8101080 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.423+0000 7fc9f5ffb640 1 -- 172.21.15.113:0/542822817 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc9ec0054f0 con 0x7fc9f8101080 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.423+0000 7fc9f5ffb640 1 -- 172.21.15.113:0/542822817 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec010690 con 0x7fc9f8101080 2024-04-22T07:16:01.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.423+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f81a5660 con 0x7fc9f8101080 2024-04-22T07:16:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.426+0000 7fc9f5ffb640 1 -- 172.21.15.113:0/542822817 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 25) ==== 138931+0+0 (secure 0 0 0) 0x7fc9ec0046c0 con 0x7fc9f8101080 2024-04-22T07:16:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.426+0000 7fca01827740 1 Processor -- start 2024-04-22T07:16:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.426+0000 7fca01827740 1 -- start start 2024-04-22T07:16:01.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fca01827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fca01827740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fca01827740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fca01827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5619ba92e1e0 con 0x5619ba9c9580 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fca01827740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5619ba8cb390 con 0x5619ba9d2a80 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fca01827740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5619ba92a1a0 con 0x5619ba9c9ea0 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9f5ffb640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc9e80a3f50 0x7fc9e80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52916/0 (socket says 172.21.15.113:52916) 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/3730541560 learned_addr learned my addr 172.21.15.113:0/3730541560 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc9e80a3f50 0x7fc9e80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/3730541560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 msgr2=0x5619ba9d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/3730541560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 msgr2=0x5619ba9d4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9f5ffb640 1 -- 172.21.15.113:0/542822817 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fc9ec0d0e60 con 0x7fc9f8101080 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/3730541560 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ba70a7d0 con 0x5619ba9c9580 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.427+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.428+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc9e80a3f50 0x7fc9e80a6410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000fc90 tx=0x7fc9f0022000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.428+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9c9960 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0011ad0 tx=0x7fc9f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dcce3b635d706c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.428+0000 7fc9e2ffd640 1 -- 172.21.15.113:0/3730541560 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011e60 con 0x5619ba9c9580 2024-04-22T07:16:01.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.428+0000 7fc9e2ffd640 1 -- 172.21.15.113:0/3730541560 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc9f0023070 con 0x5619ba9c9580 2024-04-22T07:16:01.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fc9e2ffd640 1 -- 172.21.15.113:0/3730541560 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0004510 con 0x5619ba9c9580 2024-04-22T07:16:01.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 -- 172.21.15.113:0/3730541560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 msgr2=0x5619ba9c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9c9960 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0011ad0 tx=0x7fc9f000f790 comp rx=0 tx=0).stop 2024-04-22T07:16:01.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 -- 172.21.15.113:0/3730541560 shutdown_connections 2024-04-22T07:16:01.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 --2- 172.21.15.113:0/3730541560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9c9960 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 -- 172.21.15.113:0/3730541560 >> 172.21.15.113:0/3730541560 conn(0x5619ba9c3cb0 msgr2=0x5619ba9c4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 -- 172.21.15.113:0/3730541560 shutdown_connections 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 -- 172.21.15.113:0/3730541560 wait complete. 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.429+0000 7fca01827740 1 -- start start 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 Processor -- start 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9580 0x5619ba9cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5619ba9d5850 con 0x5619ba9c9ea0 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5619ba8e72f0 con 0x5619ba9d2a80 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5619ba9d59d0 con 0x5619ba9c9580 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fca01827740 10 client.0 _ll_register_callbacks cb 0x5619ba8f4580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:01.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fc9f7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fc9f7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52926/0 (socket says 172.21.15.113:52926) 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fc9f7fff640 1 -- 172.21.15.113:0/250234732 learned_addr learned my addr 172.21.15.113:0/250234732 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.430+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9580 0x5619ba9cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9f7fff640 1 -- 172.21.15.113:0/250234732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9580 msgr2=0x5619ba9cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9580 0x5619ba9cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9f7fff640 1 -- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 msgr2=0x5619ba9d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9f7fff640 1 -- 172.21.15.113:0/250234732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ba6403f0 con 0x5619ba9c9ea0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cff40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009230 tx=0x7fc9e402cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a0ff28c31e324a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9d7fff640 1 -- 172.21.15.113:0/250234732 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4004450 con 0x5619ba9c9ea0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fc9d7fff640 1 -- 172.21.15.113:0/250234732 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fc9e4005030 con 0x5619ba9c9ea0 2024-04-22T07:16:01.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.431+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f0023070 con 0x5619ba9c9ea0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fc9d7fff640 1 -- 172.21.15.113:0/250234732 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9e4005230 con 0x5619ba9c9ea0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 10 client.16005 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5619ba6000f0 con 0x5619ba9c9ea0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fc9d7fff640 1 -- 172.21.15.113:0/250234732 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc9f0023070 con 0x5619ba9c9ea0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fc9d7fff640 1 -- 172.21.15.113:0/250234732 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fc9e404b070 con 0x5619ba9c9ea0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 10 client.16005 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 20 client.16005 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619ba9e2cf0 0x5619ba9e5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 4 client.16005 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ba93bc00 con 0x5619ba9e2cf0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.432+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619ba9e2cf0 0x5619ba9e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.433+0000 7fc9d7fff640 10 client.16005 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.433+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619ba9e2cf0 0x5619ba9e5170 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80a78910ebcdd7a server_cookie=6e5434657b28cf41 in_seq=0 out_seq=0 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.433+0000 7fc9d7fff640 1 -- 172.21.15.113:0/250234732 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619ba93bc00 con 0x5619ba9e2cf0 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.433+0000 7fc9d7fff640 10 client.16005 handle_command_reply: tid=0 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.433+0000 7fca01827740 1 client.16005 shutdown 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 msgr2=0x5619ba9cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cff40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009230 tx=0x7fc9e402cde0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 shutdown_connections 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9d2a80 0x5619ba9d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619ba9e2cf0 0x5619ba9e5170 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cff40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 --2- 172.21.15.113:0/250234732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9580 0x5619ba9cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 >> 172.21.15.113:0/250234732 conn(0x5619ba9c3cb0 msgr2=0x5619ba9c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fc9d7fff640 0 client.16005 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 shutdown_connections 2024-04-22T07:16:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 1 -- 172.21.15.113:0/250234732 wait complete. 2024-04-22T07:16:01.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.434+0000 7fca01827740 20 client.16005 trim_cache size 0 max 16384 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 Processor -- start 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 -- start start 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9580 0x5619ba9d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 0x5619ba9de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81ae420 con 0x5619ba9d2a80 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81abcf0 con 0x5619ba9c9580 2024-04-22T07:16:01.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fca01827740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81a9b60 con 0x5619ba9c9ea0 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.451+0000 7fc9fd0ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 0x5619ba9de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 0x5619ba9de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52930/0 (socket says 172.21.15.113:52930) 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/2164686152 learned_addr learned my addr 172.21.15.113:0/2164686152 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9580 0x5619ba9d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/2164686152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 msgr2=0x5619ba9ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/2164686152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9580 msgr2=0x5619ba9d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9580 0x5619ba9d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 -- 172.21.15.113:0/2164686152 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ba9c3ae0 con 0x5619ba9d2a80 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 0x5619ba9de760 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0028040 tx=0x7fc9f00040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60a02229e000392 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9e27fc640 1 -- 172.21.15.113:0/2164686152 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0033070 con 0x5619ba9d2a80 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9e27fc640 1 -- 172.21.15.113:0/2164686152 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc9f0023070 con 0x5619ba9d2a80 2024-04-22T07:16:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.452+0000 7fc9e27fc640 1 -- 172.21.15.113:0/2164686152 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0005760 con 0x5619ba9d2a80 2024-04-22T07:16:01.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 -- 172.21.15.113:0/2164686152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 msgr2=0x5619ba9de760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 0x5619ba9de760 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0028040 tx=0x7fc9f00040f0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 -- 172.21.15.113:0/2164686152 shutdown_connections 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9d2a80 0x5619ba9de760 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9c9ea0 0x5619ba9ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 --2- 172.21.15.113:0/2164686152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9580 0x5619ba9d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 -- 172.21.15.113:0/2164686152 >> 172.21.15.113:0/2164686152 conn(0x5619ba9c3cb0 msgr2=0x5619ba9c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 -- 172.21.15.113:0/2164686152 shutdown_connections 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 -- 172.21.15.113:0/2164686152 wait complete. 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.453+0000 7fca01827740 1 -- start start 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 Processor -- start 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9d2a80 0x5619ba9ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81aa1b0 con 0x5619ba9c9580 2024-04-22T07:16:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81ab620 con 0x5619ba9c9ea0 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f81aa880 con 0x5619ba9d2a80 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fca01827740 10 client.0 _ll_register_callbacks cb 0x7fc9ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52938/0 (socket says 172.21.15.113:52938) 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/3816150273 learned_addr learned my addr 172.21.15.113:0/3816150273 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/3816150273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9d2a80 msgr2=0x5619ba9ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fd0ef640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9d2a80 0x5619ba9ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9d2a80 0x5619ba9ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9ea0 msgr2=0x5619ba9cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.454+0000 7fc9fc8ee640 1 -- 172.21.15.113:0/3816150273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ba9d5460 con 0x5619ba9c9580 2024-04-22T07:16:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9fc8ee640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9db180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00c6a0 tx=0x7fc9ec06b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ca31210596bd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9d7fff640 1 -- 172.21.15.113:0/3816150273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec09cb40 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9d7fff640 1 -- 172.21.15.113:0/3816150273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fc9ec0991b0 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f0023070 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9d7fff640 1 -- 172.21.15.113:0/3816150273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9ec0aa050 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 10 client.16011 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5619baa49270 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9d7fff640 1 -- 172.21.15.113:0/3816150273 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc9f0023070 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fc9d7fff640 1 -- 172.21.15.113:0/3816150273 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fc9ec0a5320 con 0x5619ba9c9580 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 10 client.16011 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 20 client.16011 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:01.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619baa49950 0x5619baa4bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 4 client.16011 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.455+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619ba935a70 con 0x5619baa49950 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.456+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619baa49950 0x5619baa4bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.456+0000 7fc9d7fff640 10 client.16011 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.456+0000 7fc9f7fff640 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619baa49950 0x5619baa4bd70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f4c90af1a57493 server_cookie=624bc4d2afa0496b in_seq=0 out_seq=0 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.456+0000 7fc9d7fff640 1 -- 172.21.15.113:0/3816150273 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5619ba935a70 con 0x5619baa49950 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.456+0000 7fc9d7fff640 10 client.16011 handle_command_reply: tid=0 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.456+0000 7fca01827740 1 client.16011 shutdown 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 msgr2=0x5619ba9db180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9db180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00c6a0 tx=0x7fc9ec06b040 comp rx=0 tx=0).stop 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 shutdown_connections 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5619ba9d2a80 0x5619ba9ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619ba9c9ea0 0x5619ba9cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5619baa49950 0x5619baa4bd70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 --2- 172.21.15.113:0/3816150273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5619ba9c9580 0x5619ba9db180 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fc9d7fff640 0 client.16011 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 >> 172.21.15.113:0/3816150273 conn(0x5619ba9c3cb0 msgr2=0x5619ba9cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 shutdown_connections 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 1 -- 172.21.15.113:0/3816150273 wait complete. 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.457+0000 7fca01827740 20 client.16011 trim_cache size 0 max 16384 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:16:01.461 INFO:teuthology.orchestra.run.smithi113.stdout: "a661c639-e07a-4780-b8a5-79bf2e7d582c": { 2024-04-22T07:16:01.462 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:16:01.462 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "a661c639-e07a-4780-b8a5-79bf2e7d582c", 2024-04-22T07:16:01.462 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:16:01.462 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:16:01.462 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:16:01.462 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc9e80a3f50 msgr2=0x7fc9e80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc9e80a3f50 0x7fc9e80a6410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000fc90 tx=0x7fc9f0022000 comp rx=0 tx=0).stop 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 msgr2=0x7fc9f819e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 0x7fc9f819e450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec00e9f0 tx=0x7fc9ec00edf0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 shutdown_connections 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc9e80a3f50 0x7fc9e80a6410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f8198600 0x7fc9f8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9f8106150 0x7fc9f819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 --2- 172.21.15.113:0/542822817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9f8101080 0x7fc9f819e450 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:01.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 >> 172.21.15.113:0/542822817 conn(0x7fc9f8078000 msgr2=0x7fc9f80693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:01.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 shutdown_connections 2024-04-22T07:16:01.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:01.458+0000 7fc9feb52640 1 -- 172.21.15.113:0/542822817 wait complete. 2024-04-22T07:16:01.465 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:01 smithi113 ceph-mon[27933]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 618 B/s wr, 21 op/s 2024-04-22T07:16:01.466 INFO:tasks.cephfs.filesystem:scrub status for tag:a661c639-e07a-4780-b8a5-79bf2e7d582c - {'path': '/', 'tag': 'a661c639-e07a-4780-b8a5-79bf2e7d582c', 'options': 'recursive,force'} 2024-04-22T07:16:01.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:01 smithi133 ceph-mon[31198]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 618 B/s wr, 21 op/s 2024-04-22T07:16:01.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:01 smithi154 ceph-mon[31862]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 618 B/s wr, 21 op/s 2024-04-22T07:16:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:02 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:16:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:02 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:02 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:16:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:02 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:02 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:16:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:02 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:02 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:16:02.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:02 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:02 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:16:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:02 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:02 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:16:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:02 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:03 smithi133 ceph-mon[31198]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 37 op/s 2024-04-22T07:16:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:03 smithi154 ceph-mon[31862]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 37 op/s 2024-04-22T07:16:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:03 smithi113 ceph-mon[27933]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 37 op/s 2024-04-22T07:16:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:05 smithi133 ceph-mon[31198]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 32 op/s 2024-04-22T07:16:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:05 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:16:05.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:05 smithi154 ceph-mon[31862]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 32 op/s 2024-04-22T07:16:05.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:05 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:16:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:05 smithi113 ceph-mon[27933]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 32 op/s 2024-04-22T07:16:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:05 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:16:07.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:07 smithi133 ceph-mon[31198]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 35 op/s 2024-04-22T07:16:07.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:07 smithi154 ceph-mon[31862]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 35 op/s 2024-04-22T07:16:07.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:07 smithi113 ceph-mon[27933]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 35 op/s 2024-04-22T07:16:09.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:09 smithi133 ceph-mon[31198]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:16:09.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:09 smithi154 ceph-mon[31862]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:16:09.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:09 smithi113 ceph-mon[27933]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:16:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:11 smithi133 ceph-mon[31198]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:16:11.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:11 smithi154 ceph-mon[31862]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:16:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:11 smithi113 ceph-mon[27933]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:16:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:13 smithi133 ceph-mon[31198]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-22T07:16:13.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:13 smithi154 ceph-mon[31862]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-22T07:16:13.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:13 smithi113 ceph-mon[27933]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-22T07:16:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:15 smithi133 ceph-mon[31198]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:16:15.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:15 smithi154 ceph-mon[31862]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:16:15.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:15 smithi113 ceph-mon[27933]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T07:16:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:17 smithi133 ceph-mon[31198]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T07:16:17.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:17 smithi154 ceph-mon[31862]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T07:16:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:17 smithi113 ceph-mon[27933]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T07:16:19.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:19 smithi133 ceph-mon[31198]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:16:19.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:19 smithi154 ceph-mon[31862]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:16:19.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:19 smithi113 ceph-mon[27933]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:16:21.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:21 smithi133 ceph-mon[31198]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:16:21.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:21 smithi154 ceph-mon[31862]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:16:21.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:21 smithi113 ceph-mon[27933]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T07:16:23.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:23 smithi133 ceph-mon[31198]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-22T07:16:23.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:23 smithi133 ceph-mon[31198]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T07:16:23.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:23 smithi154 ceph-mon[31862]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-22T07:16:23.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:23 smithi154 ceph-mon[31862]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T07:16:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:23 smithi113 ceph-mon[27933]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-22T07:16:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:23 smithi113 ceph-mon[27933]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T07:16:25.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:25 smithi133 ceph-mon[31198]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:25.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:25 smithi154 ceph-mon[31862]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:25.586 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:25 smithi113 ceph-mon[27933]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr:Note: switching to '86f7587a5a09af35f0895e1a2d08527638fae697'. 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.309 INFO:tasks.workunit.client.0.smithi113.stderr: git switch -c 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr:Or undo this operation with: 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr: git switch - 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-04-22T07:16:27.310 INFO:tasks.workunit.client.0.smithi113.stderr:HEAD is now at 86f7587a5a0 qa/fsx: use the master branch to build the xfstest-dev 2024-04-22T07:16:27.315 DEBUG:teuthology.orchestra.run.smithi113:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T07:16:27.347 INFO:tasks.workunit.client.0.smithi113.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T07:16:27.350 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T07:16:27.350 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T07:16:27.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:27 smithi133 ceph-mon[31198]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-22T07:16:27.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:27 smithi154 ceph-mon[31862]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-22T07:16:27.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:27 smithi113 ceph-mon[27933]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-22T07:16:27.984 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T07:16:28.036 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T07:16:28.075 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T07:16:28.076 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T07:16:28.076 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T07:16:28.115 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T07:16:28.118 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:16:28.118 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T07:16:28.174 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-04-22T07:16:28.175 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-04-22T07:16:28.175 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-04-22T07:16:28.236 INFO:tasks.workunit.client.0.smithi113.stderr:+ KERNEL=linux-4.0.5 2024-04-22T07:16:28.237 INFO:tasks.workunit.client.0.smithi113.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-04-22T07:16:29.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:29 smithi133 ceph-mon[31198]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:29.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:29 smithi154 ceph-mon[31862]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:29.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:29 smithi113 ceph-mon[27933]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:31.317 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir untar_tar 2024-04-22T07:16:31.319 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd untar_tar 2024-04-22T07:16:31.319 INFO:tasks.workunit.client.0.smithi113.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-04-22T07:16:31.346 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ 2024-04-22T07:16:31.347 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/00-INDEX 2024-04-22T07:16:31.348 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/ 2024-04-22T07:16:31.349 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/README 2024-04-22T07:16:31.350 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-04-22T07:16:31.351 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-04-22T07:16:31.352 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-04-22T07:16:31.353 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-04-22T07:16:31.355 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-04-22T07:16:31.356 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-04-22T07:16:31.357 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-04-22T07:16:31.359 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-04-22T07:16:31.360 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-04-22T07:16:31.361 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-04-22T07:16:31.362 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-04-22T07:16:31.363 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-04-22T07:16:31.364 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-04-22T07:16:31.365 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-04-22T07:16:31.366 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-04-22T07:16:31.367 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-04-22T07:16:31.368 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-04-22T07:16:31.369 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-04-22T07:16:31.370 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-04-22T07:16:31.371 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-04-22T07:16:31.372 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-04-22T07:16:31.373 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-04-22T07:16:31.374 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-04-22T07:16:31.375 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-04-22T07:16:31.376 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-04-22T07:16:31.376 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-04-22T07:16:31.377 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-04-22T07:16:31.379 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-04-22T07:16:31.380 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-04-22T07:16:31.381 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-04-22T07:16:31.382 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-04-22T07:16:31.383 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-04-22T07:16:31.384 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-04-22T07:16:31.385 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-04-22T07:16:31.386 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-04-22T07:16:31.386 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-04-22T07:16:31.387 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-04-22T07:16:31.389 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-04-22T07:16:31.390 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-04-22T07:16:31.391 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-04-22T07:16:31.392 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-04-22T07:16:31.393 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-04-22T07:16:31.394 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-04-22T07:16:31.395 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-04-22T07:16:31.396 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-04-22T07:16:31.397 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-04-22T07:16:31.398 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-04-22T07:16:31.399 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-04-22T07:16:31.400 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-04-22T07:16:31.401 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-04-22T07:16:31.402 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-04-22T07:16:31.403 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-04-22T07:16:31.403 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-04-22T07:16:31.404 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-04-22T07:16:31.406 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-04-22T07:16:31.406 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-04-22T07:16:31.407 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-04-22T07:16:31.408 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-04-22T07:16:31.409 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-04-22T07:16:31.410 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-04-22T07:16:31.411 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-04-22T07:16:31.412 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-04-22T07:16:31.413 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-04-22T07:16:31.414 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-04-22T07:16:31.415 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-04-22T07:16:31.416 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-04-22T07:16:31.417 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-04-22T07:16:31.418 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-04-22T07:16:31.419 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-04-22T07:16:31.420 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-04-22T07:16:31.420 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-04-22T07:16:31.421 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-04-22T07:16:31.422 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-04-22T07:16:31.423 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-04-22T07:16:31.424 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-04-22T07:16:31.425 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-04-22T07:16:31.426 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-04-22T07:16:31.427 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-04-22T07:16:31.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-04-22T07:16:31.429 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-04-22T07:16:31.430 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-04-22T07:16:31.431 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-04-22T07:16:31.432 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-04-22T07:16:31.433 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-04-22T07:16:31.434 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-04-22T07:16:31.435 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-04-22T07:16:31.436 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-04-22T07:16:31.437 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-04-22T07:16:31.438 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-04-22T07:16:31.439 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-04-22T07:16:31.440 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-04-22T07:16:31.441 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-04-22T07:16:31.442 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-04-22T07:16:31.443 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-04-22T07:16:31.444 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-04-22T07:16:31.445 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-04-22T07:16:31.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-04-22T07:16:31.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-04-22T07:16:31.448 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-04-22T07:16:31.449 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-04-22T07:16:31.450 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-04-22T07:16:31.451 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-04-22T07:16:31.452 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-04-22T07:16:31.453 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-04-22T07:16:31.454 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-04-22T07:16:31.455 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-04-22T07:16:31.456 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-04-22T07:16:31.457 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-04-22T07:16:31.457 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-04-22T07:16:31.459 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-04-22T07:16:31.460 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-04-22T07:16:31.461 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-04-22T07:16:31.462 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-04-22T07:16:31.463 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-04-22T07:16:31.464 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-04-22T07:16:31.465 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-04-22T07:16:31.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-04-22T07:16:31.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:16:31.466 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:16:31.467 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-04-22T07:16:31.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-04-22T07:16:31.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-04-22T07:16:31.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-04-22T07:16:31.471 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-04-22T07:16:31.472 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-04-22T07:16:31.473 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-04-22T07:16:31.474 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-04-22T07:16:31.475 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-04-22T07:16:31.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-04-22T07:16:31.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-04-22T07:16:31.477 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-04-22T07:16:31.478 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-04-22T07:16:31.479 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-04-22T07:16:31.480 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-04-22T07:16:31.481 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-04-22T07:16:31.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-04-22T07:16:31.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-04-22T07:16:31.483 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-04-22T07:16:31.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-04-22T07:16:31.485 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-04-22T07:16:31.486 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-04-22T07:16:31.487 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-04-22T07:16:31.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-04-22T07:16:31.489 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-04-22T07:16:31.490 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-04-22T07:16:31.490 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-04-22T07:16:31.491 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-04-22T07:16:31.492 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-04-22T07:16:31.493 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-04-22T07:16:31.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-04-22T07:16:31.495 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-04-22T07:16:31.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-04-22T07:16:31.497 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-04-22T07:16:31.498 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-04-22T07:16:31.498 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-04-22T07:16:31.499 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-04-22T07:16:31.500 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-04-22T07:16:31.501 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-04-22T07:16:31.502 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-04-22T07:16:31.503 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-04-22T07:16:31.503 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-04-22T07:16:31.504 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-04-22T07:16:31.505 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-04-22T07:16:31.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-04-22T07:16:31.507 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-04-22T07:16:31.508 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-04-22T07:16:31.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-04-22T07:16:31.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-04-22T07:16:31.510 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-04-22T07:16:31.511 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-04-22T07:16:31.512 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-04-22T07:16:31.513 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-04-22T07:16:31.514 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-04-22T07:16:31.514 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-04-22T07:16:31.515 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-04-22T07:16:31.517 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-04-22T07:16:31.517 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-04-22T07:16:31.518 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-04-22T07:16:31.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-04-22T07:16:31.520 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-04-22T07:16:31.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-04-22T07:16:31.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-04-22T07:16:31.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-04-22T07:16:31.524 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-04-22T07:16:31.524 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-04-22T07:16:31.525 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-04-22T07:16:31.526 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-04-22T07:16:31.526 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-04-22T07:16:31.527 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-04-22T07:16:31.528 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-04-22T07:16:31.529 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-04-22T07:16:31.530 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-04-22T07:16:31.530 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-04-22T07:16:31.531 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-04-22T07:16:31.532 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-04-22T07:16:31.533 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-04-22T07:16:31.534 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-04-22T07:16:31.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:31 smithi133 ceph-mon[31198]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:31.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:31 smithi154 ceph-mon[31862]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 -- 172.21.15.113:0/460742616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 msgr2=0x7fdd6c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/460742616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c10dd50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009ca0 tx=0x7fdd6402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 -- 172.21.15.113:0/460742616 shutdown_connections 2024-04-22T07:16:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/460742616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c10dd50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/460742616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd6c101a60 0x7fdd6c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/460742616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6c1010b0 0x7fdd6c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 -- 172.21.15.113:0/460742616 >> 172.21.15.113:0/460742616 conn(0x7fdd6c0771d0 msgr2=0x7fdd6c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 -- 172.21.15.113:0/460742616 shutdown_connections 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.572+0000 7fdd72e3f640 1 -- 172.21.15.113:0/460742616 wait complete. 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 Processor -- start 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 -- start start 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6c1010b0 0x7fdd6c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd6c1035c0 0x7fdd6c1a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c111200 con 0x7fdd6c106180 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c111080 con 0x7fdd6c1010b0 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd72e3f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c111380 con 0x7fdd6c1035c0 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33030/0 (socket says 172.21.15.113:33030) 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 -- 172.21.15.113:0/500767333 learned_addr learned my addr 172.21.15.113:0/500767333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6c1010b0 0x7fdd6c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 -- 172.21.15.113:0/500767333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd6c1035c0 msgr2=0x7fdd6c1a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd6c1035c0 0x7fdd6c1a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 -- 172.21.15.113:0/500767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6c1010b0 msgr2=0x7fdd6c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6c1010b0 0x7fdd6c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 -- 172.21.15.113:0/500767333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1a7090 con 0x7fdd6c106180 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.573+0000 7fdd6bfff640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c103080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c970 tx=0x7fdd5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.574+0000 7fdd69ffb640 1 -- 172.21.15.113:0/500767333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58007bb0 con 0x7fdd6c106180 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.574+0000 7fdd69ffb640 1 -- 172.21.15.113:0/500767333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd58007d50 con 0x7fdd6c106180 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.574+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c1a7380 con 0x7fdd6c106180 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.574+0000 7fdd69ffb640 1 -- 172.21.15.113:0/500767333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800f450 con 0x7fdd6c106180 2024-04-22T07:16:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.574+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1a78d0 con 0x7fdd6c106180 2024-04-22T07:16:31.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.576+0000 7fdd69ffb640 1 -- 172.21.15.113:0/500767333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fdd580046c0 con 0x7fdd6c106180 2024-04-22T07:16:31.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.576+0000 7fdd75a2d740 1 Processor -- start 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.576+0000 7fdd69ffb640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd5c0a3d50 0x7fdd5c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.576+0000 7fdd75a2d740 1 -- start start 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd5c0a3d50 0x7fdd5c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605580 0x55d30c605960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605ea0 0x55d30c60e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd69ffb640 1 -- 172.21.15.113:0/500767333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fdd580d0590 con 0x7fdd6c106180 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c610e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d30c56a1e0 con 0x55d30c60ea80 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d30c507390 con 0x55d30c605ea0 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d30c5661a0 con 0x55d30c605580 2024-04-22T07:16:31.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c610e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c610e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33038/0 (socket says 172.21.15.113:33038) 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 -- 172.21.15.113:0/3444386659 learned_addr learned my addr 172.21.15.113:0/3444386659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605580 0x55d30c605960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd6bfff640 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605ea0 0x55d30c60e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd5c0a3d50 0x7fdd5c0a6210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009ca0 tx=0x7fdd64032d40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 -- 172.21.15.113:0/3444386659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605580 msgr2=0x55d30c605960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605580 0x55d30c605960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 -- 172.21.15.113:0/3444386659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605ea0 msgr2=0x55d30c60e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605ea0 0x55d30c60e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 -- 172.21.15.113:0/3444386659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30c3467d0 con 0x55d30c60ea80 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd6bfff640 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605ea0 0x55d30c60e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd713dc640 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c610e70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdd64005fc0 tx=0x7fdd64046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9423e73f31e34c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd56ffd640 1 -- 172.21.15.113:0/3444386659 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64003b70 con 0x55d30c60ea80 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd56ffd640 1 -- 172.21.15.113:0/3444386659 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd64003d10 con 0x55d30c60ea80 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.577+0000 7fdd56ffd640 1 -- 172.21.15.113:0/3444386659 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64034960 con 0x55d30c60ea80 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 -- 172.21.15.113:0/3444386659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 msgr2=0x55d30c610e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c610e70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdd64005fc0 tx=0x7fdd64046000 comp rx=0 tx=0).stop 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 -- 172.21.15.113:0/3444386659 shutdown_connections 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c610e70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605ea0 0x55d30c60e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/3444386659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605580 0x55d30c605960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 -- 172.21.15.113:0/3444386659 >> 172.21.15.113:0/3444386659 conn(0x55d30c5ffcb0 msgr2=0x55d30c600090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 -- 172.21.15.113:0/3444386659 shutdown_connections 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 -- 172.21.15.113:0/3444386659 wait complete. 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 -- start start 2024-04-22T07:16:31.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.578+0000 7fdd75a2d740 1 Processor -- start 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c60bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c60c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c60c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d30c6117b0 con 0x55d30c60ea80 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d30c5232f0 con 0x55d30c605580 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d30c611930 con 0x55d30c605ea0 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd75a2d740 10 client.0 _ll_register_callbacks cb 0x55d30c530580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c60c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c60c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33040/0 (socket says 172.21.15.113:33040) 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 -- 172.21.15.113:0/4088570953 learned_addr learned my addr 172.21.15.113:0/4088570953 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c60bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd6bfff640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c60c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 -- 172.21.15.113:0/4088570953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 msgr2=0x55d30c60c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c60c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 -- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 msgr2=0x55d30c60bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c60bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 -- 172.21.15.113:0/4088570953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30c27c3f0 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd713dc640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c60c540 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006010 tx=0x7fdd64034a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930833ff32fce712 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.579+0000 7fdd4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd4bfff640 1 -- 172.21.15.113:0/4088570953 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64059020 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd4bfff640 1 -- 172.21.15.113:0/4088570953 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fdd640040c0 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd64003d10 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd4bfff640 1 -- 172.21.15.113:0/4088570953 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdd64034e30 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 10 client.16023 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d30c23c0f0 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd4bfff640 1 -- 172.21.15.113:0/4088570953 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd64003d10 con 0x55d30c60ea80 2024-04-22T07:16:31.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd4bfff640 1 -- 172.21.15.113:0/4088570953 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fdd64055a70 con 0x55d30c60ea80 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 10 client.16023 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 20 client.16023 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c620c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 4 client.16023 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30c577c00 con 0x55d30c61e860 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.580+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c620c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c620c50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c21baa2c4763c3 server_cookie=9901d5382aa5f453 in_seq=0 out_seq=0 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd4bfff640 10 client.16023 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd4bfff640 1 -- 172.21.15.113:0/4088570953 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d30c577c00 con 0x55d30c61e860 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd4bfff640 10 client.16023 handle_command_reply: tid=0 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 client.16023 shutdown 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 msgr2=0x55d30c60c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c60c540 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006010 tx=0x7fdd64034a30 comp rx=0 tx=0).stop 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 shutdown_connections 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c620c50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c60c540 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c60c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd4bfff640 0 client.16023 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/4088570953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c60bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.581+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 >> 172.21.15.113:0/4088570953 conn(0x55d30c5ffcb0 msgr2=0x55d30c601a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.582+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 shutdown_connections 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.582+0000 7fdd75a2d740 1 -- 172.21.15.113:0/4088570953 wait complete. 2024-04-22T07:16:31.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.582+0000 7fdd75a2d740 20 client.16023 trim_cache size 0 max 16384 2024-04-22T07:16:31.590 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:31 smithi113 ceph-mon[27933]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-22T07:16:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 Processor -- start 2024-04-22T07:16:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 -- start start 2024-04-22T07:16:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 0x55d30c60e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c61a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c60ea80 0x55d30c60c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1b2800 con 0x55d30c605580 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1b00d0 con 0x55d30c60ea80 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1adf20 con 0x55d30c605ea0 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 0x55d30c60e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 0x55d30c60e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33054/0 (socket says 172.21.15.113:33054) 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 -- 172.21.15.113:0/2082161070 learned_addr learned my addr 172.21.15.113:0/2082161070 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 -- 172.21.15.113:0/2082161070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 msgr2=0x55d30c61a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd713dc640 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c60ea80 0x55d30c60c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c61a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 -- 172.21.15.113:0/2082161070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c60ea80 msgr2=0x55d30c60c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c60ea80 0x55d30c60c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 -- 172.21.15.113:0/2082161070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30c610e70 con 0x55d30c605580 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.599+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 0x55d30c60e1f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009b60 tx=0x7fdd60005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0d05131b5eed23 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd567fc640 1 -- 172.21.15.113:0/2082161070 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60003cd0 con 0x55d30c605580 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd567fc640 1 -- 172.21.15.113:0/2082161070 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd6001f870 con 0x55d30c605580 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd567fc640 1 -- 172.21.15.113:0/2082161070 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60012cc0 con 0x55d30c605580 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 -- 172.21.15.113:0/2082161070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 msgr2=0x55d30c60e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 0x55d30c60e1f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009b60 tx=0x7fdd60005f90 comp rx=0 tx=0).stop 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 -- 172.21.15.113:0/2082161070 shutdown_connections 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c60ea80 0x55d30c60c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c61a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/2082161070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c605580 0x55d30c60e1f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 -- 172.21.15.113:0/2082161070 >> 172.21.15.113:0/2082161070 conn(0x55d30c5ffcb0 msgr2=0x55d30c60ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 -- 172.21.15.113:0/2082161070 shutdown_connections 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 -- 172.21.15.113:0/2082161070 wait complete. 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.600+0000 7fdd75a2d740 1 -- start start 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 Processor -- start 2024-04-22T07:16:31.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c6127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c612d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c613240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1ae590 con 0x55d30c60ea80 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1afa00 con 0x55d30c605580 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1aec60 con 0x55d30c605ea0 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd75a2d740 10 client.0 _ll_register_callbacks cb 0x7fdd58098090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c613240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c613240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33058/0 (socket says 172.21.15.113:33058) 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 -- 172.21.15.113:0/1247324269 learned_addr learned my addr 172.21.15.113:0/1247324269 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c6127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 -- 172.21.15.113:0/1247324269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 msgr2=0x55d30c612d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c612d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 -- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 msgr2=0x55d30c6127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:31.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c6127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 -- 172.21.15.113:0/1247324269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30c683cb0 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd713dc640 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c613240 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdd64005fe0 tx=0x7fdd640324d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecfb68c327611f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.601+0000 7fdd4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd4bfff640 1 -- 172.21.15.113:0/1247324269 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64059040 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd4bfff640 1 -- 172.21.15.113:0/1247324269 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fdd64042c80 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6001f870 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd4bfff640 1 -- 172.21.15.113:0/1247324269 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdd64076050 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 10 client.16029 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d30c684f80 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd4bfff640 1 -- 172.21.15.113:0/1247324269 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdd6001f870 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd4bfff640 1 -- 172.21.15.113:0/1247324269 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fdd64068460 con 0x55d30c60ea80 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 10 client.16029 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 20 client.16029 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:31.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c687700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 4 client.16029 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d30c571a70 con 0x55d30c61e860 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.602+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c687700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd4bfff640 10 client.16029 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd70bdb640 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c687700 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807f920aec9de5e2 server_cookie=f52ff05e67e1b1b in_seq=0 out_seq=0 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd4bfff640 1 -- 172.21.15.113:0/1247324269 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d30c571a70 con 0x55d30c61e860 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd4bfff640 10 client.16029 handle_command_reply: tid=0 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 client.16029 shutdown 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 msgr2=0x55d30c613240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c613240 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdd64005fe0 tx=0x7fdd640324d0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 shutdown_connections 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d30c61e860 0x55d30c687700 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d30c60ea80 0x55d30c613240 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d30c605ea0 0x55d30c612d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd4bfff640 0 client.16029 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 --2- 172.21.15.113:0/1247324269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d30c605580 0x55d30c6127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 >> 172.21.15.113:0/1247324269 conn(0x55d30c5ffcb0 msgr2=0x55d30c5fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 shutdown_connections 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 1 -- 172.21.15.113:0/1247324269 wait complete. 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.603+0000 7fdd75a2d740 20 client.16029 trim_cache size 0 max 16384 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:16:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:16:31.610 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd5c0a3d50 msgr2=0x7fdd5c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd5c0a3d50 0x7fdd5c0a6210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009ca0 tx=0x7fdd64032d40 comp rx=0 tx=0).stop 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 msgr2=0x7fdd6c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c103080 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c970 tx=0x7fdd5800ce30 comp rx=0 tx=0).stop 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 shutdown_connections 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdd5c0a3d50 0x7fdd5c0a6210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd6c1035c0 0x7fdd6c1a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd6c106180 0x7fdd6c103080 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 --2- 172.21.15.113:0/500767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6c1010b0 0x7fdd6c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 >> 172.21.15.113:0/500767333 conn(0x7fdd6c0771d0 msgr2=0x7fdd6c06a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 shutdown_connections 2024-04-22T07:16:31.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.604+0000 7fdd72e3f640 1 -- 172.21.15.113:0/500767333 wait complete. 2024-04-22T07:16:31.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:16:31.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/2301603330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c100600 msgr2=0x7fe79c100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/2301603330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c100600 0x7fe79c100a80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe790009930 tx=0x7fe79002bc30 comp rx=0 tx=0).stop 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/2301603330 shutdown_connections 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/2301603330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe79c100fc0 0x7fe79c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/2301603330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c100600 0x7fe79c100a80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/2301603330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe79c1066a0 0x7fe79c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/2301603330 >> 172.21.15.113:0/2301603330 conn(0x7fe79c077480 msgr2=0x7fe79c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/2301603330 shutdown_connections 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.753+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/2301603330 wait complete. 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 Processor -- start 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 -- start start 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe79c100600 0x7fe79c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe79c100fc0 0x7fe79c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 0x7fe79c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c110130 con 0x7fe79c1066a0 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10ffb0 con 0x7fe79c100fc0 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe7a0cf2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1102b0 con 0x7fe79c100600 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 0x7fe79c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 0x7fe79c19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33082/0 (socket says 172.21.15.113:33082) 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 -- 172.21.15.113:0/946125746 learned_addr learned my addr 172.21.15.113:0/946125746 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe799d9b640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe79c100fc0 0x7fe79c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79a59c640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe79c100600 0x7fe79c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 -- 172.21.15.113:0/946125746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe79c100600 msgr2=0x7fe79c196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe79c100600 0x7fe79c196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 -- 172.21.15.113:0/946125746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe79c100fc0 msgr2=0x7fe79c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe79c100fc0 0x7fe79c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.754+0000 7fe79ad9d640 1 -- 172.21.15.113:0/946125746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c19b600 con 0x7fe79c1066a0 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.755+0000 7fe79ad9d640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 0x7fe79c19aed0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009f80 tx=0x7fe78c00f560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.755+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c003b70 con 0x7fe79c1066a0 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.755+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe78c003d10 con 0x7fe79c1066a0 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.755+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c19b8f0 con 0x7fe79c1066a0 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.755+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0179a0 con 0x7fe79c1066a0 2024-04-22T07:16:31.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.755+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1a3180 con 0x7fe79c1066a0 2024-04-22T07:16:31.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.756+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c069230 con 0x7fe79c1066a0 2024-04-22T07:16:31.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.757+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe78c017b40 con 0x7fe79c1066a0 2024-04-22T07:16:31.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.757+0000 7fe78b7fe640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7840a3d20 0x7fe7840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:31.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.758+0000 7fe79a59c640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7840a3d20 0x7fe7840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:31.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.758+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fe78c0d2800 con 0x7fe79c1066a0 2024-04-22T07:16:31.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.758+0000 7fe79a59c640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7840a3d20 0x7fe7840a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe79c101740 tx=0x7fe794005ed0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:31.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.762+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe78c012070 con 0x7fe79c1066a0 2024-04-22T07:16:31.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.975+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe79c108b90 con 0x7fe79c1066a0 2024-04-22T07:16:31.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.976+0000 7fe78b7fe640 1 -- 172.21.15.113:0/946125746 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11884 (secure 0 0 0) 0x7fe78c09ee20 con 0x7fe79c1066a0 2024-04-22T07:16:31.975 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:16:31.975 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:15:55.192202+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24445,"mds_2":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":26,"state":"up:active","state_seq":54,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":24,"state":"up:active","state_seq":53,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24433,24445,24506]},"id":1}]} 2024-04-22T07:16:31.975 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-04-22T07:16:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7840a3d20 msgr2=0x7fe7840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7840a3d20 0x7fe7840a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe79c101740 tx=0x7fe794005ed0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 msgr2=0x7fe79c19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 0x7fe79c19aed0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009f80 tx=0x7fe78c00f560 comp rx=0 tx=0).stop 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 shutdown_connections 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7840a3d20 0x7fe7840a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe79c1066a0 0x7fe79c19aed0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe79c100fc0 0x7fe79c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 --2- 172.21.15.113:0/946125746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe79c100600 0x7fe79c196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 >> 172.21.15.113:0/946125746 conn(0x7fe79c077480 msgr2=0x7fe79c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 shutdown_connections 2024-04-22T07:16:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:31.978+0000 7fe7a0cf2640 1 -- 172.21.15.113:0/946125746 wait complete. 2024-04-22T07:16:31.987 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:16:32.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.088+0000 7f16955b2640 1 -- 172.21.15.113:0/3752056256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f169010c510 msgr2=0x7f169010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.088+0000 7f16955b2640 1 --2- 172.21.15.113:0/3752056256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f169010c510 0x7f169010e9b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f168802c230 comp rx=0 tx=0).stop 2024-04-22T07:16:32.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- 172.21.15.113:0/3752056256 shutdown_connections 2024-04-22T07:16:32.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 --2- 172.21.15.113:0/3752056256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f169010c510 0x7f169010e9b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 --2- 172.21.15.113:0/3752056256 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1690103af0 0x7f169010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 --2- 172.21.15.113:0/3752056256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16901031d0 0x7f16901035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- 172.21.15.113:0/3752056256 >> 172.21.15.113:0/3752056256 conn(0x7f1690077f50 msgr2=0x7f1690078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- 172.21.15.113:0/3752056256 shutdown_connections 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- 172.21.15.113:0/3752056256 wait complete. 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 Processor -- start 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- start start 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 0x7f169019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1690103af0 0x7f169019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f169010c510 0x7f16901985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16901122e0 con 0x7f16901031d0 2024-04-22T07:16:32.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1690112160 con 0x7f1690103af0 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.089+0000 7f16955b2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1690112460 con 0x7f169010c510 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 0x7f169019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 0x7f169019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33104/0 (socket says 172.21.15.113:33104) 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 -- 172.21.15.113:0/3645481122 learned_addr learned my addr 172.21.15.113:0/3645481122 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168e7fc640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1690103af0 0x7f169019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168f7fe640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f169010c510 0x7f16901985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 -- 172.21.15.113:0/3645481122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f169010c510 msgr2=0x7f16901985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f169010c510 0x7f16901985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 -- 172.21.15.113:0/3645481122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1690103af0 msgr2=0x7f169019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1690103af0 0x7f169019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 -- 172.21.15.113:0/3645481122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1690198d60 con 0x7f16901031d0 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168e7fc640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1690103af0 0x7f169019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:16:32.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f168effd640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 0x7f169019e430 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f168000e9f0 tx=0x7f168000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f1677fff640 1 -- 172.21.15.113:0/3645481122 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168000cd80 con 0x7f16901031d0 2024-04-22T07:16:32.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1690199050 con 0x7f16901031d0 2024-04-22T07:16:32.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f1677fff640 1 -- 172.21.15.113:0/3645481122 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1680010900 con 0x7f16901031d0 2024-04-22T07:16:32.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f1677fff640 1 -- 172.21.15.113:0/3645481122 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16800188d0 con 0x7f16901031d0 2024-04-22T07:16:32.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.090+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a52f0 con 0x7f16901031d0 2024-04-22T07:16:32.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.092+0000 7f1677fff640 1 -- 172.21.15.113:0/3645481122 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1680010ab0 con 0x7f16901031d0 2024-04-22T07:16:32.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.092+0000 7f169819f740 1 Processor -- start 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 -- start start 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f1677fff640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f16780a3d20 0x7f16780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bae704b1f0 con 0x55bae70efa00 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bae6fe84b0 con 0x55bae70e64d0 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f169819f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bae70471b0 con 0x55bae70e6e80 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168effd640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f16780a3d20 0x7f16780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33110/0 (socket says 172.21.15.113:33110) 2024-04-22T07:16:32.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 -- 172.21.15.113:0/4071085296 learned_addr learned my addr 172.21.15.113:0/4071085296 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168e7fc640 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f1677fff640 1 -- 172.21.15.113:0/3645481122 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f16800d0d30 con 0x7f16901031d0 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 -- 172.21.15.113:0/4071085296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 msgr2=0x55bae70ef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 -- 172.21.15.113:0/4071085296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 msgr2=0x55bae70e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 -- 172.21.15.113:0/4071085296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae6fda1b0 con 0x55bae70efa00 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168effd640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f16780a3d20 0x7f16780a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1680099620 tx=0x7f168006e750 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.093+0000 7f168f7fe640 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70f1e50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1688036000 tx=0x7f168802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7985e8bda6d045d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f16757fa640 1 -- 172.21.15.113:0/4071085296 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688039070 con 0x55bae70efa00 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f16757fa640 1 -- 172.21.15.113:0/4071085296 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1688032bc0 con 0x55bae70efa00 2024-04-22T07:16:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f16757fa640 1 -- 172.21.15.113:0/4071085296 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688034c50 con 0x55bae70efa00 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 -- 172.21.15.113:0/4071085296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 msgr2=0x55bae70f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70f1e50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1688036000 tx=0x7f168802e040 comp rx=0 tx=0).stop 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 -- 172.21.15.113:0/4071085296 shutdown_connections 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70f1e50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 --2- 172.21.15.113:0/4071085296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 -- 172.21.15.113:0/4071085296 >> 172.21.15.113:0/4071085296 conn(0x55bae70e0ba0 msgr2=0x55bae70e0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 -- 172.21.15.113:0/4071085296 shutdown_connections 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.094+0000 7f169819f740 1 -- 172.21.15.113:0/4071085296 wait complete. 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 -- start start 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 Processor -- start 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bae70f2910 con 0x55bae70efa00 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bae7004500 con 0x55bae70e64d0 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bae70f2a90 con 0x55bae70e6e80 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f169819f740 10 client.0 _ll_register_callbacks cb 0x55bae6fd2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70ecc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33112/0 (socket says 172.21.15.113:33112) 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 -- 172.21.15.113:0/3174635737 learned_addr learned my addr 172.21.15.113:0/3174635737 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168e7fc640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 -- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 msgr2=0x55bae70ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 -- 172.21.15.113:0/3174635737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 msgr2=0x55bae70ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 -- 172.21.15.113:0/3174635737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae6d5d3f0 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168e7fc640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f168f7fe640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70ecc10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1688002230 tx=0x7f16880348f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15a3541ed57f4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.095+0000 7f1672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f1672ff5640 1 -- 172.21.15.113:0/3174635737 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688053070 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f1672ff5640 1 -- 172.21.15.113:0/3174635737 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f1688034be0 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688032bc0 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f1672ff5640 1 -- 172.21.15.113:0/3174635737 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1688039030 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 10 client.16047 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bae6e277d0 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f1672ff5640 1 -- 172.21.15.113:0/3174635737 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1688032bc0 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f1672ff5640 1 -- 172.21.15.113:0/3174635737 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f168804f950 con 0x55bae70efa00 2024-04-22T07:16:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 10 client.16047 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 10 client.16047 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 20 client.16047 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae70ffa50 0x55bae7101e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 4 client.16047 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 --> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bae6fea930 con 0x55bae70ffa50 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.096+0000 7f168e7fc640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae70ffa50 0x55bae7101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.097+0000 7f168e7fc640 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae70ffa50 0x55bae7101e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2651edab804e20ae server_cookie=587fb0ba6c59e6c9 in_seq=0 out_seq=0 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.097+0000 7f1672ff5640 10 client.16047 ms_handle_connect on v2:172.21.15.154:6834/217490889 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f1672ff5640 1 -- 172.21.15.113:0/3174635737 <== mds.2 v2:172.21.15.154:6834/217490889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bae6fea930 con 0x55bae70ffa50 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f1672ff5640 10 client.16047 handle_command_reply: tid=0 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 client.16047 shutdown 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 msgr2=0x55bae70ecc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70ecc10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1688002230 tx=0x7f16880348f0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 shutdown_connections 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae70ffa50 0x55bae7101e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70efa00 0x55bae70ecc10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70e6e80 0x55bae70ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 --2- 172.21.15.113:0/3174635737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f1672ff5640 0 client.16047 ms_handle_reset on v2:172.21.15.154:6834/217490889 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 >> 172.21.15.113:0/3174635737 conn(0x55bae70e0ba0 msgr2=0x55bae70e28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 shutdown_connections 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 1 -- 172.21.15.113:0/3174635737 wait complete. 2024-04-22T07:16:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.098+0000 7f169819f740 20 client.16047 trim_cache size 0 max 16384 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 Processor -- start 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 -- start start 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae70eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 0x55bae70ef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16901b0670 con 0x55bae70e6e80 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16901adf40 con 0x55bae70e64d0 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f169819f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16901abd90 con 0x55bae70efa00 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae70eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae70eedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33118/0 (socket says 172.21.15.113:33118) 2024-04-22T07:16:32.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 -- 172.21.15.113:0/2902750532 learned_addr learned my addr 172.21.15.113:0/2902750532 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168f7fe640 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 -- 172.21.15.113:0/2902750532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 msgr2=0x55bae70ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 0x55bae70ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 -- 172.21.15.113:0/2902750532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 msgr2=0x55bae70ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 -- 172.21.15.113:0/2902750532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae70dfa50 con 0x55bae70e6e80 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168f7fe640 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.111+0000 7f168e7fc640 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae70eedc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1684005e20 tx=0x7f168400b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd58d113cee3efb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f1674ff9640 1 -- 172.21.15.113:0/2902750532 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168401e070 con 0x55bae70e6e80 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f1674ff9640 1 -- 172.21.15.113:0/2902750532 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f168401adc0 con 0x55bae70e6e80 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f1674ff9640 1 -- 172.21.15.113:0/2902750532 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400fab0 con 0x55bae70e6e80 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 -- 172.21.15.113:0/2902750532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 msgr2=0x55bae70eedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae70eedc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1684005e20 tx=0x7f168400b370 comp rx=0 tx=0).stop 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 -- 172.21.15.113:0/2902750532 shutdown_connections 2024-04-22T07:16:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 0x55bae70ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae70eedc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 --2- 172.21.15.113:0/2902750532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae70ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 -- 172.21.15.113:0/2902750532 >> 172.21.15.113:0/2902750532 conn(0x55bae70e0ba0 msgr2=0x55bae70fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 -- 172.21.15.113:0/2902750532 shutdown_connections 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 -- 172.21.15.113:0/2902750532 wait complete. 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.112+0000 7f169819f740 1 -- start start 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 Processor -- start 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae7105350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae71026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 0x55bae7102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16901ac400 con 0x55bae70e6e80 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16901ad870 con 0x55bae70e64d0 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f16901acad0 con 0x55bae70efa00 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 10 client.0 _ll_register_callbacks cb 0x7f1688035e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae71026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae71026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33128/0 (socket says 172.21.15.113:33128) 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 -- 172.21.15.113:0/2082854915 learned_addr learned my addr 172.21.15.113:0/2082854915 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168f7fe640 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae7105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 -- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 msgr2=0x55bae7102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 0x55bae7102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 -- 172.21.15.113:0/2082854915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 msgr2=0x55bae7105350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae7105350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 -- 172.21.15.113:0/2082854915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae7103520 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f168e7fc640 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae71026e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1684004770 tx=0x7f168400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995dc83cd295817e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f1672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f1672ff5640 1 -- 172.21.15.113:0/2082854915 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168401e070 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f1672ff5640 1 -- 172.21.15.113:0/2082854915 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f1684005050 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.113+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168401adc0 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f1672ff5640 1 -- 172.21.15.113:0/2082854915 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1684005250 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 10 client.16053 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bae7165cd0 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f1672ff5640 1 -- 172.21.15.113:0/2082854915 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f168401adc0 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f1672ff5640 1 -- 172.21.15.113:0/2082854915 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f168402a070 con 0x55bae70e6e80 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 10 client.16053 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 10 client.16053 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 20 client.16053 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae7166370 0x55bae7168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 4 client.16053 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 --> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bae7058c10 con 0x55bae7166370 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f168f7fe640 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae7166370 0x55bae7168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f168f7fe640 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae7166370 0x55bae7168790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbdb0748a20eeec server_cookie=5899b863c5cbda63 in_seq=0 out_seq=0 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.114+0000 7f1672ff5640 10 client.16053 ms_handle_connect on v2:172.21.15.154:6834/217490889 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f1672ff5640 1 -- 172.21.15.113:0/2082854915 <== mds.2 v2:172.21.15.154:6834/217490889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bae7058c10 con 0x55bae7166370 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f1672ff5640 10 client.16053 handle_command_reply: tid=0 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 client.16053 shutdown 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 msgr2=0x55bae71026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae71026e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1684004770 tx=0x7f168400f7b0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 shutdown_connections 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x55bae7166370 0x55bae7168790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bae70efa00 0x55bae7102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f1672ff5640 0 client.16053 ms_handle_reset on v2:172.21.15.154:6834/217490889 2024-04-22T07:16:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bae70e6e80 0x55bae71026e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 --2- 172.21.15.113:0/2082854915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bae70e64d0 0x55bae7105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 >> 172.21.15.113:0/2082854915 conn(0x55bae70e0ba0 msgr2=0x55bae70e7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 shutdown_connections 2024-04-22T07:16:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 1 -- 172.21.15.113:0/2082854915 wait complete. 2024-04-22T07:16:32.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f169819f740 20 client.16053 trim_cache size 0 max 16384 2024-04-22T07:16:32.123 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:16:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f16780a3d20 msgr2=0x7f16780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f16780a3d20 0x7f16780a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1680099620 tx=0x7f168006e750 comp rx=0 tx=0).stop 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 msgr2=0x7f169019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 0x7f169019e430 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f168000e9f0 tx=0x7f168000eeb0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 shutdown_connections 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f16780a3d20 0x7f16780a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f169010c510 0x7f16901985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1690103af0 0x7f169019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 --2- 172.21.15.113:0/3645481122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16901031d0 0x7f169019e430 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.115+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 >> 172.21.15.113:0/3645481122 conn(0x7f1690077f50 msgr2=0x7f16900fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.116+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 shutdown_connections 2024-04-22T07:16:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.116+0000 7f16955b2640 1 -- 172.21.15.113:0/3645481122 wait complete. 2024-04-22T07:16:32.125 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:16:32.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 -- 172.21.15.113:0/2494864028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394102190 msgr2=0x7fe394102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 --2- 172.21.15.113:0/2494864028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394102190 0x7fe394102570 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe390009900 tx=0x7fe39002bc30 comp rx=0 tx=0).stop 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 -- 172.21.15.113:0/2494864028 shutdown_connections 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 --2- 172.21.15.113:0/2494864028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394107260 0x7fe39410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 --2- 172.21.15.113:0/2494864028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe394102b40 0x7fe394106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 --2- 172.21.15.113:0/2494864028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394102190 0x7fe394102570 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 -- 172.21.15.113:0/2494864028 >> 172.21.15.113:0/2494864028 conn(0x7fe3940fd990 msgr2=0x7fe3940ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 -- 172.21.15.113:0/2494864028 shutdown_connections 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.227+0000 7fe39c759640 1 -- 172.21.15.113:0/2494864028 wait complete. 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 Processor -- start 2024-04-22T07:16:32.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 -- start start 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe394102190 0x7fe39419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 0x7fe39419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394198660 0x7fe394198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe394112300 con 0x7fe394107260 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe394112180 con 0x7fe394198660 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39c759640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe394112480 con 0x7fe394102190 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 0x7fe39419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 0x7fe39419e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33136/0 (socket says 172.21.15.113:33136) 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 -- 172.21.15.113:0/2525669405 learned_addr learned my addr 172.21.15.113:0/2525669405 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39acf6640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394198660 0x7fe394198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe394102190 0x7fe39419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 -- 172.21.15.113:0/2525669405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe394102190 msgr2=0x7fe39419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe394102190 0x7fe39419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 -- 172.21.15.113:0/2525669405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394198660 msgr2=0x7fe394198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394198660 0x7fe394198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe399cf4640 1 -- 172.21.15.113:0/2525669405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe394199180 con 0x7fe394107260 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.228+0000 7fe39acf6640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394198660 0x7fe394198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.229+0000 7fe399cf4640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 0x7fe39419e9a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe38400d740 tx=0x7fe38400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.229+0000 7fe38b7fe640 1 -- 172.21.15.113:0/2525669405 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384017070 con 0x7fe394107260 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.229+0000 7fe38b7fe640 1 -- 172.21.15.113:0/2525669405 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe38400de50 con 0x7fe394107260 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.229+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe394199470 con 0x7fe394107260 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.229+0000 7fe38b7fe640 1 -- 172.21.15.113:0/2525669405 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384004d90 con 0x7fe394107260 2024-04-22T07:16:32.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.229+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a5670 con 0x7fe394107260 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.231+0000 7fe38b7fe640 1 -- 172.21.15.113:0/2525669405 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe384004f30 con 0x7fe394107260 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.231+0000 7fe39f42f740 1 Processor -- start 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.231+0000 7fe39f42f740 1 -- start start 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe38b7fe640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe3800a3da0 0x7fe3800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff5f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff68430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 0x55ec7ff6ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39f42f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ec7fec41f0 con 0x55ec7ff68a00 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39f42f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ec7fe614b0 con 0x55ec7ff5fe80 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39f42f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ec7fec01b0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe3800a3da0 0x7fe3800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe38b7fe640 1 -- 172.21.15.113:0/2525669405 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fe3840d0290 con 0x7fe394107260 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 0x55ec7ff6ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 0x55ec7ff6ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33150/0 (socket says 172.21.15.113:33150) 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 -- 172.21.15.113:0/1576978623 learned_addr learned my addr 172.21.15.113:0/1576978623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff5f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe399cf4640 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff68430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 -- 172.21.15.113:0/1576978623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5f4d0 msgr2=0x55ec7ff5f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff5f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 -- 172.21.15.113:0/1576978623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 msgr2=0x55ec7ff68430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff68430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 -- 172.21.15.113:0/1576978623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7fe531b0 con 0x55ec7ff68a00 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff5f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe3800a3da0 0x7fe3800a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe38c007fd0 tx=0x7fe38c007da0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe39acf6640 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 0x55ec7ff6ae50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe38c010df0 tx=0x7fe38c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40cdf3901f66d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe388ff9640 1 -- 172.21.15.113:0/1576978623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c021070 con 0x55ec7ff68a00 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe388ff9640 1 -- 172.21.15.113:0/1576978623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe38c013270 con 0x55ec7ff68a00 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.232+0000 7fe388ff9640 1 -- 172.21.15.113:0/1576978623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c0055d0 con 0x55ec7ff68a00 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 -- 172.21.15.113:0/1576978623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 msgr2=0x55ec7ff6ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 0x55ec7ff6ae50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe38c010df0 tx=0x7fe38c01a000 comp rx=0 tx=0).stop 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 -- 172.21.15.113:0/1576978623 shutdown_connections 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff68a00 0x55ec7ff6ae50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff68430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 --2- 172.21.15.113:0/1576978623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff5f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 -- 172.21.15.113:0/1576978623 >> 172.21.15.113:0/1576978623 conn(0x55ec7ff59ba0 msgr2=0x55ec7ff59f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 -- 172.21.15.113:0/1576978623 shutdown_connections 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 -- 172.21.15.113:0/1576978623 wait complete. 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.233+0000 7fe39f42f740 1 -- start start 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 Processor -- start 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff68320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff65c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ec7ff6b910 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ec7fe7d500 con 0x55ec7ff68a00 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ec7ff6ba90 con 0x55ec7ff5fe80 2024-04-22T07:16:32.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39f42f740 10 client.0 _ll_register_callbacks cb 0x55ec7fe4bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff68320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff68320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33158/0 (socket says 172.21.15.113:33158) 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 -- 172.21.15.113:0/2036072908 learned_addr learned my addr 172.21.15.113:0/2036072908 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39acf6640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff65c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe399cf4640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 -- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 msgr2=0x55ec7ff656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 -- 172.21.15.113:0/2036072908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 msgr2=0x55ec7ff65c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff65c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 -- 172.21.15.113:0/2036072908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7fbd63f0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff68320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe390009230 tx=0x7fe390005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68615bddfe651ee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe37dffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.234+0000 7fe37dffb640 1 -- 172.21.15.113:0/2036072908 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390005030 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe37dffb640 1 -- 172.21.15.113:0/2036072908 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fe3900051d0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe38c013270 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe37dffb640 1 -- 172.21.15.113:0/2036072908 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe39004c050 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 10 client.16065 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ec7fca07d0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe37dffb640 1 -- 172.21.15.113:0/2036072908 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe38c013270 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe37dffb640 1 -- 172.21.15.113:0/2036072908 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fe3900035f0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 10 client.16065 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 10 client.16065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 20 client.16065 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:32.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ff78ca0 0x55ec7ff7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 4 client.16065 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 --> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec7fe63930 con 0x55ec7ff78ca0 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.235+0000 7fe399cf4640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ff78ca0 0x55ec7ff7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.236+0000 7fe37dffb640 10 client.16065 ms_handle_connect on v2:172.21.15.154:6836/1615298521 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.236+0000 7fe399cf4640 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ff78ca0 0x55ec7ff7b090 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1633f223ffecc2d5 server_cookie=252efdf1cb8b77f in_seq=0 out_seq=0 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe37dffb640 1 -- 172.21.15.113:0/2036072908 <== mds.1 v2:172.21.15.154:6836/1615298521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec7fe63930 con 0x55ec7ff78ca0 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe37dffb640 10 client.16065 handle_command_reply: tid=0 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 client.16065 shutdown 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 msgr2=0x55ec7ff68320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff68320 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe390009230 tx=0x7fe390005e40 comp rx=0 tx=0).stop 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 shutdown_connections 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ff78ca0 0x55ec7ff7b090 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff65c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe37dffb640 0 client.16065 ms_handle_reset on v2:172.21.15.154:6836/1615298521 2024-04-22T07:16:32.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 --2- 172.21.15.113:0/2036072908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff68320 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 >> 172.21.15.113:0/2036072908 conn(0x55ec7ff59ba0 msgr2=0x55ec7ff5b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 shutdown_connections 2024-04-22T07:16:32.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 1 -- 172.21.15.113:0/2036072908 wait complete. 2024-04-22T07:16:32.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.237+0000 7fe39f42f740 20 client.16065 trim_cache size 0 max 16384 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 Processor -- start 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- start start 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff68a00 0x55ec7ff68320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941b0530 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941ade00 con 0x55ec7ff5fe80 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941abc70 con 0x55ec7ff68a00 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff74820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33172/0 (socket says 172.21.15.113:33172) 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4093491373 learned_addr learned my addr 172.21.15.113:0/4093491373 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4093491373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff68a00 msgr2=0x55ec7ff68320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff68a00 0x55ec7ff68320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4093491373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 msgr2=0x55ec7ff67de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4093491373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7ff5fc20 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff74820 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe39002f020 tx=0x7fe39002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0a2d34501660df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe37ffff640 1 -- 172.21.15.113:0/4093491373 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390003c30 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe37ffff640 1 -- 172.21.15.113:0/4093491373 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe3900035f0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe37ffff640 1 -- 172.21.15.113:0/4093491373 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39003d420 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- 172.21.15.113:0/4093491373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 msgr2=0x55ec7ff74820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff74820 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe39002f020 tx=0x7fe39002bc30 comp rx=0 tx=0).stop 2024-04-22T07:16:32.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- 172.21.15.113:0/4093491373 shutdown_connections 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff68a00 0x55ec7ff68320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff67de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4093491373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff74820 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- 172.21.15.113:0/4093491373 >> 172.21.15.113:0/4093491373 conn(0x55ec7ff59ba0 msgr2=0x55ec7ff58c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- 172.21.15.113:0/4093491373 shutdown_connections 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- 172.21.15.113:0/4093491373 wait complete. 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 -- start start 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.251+0000 7fe39f42f740 1 Processor -- start 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff71020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff61fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff62500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941ac2c0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941ad730 con 0x55ec7ff68a00 2024-04-22T07:16:32.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe3941ac990 con 0x55ec7ff5fe80 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 10 client.0 _ll_register_callbacks cb 0x7fe39002cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff71020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff71020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33176/0 (socket says 172.21.15.113:33176) 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4010170196 learned_addr learned my addr 172.21.15.113:0/4010170196 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe399cf4640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff61fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39acf6640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff62500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 msgr2=0x55ec7ff61fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff61fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4010170196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 msgr2=0x55ec7ff62500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff62500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 -- 172.21.15.113:0/4010170196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7ff62c40 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39acf6640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff62500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39a4f5640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff71020 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe39002c170 tx=0x7fe390005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c5eac67059cbf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe37dffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe37dffb640 1 -- 172.21.15.113:0/4010170196 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390003e40 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe37dffb640 1 -- 172.21.15.113:0/4010170196 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fe390034d50 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.252+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3900035f0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe37dffb640 1 -- 172.21.15.113:0/4010170196 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe390034440 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 10 client.16071 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ec7ffdf100 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe37dffb640 1 -- 172.21.15.113:0/4010170196 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe3900035f0 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe37dffb640 1 -- 172.21.15.113:0/4010170196 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fe390057030 con 0x55ec7ff5f4d0 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 10 client.16071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 20 client.16071 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ffdf6b0 0x55ec7ffe1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 4 client.16071 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:16:32.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 --> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec7fed1c10 con 0x55ec7ffdf6b0 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.253+0000 7fe399cf4640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ffdf6b0 0x55ec7ffe1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe399cf4640 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ffdf6b0 0x55ec7ffe1ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=919718c6e75d5f68 server_cookie=9b0a820331d1da in_seq=0 out_seq=0 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe37dffb640 10 client.16071 ms_handle_connect on v2:172.21.15.154:6836/1615298521 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe37dffb640 1 -- 172.21.15.113:0/4010170196 <== mds.1 v2:172.21.15.154:6836/1615298521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec7fed1c10 con 0x55ec7ffdf6b0 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe37dffb640 10 client.16071 handle_command_reply: tid=0 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 client.16071 shutdown 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 msgr2=0x55ec7ff71020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff71020 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe39002c170 tx=0x7fe390005930 comp rx=0 tx=0).stop 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 shutdown_connections 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x55ec7ffdf6b0 0x55ec7ffe1ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec7ff68a00 0x55ec7ff62500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ec7ff5fe80 0x55ec7ff61fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 --2- 172.21.15.113:0/4010170196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ec7ff5f4d0 0x55ec7ff71020 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe37dffb640 0 client.16071 ms_handle_reset on v2:172.21.15.154:6836/1615298521 2024-04-22T07:16:32.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 >> 172.21.15.113:0/4010170196 conn(0x55ec7ff59ba0 msgr2=0x55ec7ff60260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 shutdown_connections 2024-04-22T07:16:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 1 -- 172.21.15.113:0/4010170196 wait complete. 2024-04-22T07:16:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.254+0000 7fe39f42f740 20 client.16071 trim_cache size 0 max 16384 2024-04-22T07:16:32.260 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe3800a3da0 msgr2=0x7fe3800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe3800a3da0 0x7fe3800a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe38c007fd0 tx=0x7fe38c007da0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 msgr2=0x7fe39419e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 0x7fe39419e9a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe38400d740 tx=0x7fe38400dc00 comp rx=0 tx=0).stop 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 shutdown_connections 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe3800a3da0 0x7fe3800a6260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe394198660 0x7fe394198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe394107260 0x7fe39419e9a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 --2- 172.21.15.113:0/2525669405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe394102190 0x7fe39419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 >> 172.21.15.113:0/2525669405 conn(0x7fe3940fd990 msgr2=0x7fe3940ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 shutdown_connections 2024-04-22T07:16:32.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.255+0000 7fe39c759640 1 -- 172.21.15.113:0/2525669405 wait complete. 2024-04-22T07:16:32.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:16:32.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 -- 172.21.15.113:0/396567086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 msgr2=0x7f3fa410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/396567086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa410dd40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009ca0 tx=0x7f3f9c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 -- 172.21.15.113:0/396567086 shutdown_connections 2024-04-22T07:16:32.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/396567086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa410dd40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/396567086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fa4101a50 0x7f3fa4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/396567086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa41010a0 0x7f3fa4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 -- 172.21.15.113:0/396567086 >> 172.21.15.113:0/396567086 conn(0x7f3fa4077eb0 msgr2=0x7f3fa40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 -- 172.21.15.113:0/396567086 shutdown_connections 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.365+0000 7f3fa9e74640 1 -- 172.21.15.113:0/396567086 wait complete. 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 Processor -- start 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 -- start start 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fa41010a0 0x7f3fa4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa419a010 0x7f3fa41a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41111f0 con 0x7f3fa4106170 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa4111070 con 0x7f3fa419a010 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa9e74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa4111370 con 0x7f3fa41010a0 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa4199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33196/0 (socket says 172.21.15.113:33196) 2024-04-22T07:16:32.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2647680326 learned_addr learned my addr 172.21.15.113:0/2647680326 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa419a010 0x7f3fa41a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fa41010a0 0x7f3fa4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2647680326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fa41010a0 msgr2=0x7f3fa4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fa41010a0 0x7f3fa4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2647680326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa419a010 msgr2=0x7f3fa41a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa419a010 0x7f3fa41a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2647680326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa41a4d90 con 0x7f3fa4106170 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.366+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa419a010 0x7f3fa41a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.367+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa4199ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800c970 tx=0x7f3f9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.367+0000 7f3fa0ff9640 1 -- 172.21.15.113:0/2647680326 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98014070 con 0x7f3fa4106170 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.367+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa41a5080 con 0x7f3fa4106170 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.367+0000 7f3fa0ff9640 1 -- 172.21.15.113:0/2647680326 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3f98004440 con 0x7f3fa4106170 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.367+0000 7f3fa0ff9640 1 -- 172.21.15.113:0/2647680326 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98003e40 con 0x7f3fa4106170 2024-04-22T07:16:32.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.367+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa41a5620 con 0x7f3fa4106170 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3fa0ff9640 1 -- 172.21.15.113:0/2647680326 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f3f98017070 con 0x7f3fa4106170 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 Processor -- start 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 -- start start 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce12218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce1221e80 0x556ce122a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce122ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556ce11861f0 con 0x556ce122aa00 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ce11234b0 con 0x556ce1221e80 2024-04-22T07:16:32.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3faca6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556ce11821b0 con 0x556ce12214d0 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3fa0ff9640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3f8c0a3d20 0x7f3f8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce122ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce122ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33206/0 (socket says 172.21.15.113:33206) 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.369+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/4016445980 learned_addr learned my addr 172.21.15.113:0/4016445980 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3f8c0a3d20 0x7f3f8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce1221e80 0x556ce122a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce12218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa0ff9640 1 -- 172.21.15.113:0/2647680326 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f3f980d0700 con 0x7f3fa4106170 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/4016445980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 msgr2=0x556ce12218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce12218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/4016445980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce1221e80 msgr2=0x556ce122a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce1221e80 0x556ce122a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/4016445980 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce11151b0 con 0x556ce122aa00 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce1221e80 0x556ce122a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3f8c0a3d20 0x7f3f8c0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c006010 tx=0x7f3f9c009910 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce122ce50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3f98098ff0 tx=0x7f3f9800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4351e4925ef06d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3f89ffb640 1 -- 172.21.15.113:0/4016445980 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9809cd20 con 0x556ce122aa00 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3f89ffb640 1 -- 172.21.15.113:0/4016445980 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3f9809c620 con 0x556ce122aa00 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.370+0000 7f3f89ffb640 1 -- 172.21.15.113:0/4016445980 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f980035f0 con 0x556ce122aa00 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 -- 172.21.15.113:0/4016445980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 msgr2=0x556ce122ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce122ce50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3f98098ff0 tx=0x7f3f9800cf60 comp rx=0 tx=0).stop 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 -- 172.21.15.113:0/4016445980 shutdown_connections 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce122ce50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce1221e80 0x556ce122a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 --2- 172.21.15.113:0/4016445980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce12218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 -- 172.21.15.113:0/4016445980 >> 172.21.15.113:0/4016445980 conn(0x556ce121bba0 msgr2=0x556ce121bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 -- 172.21.15.113:0/4016445980 shutdown_connections 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 -- 172.21.15.113:0/4016445980 wait complete. 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 -- start start 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.371+0000 7f3faca6b740 1 Processor -- start 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce122a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce12276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce122aa00 0x556ce1227cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556ce122d970 con 0x556ce1221e80 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ce113f500 con 0x556ce122aa00 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556ce122daf0 con 0x556ce12214d0 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 10 client.0 _ll_register_callbacks cb 0x556ce110db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce12276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce12276c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33216/0 (socket says 172.21.15.113:33216) 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/978257990 learned_addr learned my addr 172.21.15.113:0/978257990 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce122a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/978257990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 msgr2=0x556ce122a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce122aa00 0x556ce1227cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce122a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce122aa00 msgr2=0x556ce1227cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce122aa00 0x556ce1227cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/978257990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce0e983f0 con 0x556ce1221e80 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce122a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce12276c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3f9809c320 tx=0x7f3f98075530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae02fe41d5346f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3f82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3f82ffd640 1 -- 172.21.15.113:0/978257990 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f980a5290 con 0x556ce1221e80 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3f82ffd640 1 -- 172.21.15.113:0/978257990 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f3f980a4e00 con 0x556ce1221e80 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9809c620 con 0x556ce1221e80 2024-04-22T07:16:32.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3f82ffd640 1 -- 172.21.15.113:0/978257990 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3f9809cd20 con 0x556ce1221e80 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 10 client.16083 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.372+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556ce0f627d0 con 0x556ce1221e80 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3f82ffd640 1 -- 172.21.15.113:0/978257990 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3f9809c620 con 0x556ce1221e80 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3f82ffd640 1 -- 172.21.15.113:0/978257990 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f3f9800ed30 con 0x556ce1221e80 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3faca6b740 10 client.16083 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3faca6b740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3faca6b740 20 client.16083 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3faca6b740 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce123aac0 0x556ce123ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3faca6b740 4 client.16083 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ce1125930 con 0x556ce123aac0 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.373+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce123aac0 0x556ce123ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce123aac0 0x556ce123ceb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f738245b30ddc26 server_cookie=49ec471f46b2be62 in_seq=0 out_seq=0 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3f82ffd640 10 client.16083 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3f82ffd640 1 -- 172.21.15.113:0/978257990 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ce1125930 con 0x556ce123aac0 2024-04-22T07:16:32.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3f82ffd640 10 client.16083 handle_command_reply: tid=0 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 client.16083 shutdown 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 msgr2=0x556ce12276c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce12276c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3f9809c320 tx=0x7f3f98075530 comp rx=0 tx=0).stop 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 shutdown_connections 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce122aa00 0x556ce1227cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce123aac0 0x556ce123ceb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce12276c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 --2- 172.21.15.113:0/978257990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce12214d0 0x556ce122a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3f82ffd640 0 client.16083 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.374+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 >> 172.21.15.113:0/978257990 conn(0x556ce121bba0 msgr2=0x556ce121d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.375+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 shutdown_connections 2024-04-22T07:16:32.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.375+0000 7f3faca6b740 1 -- 172.21.15.113:0/978257990 wait complete. 2024-04-22T07:16:32.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.375+0000 7f3faca6b740 20 client.16083 trim_cache size 0 max 16384 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 Processor -- start 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 -- start start 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce1229df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce1221e80 0x556ce122a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce123fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41b0610 con 0x556ce122aa00 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41adee0 con 0x556ce12214d0 2024-04-22T07:16:32.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3faca6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41abd30 con 0x556ce1221e80 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce123fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce123fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33222/0 (socket says 172.21.15.113:33222) 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 -- 172.21.15.113:0/2507750606 learned_addr learned my addr 172.21.15.113:0/2507750606 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce1229df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 -- 172.21.15.113:0/2507750606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce1221e80 msgr2=0x556ce122a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce1221e80 0x556ce122a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 -- 172.21.15.113:0/2507750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 msgr2=0x556ce1229df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce1229df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 -- 172.21.15.113:0/2507750606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce1221c20 con 0x556ce122aa00 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3fa3fff640 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce123fed0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c036000 tx=0x7f3f9c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf4ea8a3ed42d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3f897fa640 1 -- 172.21.15.113:0/2507750606 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c04a040 con 0x556ce122aa00 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3f897fa640 1 -- 172.21.15.113:0/2507750606 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3f9c053070 con 0x556ce122aa00 2024-04-22T07:16:32.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.388+0000 7f3f897fa640 1 -- 172.21.15.113:0/2507750606 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0346a0 con 0x556ce122aa00 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 -- 172.21.15.113:0/2507750606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 msgr2=0x556ce123fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce123fed0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c036000 tx=0x7f3f9c03a000 comp rx=0 tx=0).stop 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 -- 172.21.15.113:0/2507750606 shutdown_connections 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce122aa00 0x556ce123fed0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce1221e80 0x556ce122a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2507750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce1229df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 -- 172.21.15.113:0/2507750606 >> 172.21.15.113:0/2507750606 conn(0x556ce121bba0 msgr2=0x556ce12365d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 -- 172.21.15.113:0/2507750606 shutdown_connections 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 -- 172.21.15.113:0/2507750606 wait complete. 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.389+0000 7f3faca6b740 1 -- start start 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 Processor -- start 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce123da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce123df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce122aa00 0x556ce123e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41ac3a0 con 0x556ce1221e80 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41ad810 con 0x556ce12214d0 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa41aca70 con 0x556ce122aa00 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 10 client.0 _ll_register_callbacks cb 0x7f3f9809c4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce123df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce123df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33230/0 (socket says 172.21.15.113:33230) 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2834165773 learned_addr learned my addr 172.21.15.113:0/2834165773 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce123da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2834165773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce122aa00 msgr2=0x556ce123e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce122aa00 0x556ce123e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 msgr2=0x556ce123da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce123da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 -- 172.21.15.113:0/2834165773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce12a4180 con 0x556ce1221e80 2024-04-22T07:16:32.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce123da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3fa2ffd640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce123df80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f98098a80 tx=0x7f3f98098990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd72e27f30f372 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3f82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3f82ffd640 1 -- 172.21.15.113:0/2834165773 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f980a5000 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3f82ffd640 1 -- 172.21.15.113:0/2834165773 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f3f9800ec10 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.390+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9c053070 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3f82ffd640 1 -- 172.21.15.113:0/2834165773 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3f980a5290 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 10 client.16089 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556ce12a4180 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3f82ffd640 1 -- 172.21.15.113:0/2834165773 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3f9c053070 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3f82ffd640 1 -- 172.21.15.113:0/2834165773 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f3f98098370 con 0x556ce1221e80 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 10 client.16089 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 20 client.16089 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce12a5860 0x556ce12a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:32.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 4 client.16089 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ce1193c10 con 0x556ce12a5860 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce12a5860 0x556ce12a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3f82ffd640 10 client.16089 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.391+0000 7f3fa37fe640 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce12a5860 0x556ce12a7c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9a0f0f3a3fc592 server_cookie=d984bfebce98add4 in_seq=0 out_seq=0 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3f82ffd640 1 -- 172.21.15.113:0/2834165773 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ce1193c10 con 0x556ce12a5860 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3f82ffd640 10 client.16089 handle_command_reply: tid=0 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 client.16089 shutdown 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 msgr2=0x556ce123df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce123df80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f98098a80 tx=0x7f3f98098990 comp rx=0 tx=0).stop 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 shutdown_connections 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce122aa00 0x556ce123e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x556ce12a5860 0x556ce12a7c80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce1221e80 0x556ce123df80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 --2- 172.21.15.113:0/2834165773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce12214d0 0x556ce123da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3f82ffd640 0 client.16089 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:32.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 >> 172.21.15.113:0/2834165773 conn(0x556ce121bba0 msgr2=0x556ce1235d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 shutdown_connections 2024-04-22T07:16:32.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 1 -- 172.21.15.113:0/2834165773 wait complete. 2024-04-22T07:16:32.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3faca6b740 20 client.16089 trim_cache size 0 max 16384 2024-04-22T07:16:32.399 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3f8c0a3d20 msgr2=0x7f3f8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3f8c0a3d20 0x7f3f8c0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c006010 tx=0x7f3f9c009910 comp rx=0 tx=0).stop 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 msgr2=0x7f3fa4199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa4199ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800c970 tx=0x7f3f9800ce30 comp rx=0 tx=0).stop 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 shutdown_connections 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3f8c0a3d20 0x7f3f8c0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fa419a010 0x7f3fa41a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fa4106170 0x7f3fa4199ad0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 --2- 172.21.15.113:0/2647680326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3fa41010a0 0x7f3fa4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:32.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.392+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 >> 172.21.15.113:0/2647680326 conn(0x7f3fa4077eb0 msgr2=0x7f3fa40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:32.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.393+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 shutdown_connections 2024-04-22T07:16:32.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:32.393+0000 7f3fa9e74640 1 -- 172.21.15.113:0/2647680326 wait complete. 2024-04-22T07:16:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:32 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/946125746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:16:32.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:32 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/946125746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:16:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:32 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/946125746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:16:33.403 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:16:33.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:16:33.403 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 -- 172.21.15.113:0/1991497747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228102b40 msgr2=0x7fb228106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/1991497747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228102b40 0x7fb228106b80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb210009900 tx=0x7fb21002bc30 comp rx=0 tx=0).stop 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 -- 172.21.15.113:0/1991497747 shutdown_connections 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/1991497747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228107260 0x7fb22810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/1991497747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228102b40 0x7fb228106b80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/1991497747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb228102190 0x7fb228102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 -- 172.21.15.113:0/1991497747 >> 172.21.15.113:0/1991497747 conn(0x7fb2280793a0 msgr2=0x7fb2280797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 -- 172.21.15.113:0/1991497747 shutdown_connections 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.564+0000 7fb22cc9a640 1 -- 172.21.15.113:0/1991497747 wait complete. 2024-04-22T07:16:33.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 Processor -- start 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 -- start start 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228102190 0x7fb22819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 0x7fb22819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb228198680 0x7fb228198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb228112340 con 0x7fb228107260 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281121c0 con 0x7fb228198680 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22cc9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281124c0 con 0x7fb228102190 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 0x7fb22819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 0x7fb22819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33246/0 (socket says 172.21.15.113:33246) 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb21ffff640 1 -- 172.21.15.113:0/3137386049 learned_addr learned my addr 172.21.15.113:0/3137386049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb226d9d640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb228198680 0x7fb228198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.565+0000 7fb22659c640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228102190 0x7fb22819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21ffff640 1 -- 172.21.15.113:0/3137386049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228102190 msgr2=0x7fb22819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21ffff640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228102190 0x7fb22819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21ffff640 1 -- 172.21.15.113:0/3137386049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb228198680 msgr2=0x7fb228198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21ffff640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb228198680 0x7fb228198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21ffff640 1 -- 172.21.15.113:0/3137386049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb228199280 con 0x7fb228107260 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb22659c640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228102190 0x7fb22819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21ffff640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 0x7fb22819eb00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb210000c00 tx=0x7fb21002cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21f7fe640 1 -- 172.21.15.113:0/3137386049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210041070 con 0x7fb228107260 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21f7fe640 1 -- 172.21.15.113:0/3137386049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb2100052f0 con 0x7fb228107260 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb228199510 con 0x7fb228107260 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb21f7fe640 1 -- 172.21.15.113:0/3137386049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210034440 con 0x7fb228107260 2024-04-22T07:16:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.566+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb228074180 con 0x7fb228107260 2024-04-22T07:16:33.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:33 smithi154 ceph-mon[31862]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-22T07:16:33.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:33 smithi133 ceph-mon[31198]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.568+0000 7fb21f7fe640 1 -- 172.21.15.113:0/3137386049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb210044070 con 0x7fb228107260 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.569+0000 7fb21f7fe640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2140a8130 0x7fb2140aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.569+0000 7fb22659c640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2140a8130 0x7fb2140aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.569+0000 7fb21f7fe640 1 -- 172.21.15.113:0/3137386049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fb2100f2300 con 0x7fb228107260 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 Processor -- start 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22659c640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2140a8130 0x7fb2140aa5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb218009930 tx=0x7fb218008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 -- start start 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 0x5604935f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56049354bf00 con 0x5604935f0e10 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5604934e9590 con 0x5604935e78e0 2024-04-22T07:16:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb22f96d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560493547ec0 con 0x5604935e8290 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.570+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 0x5604935f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 0x5604935f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:52704/0 (socket says 172.21.15.113:52704) 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 -- 172.21.15.113:0/1704630166 learned_addr learned my addr 172.21.15.113:0/1704630166 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb22659c640 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb226d9d640 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 -- 172.21.15.113:0/1704630166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 msgr2=0x5604935e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 -- 172.21.15.113:0/1704630166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 msgr2=0x5604935f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.571+0000 7fb21ffff640 1 -- 172.21.15.113:0/1704630166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604933267e0 con 0x5604935e8290 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.572+0000 7fb21ffff640 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 0x5604935f0840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2100bad00 tx=0x7fb210049000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1342a156ac55e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.572+0000 7fb21cff9640 1 -- 172.21.15.113:0/1704630166 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21004b040 con 0x5604935e8290 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.572+0000 7fb21cff9640 1 -- 172.21.15.113:0/1704630166 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb2100be330 con 0x5604935e8290 2024-04-22T07:16:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.572+0000 7fb21cff9640 1 -- 172.21.15.113:0/1704630166 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100bab00 con 0x5604935e8290 2024-04-22T07:16:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 -- 172.21.15.113:0/1704630166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 msgr2=0x5604935f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 0x5604935f0840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2100bad00 tx=0x7fb210049000 comp rx=0 tx=0).stop 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 -- 172.21.15.113:0/1704630166 shutdown_connections 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e8290 0x5604935f0840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1704630166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 -- 172.21.15.113:0/1704630166 >> 172.21.15.113:0/1704630166 conn(0x5604935e1fb0 msgr2=0x5604935e2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 -- 172.21.15.113:0/1704630166 shutdown_connections 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 -- 172.21.15.113:0/1704630166 wait complete. 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 -- start start 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.573+0000 7fb22f96d740 1 Processor -- start 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935e8290 0x5604935ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 0x5604935ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5604935f3b40 con 0x5604935e8290 2024-04-22T07:16:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560493505580 con 0x5604935e78e0 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5604935f3cc0 con 0x5604935f0e10 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22f96d740 10 client.0 _ll_register_callbacks cb 0x56049350b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb226d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 0x5604935ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb226d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 0x5604935ee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:52714/0 (socket says 172.21.15.113:52714) 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb226d9d640 1 -- 172.21.15.113:0/4059873903 learned_addr learned my addr 172.21.15.113:0/4059873903 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb22659c640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.574+0000 7fb21ffff640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935e8290 0x5604935ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb226d9d640 1 -- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 msgr2=0x5604935eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb226d9d640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb226d9d640 1 -- 172.21.15.113:0/4059873903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935e8290 msgr2=0x5604935ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb226d9d640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935e8290 0x5604935ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb226d9d640 1 -- 172.21.15.113:0/4059873903 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560493386f70 con 0x5604935f0e10 2024-04-22T07:16:33.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb226d9d640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 0x5604935ee840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb2200109f0 tx=0x7fb220010eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5e429d1958fb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb20dffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb20dffb640 1 -- 172.21.15.113:0/4059873903 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22000ec50 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb20dffb640 1 -- 172.21.15.113:0/4059873903 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fb22000edf0 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2100be330 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb20dffb640 1 -- 172.21.15.113:0/4059873903 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb220011440 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb22f96d740 10 client.25399 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.575+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56049321e170 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb20dffb640 1 -- 172.21.15.113:0/4059873903 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb2100be330 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb20dffb640 1 -- 172.21.15.113:0/4059873903 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fb22001a870 con 0x5604935f0e10 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb22f96d740 10 client.25399 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb22f96d740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb22f96d740 20 client.25399 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb22f96d740 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5604936014d0 0x5604936038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb22f96d740 4 client.25399 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:33.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.576+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560493559920 con 0x5604936014d0 2024-04-22T07:16:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.577+0000 7fb22659c640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5604936014d0 0x5604936038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.579+0000 7fb20dffb640 10 client.25399 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.579+0000 7fb22659c640 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5604936014d0 0x5604936038f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fe4b0669e81197 server_cookie=8fa2c2004e54b486 in_seq=0 out_seq=0 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb20dffb640 1 -- 172.21.15.113:0/4059873903 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560493559920 con 0x5604936014d0 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb20dffb640 10 client.25399 handle_command_reply: tid=0 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 client.25399 shutdown 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 msgr2=0x5604935ee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 0x5604935ee840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb2200109f0 tx=0x7fb220010eb0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 shutdown_connections 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935f0e10 0x5604935ee840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5604936014d0 0x5604936038f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935e8290 0x5604935ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 --2- 172.21.15.113:0/4059873903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e78e0 0x5604935eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 >> 172.21.15.113:0/4059873903 conn(0x5604935e1fb0 msgr2=0x5604935e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb20dffb640 0 client.25399 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 shutdown_connections 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 1 -- 172.21.15.113:0/4059873903 wait complete. 2024-04-22T07:16:33.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.581+0000 7fb22f96d740 20 client.25399 trim_cache size 0 max 16384 2024-04-22T07:16:33.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:33 smithi113 ceph-mon[27933]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 Processor -- start 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 -- start start 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935ee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281ae420 con 0x5604935f0e10 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281abcf0 con 0x5604935e8290 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.600+0000 7fb22f96d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281a9b60 con 0x5604935e78e0 2024-04-22T07:16:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935e3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33270/0 (socket says 172.21.15.113:33270) 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 -- 172.21.15.113:0/3226979651 learned_addr learned my addr 172.21.15.113:0/3226979651 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb21ffff640 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb22659c640 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 -- 172.21.15.113:0/3226979651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 msgr2=0x5604935ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 -- 172.21.15.113:0/3226979651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 msgr2=0x5604935fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935fc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 -- 172.21.15.113:0/3226979651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604935e6c00 con 0x5604935f0e10 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb21ffff640 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb226d9d640 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935e3a80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb220007f50 tx=0x7fb220007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab25a08f3a0c5314 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb20ffff640 1 -- 172.21.15.113:0/3226979651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22000ec50 con 0x5604935f0e10 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb20ffff640 1 -- 172.21.15.113:0/3226979651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb22000e720 con 0x5604935f0e10 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.601+0000 7fb20ffff640 1 -- 172.21.15.113:0/3226979651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220005030 con 0x5604935f0e10 2024-04-22T07:16:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 -- 172.21.15.113:0/3226979651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 msgr2=0x5604935e3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935e3a80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb220007f50 tx=0x7fb220007780 comp rx=0 tx=0).stop 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 -- 172.21.15.113:0/3226979651 shutdown_connections 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935e3a80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 --2- 172.21.15.113:0/3226979651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 -- 172.21.15.113:0/3226979651 >> 172.21.15.113:0/3226979651 conn(0x5604935e1fb0 msgr2=0x5604935e3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 -- 172.21.15.113:0/3226979651 shutdown_connections 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 -- 172.21.15.113:0/3226979651 wait complete. 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 -- start start 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.602+0000 7fb22f96d740 1 Processor -- start 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935ea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281aa1b0 con 0x5604935f0e10 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281ab620 con 0x5604935e8290 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281aa880 con 0x5604935e78e0 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 10 client.0 _ll_register_callbacks cb 0x7fb220010600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:52718/0 (socket says 172.21.15.113:52718) 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22659c640 1 -- 172.21.15.113:0/1382667899 learned_addr learned my addr 172.21.15.113:0/1382667899 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb21ffff640 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 -- 172.21.15.113:0/1382667899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 msgr2=0x5604935f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 -- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 msgr2=0x5604935ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935ea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 -- 172.21.15.113:0/1382667899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604935eb290 con 0x5604935f0e10 2024-04-22T07:16:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb226d9d640 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935eabd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb22001ac10 tx=0x7fb22001a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0e063bd342570d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb20dffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb20dffb640 1 -- 172.21.15.113:0/1382667899 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22000ea50 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb20dffb640 1 -- 172.21.15.113:0/1382667899 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fb22001ae00 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb22000e720 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb20dffb640 1 -- 172.21.15.113:0/1382667899 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb22003b050 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 10 client.16104 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.603+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560493667930 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb20dffb640 1 -- 172.21.15.113:0/1382667899 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb22000e720 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb20dffb640 1 -- 172.21.15.113:0/1382667899 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fb22002a020 con 0x5604935f0e10 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22f96d740 10 client.16104 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22f96d740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22f96d740 20 client.16104 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560493667ee0 0x56049366a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22f96d740 4 client.16104 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:16:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560493553790 con 0x560493667ee0 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22659c640 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560493667ee0 0x56049366a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb20dffb640 10 client.16104 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.604+0000 7fb22659c640 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560493667ee0 0x56049366a2d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f8cec7a1bd9c65 server_cookie=b8b55389ae2c8e8c in_seq=0 out_seq=0 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.605+0000 7fb20dffb640 1 -- 172.21.15.113:0/1382667899 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560493553790 con 0x560493667ee0 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.605+0000 7fb20dffb640 10 client.16104 handle_command_reply: tid=0 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.605+0000 7fb22f96d740 1 client.16104 shutdown 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 msgr2=0x5604935eabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935eabd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb22001ac10 tx=0x7fb22001a660 comp rx=0 tx=0).stop 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 shutdown_connections 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x560493667ee0 0x56049366a2d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5604935f0e10 0x5604935eabd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604935e8290 0x5604935ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 --2- 172.21.15.113:0/1382667899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5604935e78e0 0x5604935f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb20dffb640 0 client.16104 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 >> 172.21.15.113:0/1382667899 conn(0x5604935e1fb0 msgr2=0x5604935e1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 shutdown_connections 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 1 -- 172.21.15.113:0/1382667899 wait complete. 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22f96d740 20 client.16104 trim_cache size 0 max 16384 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "4e5b9972-be7c-41c6-b41d-5155a1c5a967", 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:16:33.610 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:16:33.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2140a8130 msgr2=0x7fb2140aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2140a8130 0x7fb2140aa5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb218009930 tx=0x7fb218008040 comp rx=0 tx=0).stop 2024-04-22T07:16:33.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 msgr2=0x7fb22819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 0x7fb22819eb00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb210000c00 tx=0x7fb21002cdf0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 shutdown_connections 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2140a8130 0x7fb2140aa5f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb228198680 0x7fb228198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb228107260 0x7fb22819eb00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 --2- 172.21.15.113:0/3137386049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb228102190 0x7fb22819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 >> 172.21.15.113:0/3137386049 conn(0x7fb2280793a0 msgr2=0x7fb2280ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 shutdown_connections 2024-04-22T07:16:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.606+0000 7fb22cc9a640 1 -- 172.21.15.113:0/3137386049 wait complete. 2024-04-22T07:16:33.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:16:33.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:16:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 -- 172.21.15.113:0/931473932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 msgr2=0x7fad6410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 --2- 172.21.15.113:0/931473932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6410ee40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009ca0 tx=0x7fad5c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 -- 172.21.15.113:0/931473932 shutdown_connections 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 --2- 172.21.15.113:0/931473932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6410ee40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 --2- 172.21.15.113:0/931473932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102b50 0x7fad64106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 --2- 172.21.15.113:0/931473932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad641021a0 0x7fad64102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 -- 172.21.15.113:0/931473932 >> 172.21.15.113:0/931473932 conn(0x7fad640793e0 msgr2=0x7fad640797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 -- 172.21.15.113:0/931473932 shutdown_connections 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.721+0000 7fad6a992640 1 -- 172.21.15.113:0/931473932 wait complete. 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 Processor -- start 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 -- start start 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad641021a0 0x7fad6419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64198630 0x7fad64198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112310 con 0x7fad64107270 2024-04-22T07:16:33.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112190 con 0x7fad64198630 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.722+0000 7fad6a992640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad64112490 con 0x7fad641021a0 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad63fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad641021a0 0x7fad6419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33298/0 (socket says 172.21.15.113:33298) 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 -- 172.21.15.113:0/4281918815 learned_addr learned my addr 172.21.15.113:0/4281918815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad68f2f640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64198630 0x7fad64198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 -- 172.21.15.113:0/4281918815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad641021a0 msgr2=0x7fad6419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad641021a0 0x7fad6419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 -- 172.21.15.113:0/4281918815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64198630 msgr2=0x7fad64198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64198630 0x7fad64198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 -- 172.21.15.113:0/4281918815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad64199150 con 0x7fad64107270 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad637fe640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6419e970 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fad5800e9c0 tx=0x7fad5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad617fa640 1 -- 172.21.15.113:0/4281918815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5800cd80 con 0x7fad64107270 2024-04-22T07:16:33.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad617fa640 1 -- 172.21.15.113:0/4281918815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad580054f0 con 0x7fad64107270 2024-04-22T07:16:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad64199440 con 0x7fad64107270 2024-04-22T07:16:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad617fa640 1 -- 172.21.15.113:0/4281918815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58010690 con 0x7fad64107270 2024-04-22T07:16:33.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.723+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641a5640 con 0x7fad64107270 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.725+0000 7fad617fa640 1 -- 172.21.15.113:0/4281918815 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fad580046c0 con 0x7fad64107270 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 Processor -- start 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 -- start start 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad617fa640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae88960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae91540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae93e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ab2aded1e0 con 0x55ab2ae88580 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ab2ad8a390 con 0x55ab2ae91a80 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad6d578740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ab2ade91a0 con 0x55ab2ae88ea0 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae88960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad68f2f640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae88960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33304/0 (socket says 172.21.15.113:33304) 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 -- 172.21.15.113:0/460174098 learned_addr learned my addr 172.21.15.113:0/460174098 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad637fe640 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae91540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad68f2f640 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae93e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad617fa640 1 -- 172.21.15.113:0/4281918815 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fad580d0d40 con 0x7fad64107270 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 -- 172.21.15.113:0/460174098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 msgr2=0x55ab2ae91540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae91540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 -- 172.21.15.113:0/460174098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 msgr2=0x55ab2ae93e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae93e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad63fff640 1 -- 172.21.15.113:0/460174098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2abc97d0 con 0x55ab2ae88580 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad637fe640 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae91540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.726+0000 7fad68f2f640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad5c004af0 tx=0x7fad5c0323c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad63fff640 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae88960 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fad54009d20 tx=0x7fad54011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4c475cca91989d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad4a7fc640 1 -- 172.21.15.113:0/460174098 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5401e070 con 0x55ab2ae88580 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad4a7fc640 1 -- 172.21.15.113:0/460174098 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad54019b00 con 0x55ab2ae88580 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad4a7fc640 1 -- 172.21.15.113:0/460174098 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5400eaa0 con 0x55ab2ae88580 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 -- 172.21.15.113:0/460174098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 msgr2=0x55ab2ae88960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae88960 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fad54009d20 tx=0x7fad54011810 comp rx=0 tx=0).stop 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 -- 172.21.15.113:0/460174098 shutdown_connections 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae93e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae91540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 --2- 172.21.15.113:0/460174098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae88960 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 -- 172.21.15.113:0/460174098 >> 172.21.15.113:0/460174098 conn(0x55ab2ae82cb0 msgr2=0x55ab2ae83090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.727+0000 7fad6d578740 1 -- 172.21.15.113:0/460174098 shutdown_connections 2024-04-22T07:16:33.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 -- 172.21.15.113:0/460174098 wait complete. 2024-04-22T07:16:33.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 -- start start 2024-04-22T07:16:33.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 Processor -- start 2024-04-22T07:16:33.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae8eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae8f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ab2ae94870 con 0x55ab2ae88580 2024-04-22T07:16:33.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ab2ada62f0 con 0x55ab2ae91a80 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ab2ae949f0 con 0x55ab2ae88ea0 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad6d578740 10 client.0 _ll_register_callbacks cb 0x55ab2adb3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae8eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae8eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33312/0 (socket says 172.21.15.113:33312) 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad63fff640 1 -- 172.21.15.113:0/2823439670 learned_addr learned my addr 172.21.15.113:0/2823439670 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad68f2f640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae8f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.728+0000 7fad637fe640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad63fff640 1 -- 172.21.15.113:0/2823439670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 msgr2=0x55ab2ae8f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad63fff640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad63fff640 1 -- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 msgr2=0x55ab2ae8f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad63fff640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae8f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad63fff640 1 -- 172.21.15.113:0/2823439670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2aaff3f0 con 0x55ab2ae88580 2024-04-22T07:16:33.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad637fe640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad63fff640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae8eae0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fad54005890 tx=0x7fad54019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c6aa31d288374d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad437fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad437fe640 1 -- 172.21.15.113:0/2823439670 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54027020 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad437fe640 1 -- 172.21.15.113:0/2823439670 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fad54005250 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad54019b00 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad437fe640 1 -- 172.21.15.113:0/2823439670 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fad54032050 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad6d578740 10 client.16116 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ab2aabf0f0 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad437fe640 1 -- 172.21.15.113:0/2823439670 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad54019b00 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad437fe640 1 -- 172.21.15.113:0/2823439670 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fad540046c0 con 0x55ab2ae88580 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad6d578740 10 client.16116 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad6d578740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.729+0000 7fad6d578740 20 client.16116 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.730+0000 7fad6d578740 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2aea1a40 0x55ab2aea3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.730+0000 7fad6d578740 4 client.16116 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.730+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab2adfac00 con 0x55ab2aea1a40 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.730+0000 7fad637fe640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2aea1a40 0x55ab2aea3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.730+0000 7fad437fe640 10 client.16116 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.730+0000 7fad637fe640 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2aea1a40 0x55ab2aea3e30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d3e23f1e07613d server_cookie=2bea46b293e67771 in_seq=0 out_seq=0 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad437fe640 1 -- 172.21.15.113:0/2823439670 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab2adfac00 con 0x55ab2aea1a40 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad437fe640 10 client.16116 handle_command_reply: tid=0 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 client.16116 shutdown 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 msgr2=0x55ab2ae8eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae8eae0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fad54005890 tx=0x7fad54019660 comp rx=0 tx=0).stop 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 shutdown_connections 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae91a80 0x55ab2ae8f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2aea1a40 0x55ab2aea3e30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 --2- 172.21.15.113:0/2823439670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae8eae0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 >> 172.21.15.113:0/2823439670 conn(0x55ab2ae82cb0 msgr2=0x55ab2ae84a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad437fe640 0 client.16116 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 shutdown_connections 2024-04-22T07:16:33.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 1 -- 172.21.15.113:0/2823439670 wait complete. 2024-04-22T07:16:33.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.732+0000 7fad6d578740 20 client.16116 trim_cache size 0 max 16384 2024-04-22T07:16:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 Processor -- start 2024-04-22T07:16:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 -- start start 2024-04-22T07:16:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae91210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae9d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad641b0690 con 0x55ab2ae88580 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad641adf60 con 0x55ab2ae88ea0 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad6d578740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad641abdd0 con 0x55ab2ae91a80 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae91210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae91210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33328/0 (socket says 172.21.15.113:33328) 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 -- 172.21.15.113:0/32118535 learned_addr learned my addr 172.21.15.113:0/32118535 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad637fe640 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad68f2f640 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae9d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 -- 172.21.15.113:0/32118535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 msgr2=0x55ab2ae9d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae9d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 -- 172.21.15.113:0/32118535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88ea0 msgr2=0x55ab2ae8da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 -- 172.21.15.113:0/32118535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab2ae88cd0 con 0x55ab2ae88580 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad63fff640 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae91210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad54011d10 tx=0x7fad54009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21753047a39530c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.751+0000 7fad49ffb640 1 -- 172.21.15.113:0/32118535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54027040 con 0x55ab2ae88580 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad49ffb640 1 -- 172.21.15.113:0/32118535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad5401f040 con 0x55ab2ae88580 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad49ffb640 1 -- 172.21.15.113:0/32118535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5401e070 con 0x55ab2ae88580 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 -- 172.21.15.113:0/32118535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 msgr2=0x55ab2ae91210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae91210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad54011d10 tx=0x7fad54009c20 comp rx=0 tx=0).stop 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 -- 172.21.15.113:0/32118535 shutdown_connections 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae9d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae8da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 --2- 172.21.15.113:0/32118535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88580 0x55ab2ae91210 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 -- 172.21.15.113:0/32118535 >> 172.21.15.113:0/32118535 conn(0x55ab2ae82cb0 msgr2=0x55ab2ae81d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 -- 172.21.15.113:0/32118535 shutdown_connections 2024-04-22T07:16:33.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 -- 172.21.15.113:0/32118535 wait complete. 2024-04-22T07:16:33.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.752+0000 7fad6d578740 1 -- start start 2024-04-22T07:16:33.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 Processor -- start 2024-04-22T07:16:33.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88580 0x55ab2ae95790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae95cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae96210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad641ac420 con 0x55ab2ae88ea0 2024-04-22T07:16:33.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad641ad890 con 0x55ab2ae88580 2024-04-22T07:16:33.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad641acaf0 con 0x55ab2ae91a80 2024-04-22T07:16:33.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad6d578740 10 client.0 _ll_register_callbacks cb 0x7fad5401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:16:33.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae95cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae95cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33336/0 (socket says 172.21.15.113:33336) 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 -- 172.21.15.113:0/3169464264 learned_addr learned my addr 172.21.15.113:0/3169464264 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad63fff640 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88580 0x55ab2ae95790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 -- 172.21.15.113:0/3169464264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 msgr2=0x55ab2ae96210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae96210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 -- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88580 msgr2=0x55ab2ae95790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88580 0x55ab2ae95790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 -- 172.21.15.113:0/3169464264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab2af06fb0 con 0x55ab2ae88ea0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad637fe640 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae95cd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fad580ce470 tx=0x7fad5809d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34f9cc97241e866 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.753+0000 7fad437fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad437fe640 1 -- 172.21.15.113:0/3169464264 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad580a1010 con 0x55ab2ae88ea0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad437fe640 1 -- 172.21.15.113:0/3169464264 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fad580b1330 con 0x55ab2ae88ea0 2024-04-22T07:16:33.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad5401f040 con 0x55ab2ae88ea0 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad437fe640 1 -- 172.21.15.113:0/3169464264 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fad580a58d0 con 0x55ab2ae88ea0 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 10 client.16122 fetch_fsmap learned FSMap version 31 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ab2af08300 con 0x55ab2ae88ea0 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad437fe640 1 -- 172.21.15.113:0/3169464264 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad5401f040 con 0x55ab2ae88ea0 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad437fe640 1 -- 172.21.15.113:0/3169464264 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fad580a54b0 con 0x55ab2ae88ea0 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 10 client.16122 fetch_fsmap finished waiting for FSMap version 31 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:16:33.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 20 client.16122 populate_metadata read hostname 'smithi113' 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2af08a30 0x55ab2af0ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 4 client.16122 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab2adf4a70 con 0x55ab2af08a30 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.754+0000 7fad63fff640 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2af08a30 0x55ab2af0ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad437fe640 10 client.16122 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad63fff640 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2af08a30 0x55ab2af0ae50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3898f1f4a0342f5 server_cookie=1c95c16be5448c31 in_seq=0 out_seq=0 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad437fe640 1 -- 172.21.15.113:0/3169464264 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab2adf4a70 con 0x55ab2af08a30 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad437fe640 10 client.16122 handle_command_reply: tid=0 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 client.16122 shutdown 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 msgr2=0x55ab2ae95cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae95cd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fad580ce470 tx=0x7fad5809d420 comp rx=0 tx=0).stop 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 shutdown_connections 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ab2ae91a80 0x55ab2ae96210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ab2af08a30 0x55ab2af0ae50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ab2ae88ea0 0x55ab2ae95cd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 --2- 172.21.15.113:0/3169464264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ab2ae88580 0x55ab2ae95790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad437fe640 0 client.16122 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 >> 172.21.15.113:0/3169464264 conn(0x55ab2ae82cb0 msgr2=0x55ab2ae8ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 shutdown_connections 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.755+0000 7fad6d578740 1 -- 172.21.15.113:0/3169464264 wait complete. 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6d578740 20 client.16122 trim_cache size 0 max 16384 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:16:33.761 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "4e5b9972-be7c-41c6-b41d-5155a1c5a967": { 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "4e5b9972-be7c-41c6-b41d-5155a1c5a967", 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:16:33.762 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad4c0a3d20 msgr2=0x7fad4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad5c004af0 tx=0x7fad5c0323c0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 msgr2=0x7fad6419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6419e970 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fad5800e9c0 tx=0x7fad5800ee80 comp rx=0 tx=0).stop 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 shutdown_connections 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad4c0a3d20 0x7fad4c0a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64198630 0x7fad64198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad64107270 0x7fad6419e970 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 --2- 172.21.15.113:0/4281918815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad641021a0 0x7fad6419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:16:33.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 >> 172.21.15.113:0/4281918815 conn(0x7fad640793e0 msgr2=0x7fad640ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:16:33.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 shutdown_connections 2024-04-22T07:16:33.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:16:33.756+0000 7fad6a992640 1 -- 172.21.15.113:0/4281918815 wait complete. 2024-04-22T07:16:33.765 INFO:tasks.cephfs.filesystem:scrub status for tag:4e5b9972-be7c-41c6-b41d-5155a1c5a967 - {'path': '/', 'tag': '4e5b9972-be7c-41c6-b41d-5155a1c5a967', 'options': 'recursive,force'} 2024-04-22T07:16:33.785 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-04-22T07:16:33.787 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-04-22T07:16:33.788 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-04-22T07:16:33.789 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-04-22T07:16:33.790 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-04-22T07:16:33.791 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-04-22T07:16:33.791 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-04-22T07:16:33.792 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-04-22T07:16:33.793 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-04-22T07:16:33.794 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-04-22T07:16:33.795 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-04-22T07:16:33.796 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-04-22T07:16:33.797 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-04-22T07:16:33.798 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-04-22T07:16:33.800 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-04-22T07:16:33.800 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-04-22T07:16:33.801 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-04-22T07:16:33.802 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-04-22T07:16:33.803 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-04-22T07:16:33.804 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-04-22T07:16:33.805 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-04-22T07:16:33.806 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-04-22T07:16:33.807 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-04-22T07:16:33.808 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-04-22T07:16:33.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-04-22T07:16:33.810 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-04-22T07:16:33.811 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-04-22T07:16:33.812 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-04-22T07:16:33.813 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-04-22T07:16:33.814 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-04-22T07:16:33.815 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-04-22T07:16:33.816 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-04-22T07:16:33.817 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-04-22T07:16:33.818 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-04-22T07:16:33.819 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-04-22T07:16:33.820 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-04-22T07:16:33.821 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-04-22T07:16:33.822 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-04-22T07:16:33.823 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-04-22T07:16:33.824 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-04-22T07:16:33.825 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-04-22T07:16:33.826 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-04-22T07:16:33.827 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-04-22T07:16:33.828 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-04-22T07:16:33.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-04-22T07:16:33.830 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-04-22T07:16:33.831 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-04-22T07:16:33.832 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-04-22T07:16:33.833 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-04-22T07:16:33.834 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-04-22T07:16:33.835 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-04-22T07:16:33.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-04-22T07:16:33.837 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-04-22T07:16:33.838 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-04-22T07:16:33.839 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-04-22T07:16:33.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-04-22T07:16:33.841 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-04-22T07:16:33.842 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-04-22T07:16:33.843 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-04-22T07:16:33.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-04-22T07:16:33.845 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-04-22T07:16:33.846 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-04-22T07:16:33.847 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-04-22T07:16:33.848 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/Changes 2024-04-22T07:16:33.849 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-04-22T07:16:33.850 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/CodingStyle 2024-04-22T07:16:33.851 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-04-22T07:16:33.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-04-22T07:16:33.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-04-22T07:16:33.854 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-04-22T07:16:33.855 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/ 2024-04-22T07:16:33.856 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-04-22T07:16:33.857 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-04-22T07:16:33.858 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-04-22T07:16:33.859 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-04-22T07:16:33.860 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-04-22T07:16:33.860 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-04-22T07:16:33.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-04-22T07:16:33.862 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-04-22T07:16:33.863 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-04-22T07:16:33.866 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-04-22T07:16:33.867 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-04-22T07:16:33.867 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-04-22T07:16:33.868 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-04-22T07:16:33.869 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-04-22T07:16:33.870 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-04-22T07:16:33.871 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-04-22T07:16:33.872 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-04-22T07:16:33.873 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-04-22T07:16:33.874 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-04-22T07:16:33.875 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-04-22T07:16:33.876 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-04-22T07:16:33.876 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-04-22T07:16:33.877 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-04-22T07:16:33.879 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-04-22T07:16:33.880 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-04-22T07:16:33.880 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-04-22T07:16:33.881 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-04-22T07:16:33.883 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-04-22T07:16:33.883 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-04-22T07:16:33.884 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-04-22T07:16:33.885 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-04-22T07:16:33.887 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-04-22T07:16:33.888 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-04-22T07:16:33.889 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-04-22T07:16:33.890 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-04-22T07:16:33.891 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-04-22T07:16:33.892 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-04-22T07:16:33.893 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-04-22T07:16:33.894 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-04-22T07:16:33.895 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-04-22T07:16:33.897 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-04-22T07:16:33.899 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-04-22T07:16:33.900 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-04-22T07:16:33.901 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-04-22T07:16:33.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-04-22T07:16:33.903 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-04-22T07:16:33.904 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-04-22T07:16:33.905 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-04-22T07:16:33.906 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-04-22T07:16:33.907 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-04-22T07:16:33.908 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-04-22T07:16:33.909 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-04-22T07:16:33.912 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-04-22T07:16:33.913 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-04-22T07:16:33.914 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-04-22T07:16:33.915 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-04-22T07:16:33.915 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-04-22T07:16:33.916 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-04-22T07:16:33.917 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-04-22T07:16:33.918 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-04-22T07:16:33.919 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-04-22T07:16:33.920 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-04-22T07:16:33.921 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-04-22T07:16:33.922 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-04-22T07:16:33.923 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-04-22T07:16:33.924 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-04-22T07:16:33.925 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-04-22T07:16:33.926 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-04-22T07:16:33.927 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-04-22T07:16:33.928 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-04-22T07:16:33.929 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-04-22T07:16:33.930 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-04-22T07:16:33.931 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-04-22T07:16:33.932 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-04-22T07:16:33.933 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-04-22T07:16:33.934 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-04-22T07:16:33.935 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-04-22T07:16:33.936 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-04-22T07:16:33.937 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-04-22T07:16:33.938 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-04-22T07:16:33.939 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-04-22T07:16:33.940 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-04-22T07:16:33.941 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-04-22T07:16:33.942 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-04-22T07:16:33.943 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-04-22T07:16:33.944 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-04-22T07:16:33.945 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-04-22T07:16:33.946 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-04-22T07:16:33.947 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-04-22T07:16:33.948 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-04-22T07:16:33.949 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-04-22T07:16:33.950 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-04-22T07:16:33.951 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-04-22T07:16:33.952 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-04-22T07:16:33.953 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-04-22T07:16:33.954 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-04-22T07:16:33.955 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-04-22T07:16:33.956 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-04-22T07:16:33.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-04-22T07:16:33.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-04-22T07:16:33.958 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-04-22T07:16:33.960 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-04-22T07:16:33.961 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-04-22T07:16:33.961 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-04-22T07:16:33.962 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-04-22T07:16:33.963 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-04-22T07:16:33.964 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-04-22T07:16:33.965 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-04-22T07:16:33.966 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-04-22T07:16:33.967 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-04-22T07:16:33.968 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-04-22T07:16:33.969 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-04-22T07:16:33.970 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-04-22T07:16:33.970 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-04-22T07:16:33.971 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-04-22T07:16:33.972 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-04-22T07:16:33.973 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-04-22T07:16:33.974 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-04-22T07:16:33.975 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-04-22T07:16:33.976 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-04-22T07:16:33.976 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-04-22T07:16:33.977 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-04-22T07:16:33.978 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-04-22T07:16:33.979 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-04-22T07:16:33.980 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-04-22T07:16:33.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-04-22T07:16:33.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-04-22T07:16:33.982 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-04-22T07:16:33.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-04-22T07:16:33.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-04-22T07:16:33.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-04-22T07:16:33.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-04-22T07:16:33.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-04-22T07:16:33.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-04-22T07:16:33.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-04-22T07:16:33.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-04-22T07:16:33.991 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-04-22T07:16:33.992 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-04-22T07:16:33.993 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-04-22T07:16:33.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-04-22T07:16:33.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-04-22T07:16:33.996 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-04-22T07:16:33.996 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-04-22T07:16:33.997 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-04-22T07:16:33.998 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-04-22T07:16:33.999 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-04-22T07:16:34.000 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-04-22T07:16:34.001 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-04-22T07:16:34.002 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-04-22T07:16:34.003 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-04-22T07:16:34.004 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-04-22T07:16:34.005 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-04-22T07:16:34.006 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-04-22T07:16:34.007 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-04-22T07:16:34.008 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-04-22T07:16:34.009 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-04-22T07:16:34.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-04-22T07:16:34.011 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-04-22T07:16:34.012 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-04-22T07:16:34.013 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-04-22T07:16:34.014 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-04-22T07:16:34.015 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-04-22T07:16:34.016 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-04-22T07:16:34.017 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-04-22T07:16:34.018 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-04-22T07:16:34.019 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-04-22T07:16:34.019 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-04-22T07:16:34.020 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-04-22T07:16:34.022 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-04-22T07:16:34.022 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-04-22T07:16:34.023 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-04-22T07:16:34.024 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-04-22T07:16:34.025 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-04-22T07:16:34.026 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-04-22T07:16:34.026 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-04-22T07:16:34.028 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-04-22T07:16:34.028 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-04-22T07:16:34.030 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-04-22T07:16:34.031 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-04-22T07:16:34.032 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-04-22T07:16:34.033 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-04-22T07:16:34.034 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-04-22T07:16:34.035 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-04-22T07:16:34.036 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-04-22T07:16:34.037 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-04-22T07:16:34.038 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-04-22T07:16:34.039 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-04-22T07:16:34.040 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-04-22T07:16:34.041 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-04-22T07:16:34.041 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-04-22T07:16:34.042 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-04-22T07:16:34.044 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-04-22T07:16:34.044 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-04-22T07:16:34.045 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-04-22T07:16:34.046 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-04-22T07:16:34.047 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-04-22T07:16:34.048 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-04-22T07:16:34.049 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-04-22T07:16:34.645 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-04-22T07:16:34.646 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-04-22T07:16:34.647 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-04-22T07:16:34.648 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-04-22T07:16:34.649 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-04-22T07:16:34.650 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-04-22T07:16:34.651 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-04-22T07:16:34.652 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-04-22T07:16:34.653 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-04-22T07:16:34.654 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-04-22T07:16:34.655 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-04-22T07:16:34.657 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-04-22T07:16:34.658 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-04-22T07:16:34.659 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-04-22T07:16:34.660 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-04-22T07:16:34.661 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-04-22T07:16:34.662 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-04-22T07:16:34.664 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-04-22T07:16:34.665 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-04-22T07:16:34.666 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-04-22T07:16:34.667 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-04-22T07:16:34.668 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-04-22T07:16:34.669 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-04-22T07:16:34.670 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-04-22T07:16:34.671 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-04-22T07:16:34.672 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-04-22T07:16:34.673 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-04-22T07:16:34.674 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-04-22T07:16:34.675 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-04-22T07:16:34.676 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-04-22T07:16:34.677 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-04-22T07:16:34.680 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-04-22T07:16:34.681 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-04-22T07:16:34.682 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-04-22T07:16:34.683 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/ 2024-04-22T07:16:34.684 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-04-22T07:16:34.685 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-04-22T07:16:34.686 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-04-22T07:16:34.687 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-04-22T07:16:34.688 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-04-22T07:16:34.689 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-04-22T07:16:34.689 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-04-22T07:16:34.691 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-04-22T07:16:34.692 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-04-22T07:16:34.693 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/EDID/hex 2024-04-22T07:16:34.694 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/HOWTO 2024-04-22T07:16:34.695 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-04-22T07:16:34.696 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-04-22T07:16:34.697 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-04-22T07:16:34.698 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-04-22T07:16:34.699 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-04-22T07:16:34.701 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/Makefile 2024-04-22T07:16:34.702 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-04-22T07:16:34.703 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/ 2024-04-22T07:16:34.704 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-04-22T07:16:34.705 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-04-22T07:16:34.706 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-04-22T07:16:34.707 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-04-22T07:16:34.708 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-04-22T07:16:34.709 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-04-22T07:16:34.710 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-04-22T07:16:34.711 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/ 2024-04-22T07:16:34.712 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-04-22T07:16:34.713 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-04-22T07:16:34.714 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-04-22T07:16:34.715 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-04-22T07:16:34.716 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-04-22T07:16:34.717 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-04-22T07:16:34.718 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-04-22T07:16:34.719 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-04-22T07:16:34.720 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-04-22T07:16:34.721 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-04-22T07:16:34.722 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-04-22T07:16:34.723 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-04-22T07:16:34.724 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-04-22T07:16:34.725 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-04-22T07:16:34.726 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-04-22T07:16:34.727 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-04-22T07:16:34.728 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-04-22T07:16:34.729 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-04-22T07:16:34.730 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/SAK.txt 2024-04-22T07:16:34.731 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/SM501.txt 2024-04-22T07:16:34.732 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-04-22T07:16:34.733 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-04-22T07:16:34.734 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-04-22T07:16:34.735 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-04-22T07:16:34.736 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-04-22T07:16:34.737 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/ 2024-04-22T07:16:34.738 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-04-22T07:16:34.739 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-04-22T07:16:34.740 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-04-22T07:16:34.741 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-04-22T07:16:34.742 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-04-22T07:16:34.743 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-04-22T07:16:34.744 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-04-22T07:16:34.745 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/ 2024-04-22T07:16:34.746 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-04-22T07:16:34.747 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-04-22T07:16:34.748 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-04-22T07:16:34.749 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-04-22T07:16:34.750 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-04-22T07:16:34.751 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-04-22T07:16:34.752 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-04-22T07:16:34.753 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-04-22T07:16:34.754 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-04-22T07:16:34.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-04-22T07:16:34.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-04-22T07:16:34.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-04-22T07:16:34.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-04-22T07:16:34.759 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/ 2024-04-22T07:16:34.760 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-04-22T07:16:34.761 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-04-22T07:16:34.762 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-04-22T07:16:34.763 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-04-22T07:16:34.764 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-04-22T07:16:34.765 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-04-22T07:16:34.766 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-04-22T07:16:34.767 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/ 2024-04-22T07:16:34.768 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-04-22T07:16:34.769 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-04-22T07:16:34.770 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-04-22T07:16:34.770 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Booting 2024-04-22T07:16:34.771 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-04-22T07:16:34.772 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-04-22T07:16:34.773 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-04-22T07:16:34.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-04-22T07:16:34.775 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-04-22T07:16:34.776 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-04-22T07:16:34.777 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-04-22T07:16:34.778 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-04-22T07:16:34.778 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-04-22T07:16:34.779 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-04-22T07:16:34.781 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Porting 2024-04-22T07:16:34.782 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/README 2024-04-22T07:16:34.783 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-04-22T07:16:34.784 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-04-22T07:16:34.784 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-04-22T07:16:34.785 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-04-22T07:16:34.786 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-04-22T07:16:34.787 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-04-22T07:16:34.788 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-04-22T07:16:34.789 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-04-22T07:16:34.790 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-04-22T07:16:34.791 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-04-22T07:16:34.791 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-04-22T07:16:34.793 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-04-22T07:16:34.793 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-04-22T07:16:34.794 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-04-22T07:16:34.796 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-04-22T07:16:34.796 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-04-22T07:16:34.797 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-04-22T07:16:34.798 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-04-22T07:16:34.799 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-04-22T07:16:34.800 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-04-22T07:16:34.801 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-04-22T07:16:34.802 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-04-22T07:16:34.803 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-04-22T07:16:34.804 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-04-22T07:16:34.805 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-04-22T07:16:34.806 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-04-22T07:16:34.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-04-22T07:16:34.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-04-22T07:16:34.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-04-22T07:16:34.810 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-04-22T07:16:34.810 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-04-22T07:16:34.811 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-04-22T07:16:34.812 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-04-22T07:16:34.813 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-04-22T07:16:34.814 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-04-22T07:16:34.815 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-04-22T07:16:34.816 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-04-22T07:16:34.817 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-04-22T07:16:34.818 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-04-22T07:16:34.819 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-04-22T07:16:34.820 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-04-22T07:16:34.820 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-04-22T07:16:34.822 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-04-22T07:16:34.823 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/Setup 2024-04-22T07:16:34.823 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-04-22T07:16:34.824 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-04-22T07:16:34.825 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-04-22T07:16:34.826 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-04-22T07:16:34.827 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-04-22T07:16:34.828 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-04-22T07:16:34.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-04-22T07:16:34.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-04-22T07:16:34.830 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-04-22T07:16:34.831 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-04-22T07:16:34.832 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-04-22T07:16:34.833 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-04-22T07:16:34.834 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-04-22T07:16:34.834 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-04-22T07:16:34.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-04-22T07:16:34.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-04-22T07:16:34.837 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-04-22T07:16:34.838 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-04-22T07:16:34.839 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-04-22T07:16:34.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-04-22T07:16:34.841 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-04-22T07:16:34.842 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-04-22T07:16:34.843 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-04-22T07:16:34.843 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-04-22T07:16:34.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-04-22T07:16:34.845 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-04-22T07:16:34.846 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-04-22T07:16:34.848 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-04-22T07:16:34.848 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-04-22T07:16:34.849 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-04-22T07:16:34.850 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm64/ 2024-04-22T07:16:34.851 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-04-22T07:16:34.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-04-22T07:16:34.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-04-22T07:16:34.854 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-04-22T07:16:34.855 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-04-22T07:16:34.856 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-04-22T07:16:34.857 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-04-22T07:16:34.858 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-04-22T07:16:34.859 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-04-22T07:16:34.860 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-04-22T07:16:34.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-04-22T07:16:34.862 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-04-22T07:16:34.862 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/backlight/ 2024-04-22T07:16:34.863 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-04-22T07:16:34.864 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-04-22T07:16:34.865 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-04-22T07:16:34.867 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/bcache.txt 2024-04-22T07:16:34.868 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-04-22T07:16:34.869 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blackfin/ 2024-04-22T07:16:34.870 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-04-22T07:16:34.871 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-04-22T07:16:34.872 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-04-22T07:16:34.873 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-04-22T07:16:34.874 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-04-22T07:16:34.875 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/ 2024-04-22T07:16:34.876 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-04-22T07:16:34.877 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-04-22T07:16:34.878 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-04-22T07:16:34.879 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-04-22T07:16:34.880 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-04-22T07:16:34.881 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-04-22T07:16:34.882 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-04-22T07:16:34.883 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-04-22T07:16:34.883 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-04-22T07:16:34.885 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-04-22T07:16:34.886 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-04-22T07:16:34.886 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/request.txt 2024-04-22T07:16:34.888 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-04-22T07:16:34.888 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-04-22T07:16:34.889 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-04-22T07:16:34.890 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/ 2024-04-22T07:16:34.891 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-04-22T07:16:34.892 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-04-22T07:16:34.893 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-04-22T07:16:34.894 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-04-22T07:16:34.895 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-04-22T07:16:34.896 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-04-22T07:16:34.897 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-04-22T07:16:34.898 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-04-22T07:16:34.899 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-04-22T07:16:34.900 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-04-22T07:16:34.901 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-04-22T07:16:34.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-04-22T07:16:34.903 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-04-22T07:16:34.904 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-04-22T07:16:34.905 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-04-22T07:16:34.906 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-04-22T07:16:34.907 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-04-22T07:16:34.908 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-04-22T07:16:34.909 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-04-22T07:16:34.910 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-04-22T07:16:34.911 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-04-22T07:16:34.912 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-04-22T07:16:34.913 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-04-22T07:16:34.914 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-04-22T07:16:34.915 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-04-22T07:16:34.916 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-04-22T07:16:34.917 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cdrom/ 2024-04-22T07:16:34.918 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-04-22T07:16:34.919 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-04-22T07:16:34.920 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-04-22T07:16:34.922 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-04-22T07:16:34.922 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-04-22T07:16:34.924 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/ 2024-04-22T07:16:34.925 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-04-22T07:16:34.926 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-04-22T07:16:34.927 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-04-22T07:16:34.928 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-04-22T07:16:34.929 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-04-22T07:16:34.930 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-04-22T07:16:34.931 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-04-22T07:16:34.932 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-04-22T07:16:34.933 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-04-22T07:16:34.934 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-04-22T07:16:34.935 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-04-22T07:16:34.936 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-04-22T07:16:34.937 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-04-22T07:16:34.938 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-04-22T07:16:34.939 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/clk.txt 2024-04-22T07:16:34.940 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-04-22T07:16:34.941 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/connector/ 2024-04-22T07:16:34.942 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-04-22T07:16:34.943 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-04-22T07:16:34.944 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-04-22T07:16:34.945 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-04-22T07:16:34.946 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-04-22T07:16:34.947 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/console/ 2024-04-22T07:16:34.948 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/console/console.txt 2024-04-22T07:16:34.949 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-04-22T07:16:34.950 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-04-22T07:16:34.951 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-04-22T07:16:34.952 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-04-22T07:16:34.953 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-04-22T07:16:34.954 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-04-22T07:16:34.955 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-04-22T07:16:34.956 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-04-22T07:16:34.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-04-22T07:16:34.958 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-04-22T07:16:34.959 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-04-22T07:16:34.960 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-04-22T07:16:34.961 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-04-22T07:16:34.962 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-04-22T07:16:34.963 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-04-22T07:16:34.964 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-04-22T07:16:34.965 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-04-22T07:16:34.966 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-04-22T07:16:34.967 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-04-22T07:16:34.968 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-04-22T07:16:34.969 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crc32.txt 2024-04-22T07:16:34.970 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cris/ 2024-04-22T07:16:34.971 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/cris/README 2024-04-22T07:16:34.972 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crypto/ 2024-04-22T07:16:34.973 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-04-22T07:16:34.974 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-04-22T07:16:34.975 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-04-22T07:16:34.976 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-04-22T07:16:34.977 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-04-22T07:16:34.978 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-04-22T07:16:34.979 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-04-22T07:16:34.980 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-04-22T07:16:34.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-04-22T07:16:34.982 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/ 2024-04-22T07:16:34.983 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-04-22T07:16:34.983 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-04-22T07:16:34.984 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-04-22T07:16:34.985 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-04-22T07:16:34.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-04-22T07:16:34.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-04-22T07:16:34.988 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-04-22T07:16:34.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-04-22T07:16:34.991 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-04-22T07:16:34.992 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-04-22T07:16:34.992 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-04-22T07:16:34.993 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-04-22T07:16:34.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-04-22T07:16:34.995 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-04-22T07:16:34.996 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-04-22T07:16:34.997 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-04-22T07:16:34.998 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-04-22T07:16:34.999 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-04-22T07:16:35.000 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-04-22T07:16:35.001 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-04-22T07:16:35.002 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-04-22T07:16:35.003 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-04-22T07:16:35.004 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-04-22T07:16:35.005 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-04-22T07:16:35.006 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-04-22T07:16:35.009 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-04-22T07:16:35.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-04-22T07:16:35.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-04-22T07:16:35.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-04-22T07:16:35.011 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-04-22T07:16:35.012 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-04-22T07:16:35.013 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devices.txt 2024-04-22T07:16:35.014 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/ 2024-04-22T07:16:35.016 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-04-22T07:16:35.016 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-04-22T07:16:35.017 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-04-22T07:16:35.018 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-04-22T07:16:35.019 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-04-22T07:16:35.021 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-04-22T07:16:35.021 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-04-22T07:16:35.022 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-04-22T07:16:35.022 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-04-22T07:16:35.024 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-04-22T07:16:35.025 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-04-22T07:16:35.026 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-04-22T07:16:35.027 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-04-22T07:16:35.028 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-04-22T07:16:35.029 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-04-22T07:16:35.030 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-04-22T07:16:35.031 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-04-22T07:16:35.032 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-04-22T07:16:35.033 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-04-22T07:16:35.034 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-04-22T07:16:35.035 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-04-22T07:16:35.035 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-04-22T07:16:35.036 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-04-22T07:16:35.037 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-04-22T07:16:35.038 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-04-22T07:16:35.039 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-04-22T07:16:35.040 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-04-22T07:16:35.041 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-04-22T07:16:35.042 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-04-22T07:16:35.043 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-04-22T07:16:35.044 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-04-22T07:16:35.045 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-04-22T07:16:35.046 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-04-22T07:16:35.047 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-04-22T07:16:35.048 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-04-22T07:16:35.049 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-04-22T07:16:35.050 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-04-22T07:16:35.051 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-04-22T07:16:35.052 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-04-22T07:16:35.052 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-04-22T07:16:35.053 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-04-22T07:16:35.055 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-04-22T07:16:35.056 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-04-22T07:16:35.057 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-04-22T07:16:35.058 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-04-22T07:16:35.058 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-04-22T07:16:35.059 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-04-22T07:16:35.060 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-04-22T07:16:35.061 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-04-22T07:16:35.062 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-04-22T07:16:35.063 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-04-22T07:16:35.064 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-04-22T07:16:35.065 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-04-22T07:16:35.066 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-04-22T07:16:35.067 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-04-22T07:16:35.068 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-04-22T07:16:35.069 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-04-22T07:16:35.070 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-04-22T07:16:35.070 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-04-22T07:16:35.071 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-04-22T07:16:35.072 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-04-22T07:16:35.073 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-04-22T07:16:35.074 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-04-22T07:16:35.075 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-04-22T07:16:35.076 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-04-22T07:16:35.077 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-04-22T07:16:35.077 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-04-22T07:16:35.078 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-04-22T07:16:35.079 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-04-22T07:16:35.080 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-04-22T07:16:35.081 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-04-22T07:16:35.082 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-04-22T07:16:35.084 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-04-22T07:16:35.084 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-04-22T07:16:35.085 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-04-22T07:16:35.086 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-04-22T07:16:35.087 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-04-22T07:16:35.088 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-04-22T07:16:35.089 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-04-22T07:16:35.090 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-04-22T07:16:35.090 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-04-22T07:16:35.092 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-04-22T07:16:35.093 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-04-22T07:16:35.093 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-04-22T07:16:35.094 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-04-22T07:16:35.095 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-04-22T07:16:35.096 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-04-22T07:16:35.097 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-04-22T07:16:35.098 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-04-22T07:16:35.098 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-04-22T07:16:35.099 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-04-22T07:16:35.100 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-04-22T07:16:35.102 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-04-22T07:16:35.103 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-04-22T07:16:35.104 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-04-22T07:16:35.104 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-04-22T07:16:35.106 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-04-22T07:16:35.106 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-04-22T07:16:35.107 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-04-22T07:16:35.108 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-04-22T07:16:35.110 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-04-22T07:16:35.111 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-04-22T07:16:35.112 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-04-22T07:16:35.113 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-04-22T07:16:35.114 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-04-22T07:16:35.115 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-04-22T07:16:35.116 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-04-22T07:16:35.117 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-04-22T07:16:35.118 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-04-22T07:16:35.119 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-04-22T07:16:35.120 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-04-22T07:16:35.121 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-04-22T07:16:35.122 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-04-22T07:16:35.123 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-04-22T07:16:35.124 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-04-22T07:16:35.125 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-04-22T07:16:35.126 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-04-22T07:16:35.127 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-04-22T07:16:35.128 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-04-22T07:16:35.129 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-04-22T07:16:35.131 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-04-22T07:16:35.131 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-04-22T07:16:35.132 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-04-22T07:16:35.133 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-04-22T07:16:35.134 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-04-22T07:16:35.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-04-22T07:16:35.136 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-04-22T07:16:35.137 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-04-22T07:16:35.138 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-04-22T07:16:35.139 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-04-22T07:16:35.140 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-04-22T07:16:35.141 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-04-22T07:16:35.142 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-04-22T07:16:35.143 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-04-22T07:16:35.144 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-04-22T07:16:35.145 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-04-22T07:16:35.146 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-04-22T07:16:35.147 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-04-22T07:16:35.148 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-04-22T07:16:35.149 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-04-22T07:16:35.150 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-04-22T07:16:35.151 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-04-22T07:16:35.152 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-04-22T07:16:35.154 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-04-22T07:16:35.155 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-04-22T07:16:35.157 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-04-22T07:16:35.159 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-04-22T07:16:35.160 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-04-22T07:16:35.162 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-04-22T07:16:35.163 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-04-22T07:16:35.165 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-04-22T07:16:35.167 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-04-22T07:16:35.168 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-04-22T07:16:35.170 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-04-22T07:16:35.171 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-04-22T07:16:35.173 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-04-22T07:16:35.174 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-04-22T07:16:35.176 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-04-22T07:16:35.434 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-04-22T07:16:35.435 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-04-22T07:16:35.437 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-04-22T07:16:35.439 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-04-22T07:16:35.441 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-04-22T07:16:35.442 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-04-22T07:16:35.444 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-04-22T07:16:35.445 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-04-22T07:16:35.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-04-22T07:16:35.448 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-04-22T07:16:35.450 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-04-22T07:16:35.451 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-04-22T07:16:35.453 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-04-22T07:16:35.454 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-04-22T07:16:35.456 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-04-22T07:16:35.458 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-04-22T07:16:35.459 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-04-22T07:16:35.461 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-04-22T07:16:35.462 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-04-22T07:16:35.463 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-04-22T07:16:35.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-04-22T07:16:35.469 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-04-22T07:16:35.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-04-22T07:16:35.471 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-04-22T07:16:35.473 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-04-22T07:16:35.475 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-04-22T07:16:35.477 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-04-22T07:16:35.479 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-04-22T07:16:35.480 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-04-22T07:16:35.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-04-22T07:16:35.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-04-22T07:16:35.493 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-04-22T07:16:35.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-04-22T07:16:35.499 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-04-22T07:16:35.502 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-04-22T07:16:35.505 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-04-22T07:16:35.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-04-22T07:16:35.513 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-04-22T07:16:35.517 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-04-22T07:16:35.520 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-04-22T07:16:35.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-04-22T07:16:35.528 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-04-22T07:16:35.529 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-04-22T07:16:35.535 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-04-22T07:16:35.537 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-04-22T07:16:35.538 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-04-22T07:16:35.539 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-04-22T07:16:35.544 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-04-22T07:16:35.549 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-04-22T07:16:35.550 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-04-22T07:16:35.553 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-04-22T07:16:35.555 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-04-22T07:16:35.557 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-04-22T07:16:35.560 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-04-22T07:16:35.562 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-04-22T07:16:35.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:35 smithi133 ceph-mon[31198]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-22T07:16:35.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:35 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:16:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:35 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:35 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:16:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:35 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:35 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:16:35.564 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-04-22T07:16:35.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:35 smithi154 ceph-mon[31862]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-22T07:16:35.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:35 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:16:35.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:35 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:35.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:35 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:16:35.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:35 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:35.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:35 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:16:35.567 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-04-22T07:16:35.574 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-04-22T07:16:35.575 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-04-22T07:16:35.577 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-04-22T07:16:35.579 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-04-22T07:16:35.581 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-04-22T07:16:35.583 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-04-22T07:16:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:35 smithi113 ceph-mon[27933]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-22T07:16:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:35 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:16:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:35 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:35 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:16:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:35 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:16:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:35 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:16:35.585 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-04-22T07:16:35.586 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-04-22T07:16:35.589 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-04-22T07:16:35.591 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-04-22T07:16:35.592 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-04-22T07:16:35.595 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-04-22T07:16:35.597 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-04-22T07:16:35.598 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-04-22T07:16:35.600 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-04-22T07:16:35.603 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-04-22T07:16:35.605 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-04-22T07:16:35.607 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-04-22T07:16:35.621 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-04-22T07:16:35.623 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-04-22T07:16:36.380 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-04-22T07:16:36.381 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-04-22T07:16:36.382 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-04-22T07:16:36.383 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-04-22T07:16:36.384 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-04-22T07:16:36.385 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-04-22T07:16:36.386 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-04-22T07:16:36.387 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-04-22T07:16:36.388 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-04-22T07:16:36.389 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-04-22T07:16:36.391 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-04-22T07:16:36.392 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-04-22T07:16:36.393 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-04-22T07:16:36.394 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-04-22T07:16:36.395 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-04-22T07:16:36.396 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-04-22T07:16:36.397 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-04-22T07:16:36.398 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-04-22T07:16:36.399 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-04-22T07:16:36.400 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-04-22T07:16:36.401 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-04-22T07:16:36.402 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-04-22T07:16:36.403 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-04-22T07:16:36.404 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-04-22T07:16:36.405 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-04-22T07:16:36.406 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-04-22T07:16:36.407 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-04-22T07:16:36.408 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-04-22T07:16:36.409 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-04-22T07:16:36.410 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-04-22T07:16:36.411 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-04-22T07:16:36.412 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-04-22T07:16:36.414 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-04-22T07:16:36.415 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-04-22T07:16:36.416 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-04-22T07:16:36.417 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-04-22T07:16:36.418 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-04-22T07:16:36.419 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-04-22T07:16:36.420 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-04-22T07:16:36.421 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-04-22T07:16:36.422 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-04-22T07:16:36.423 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-04-22T07:16:36.424 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-04-22T07:16:36.425 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-04-22T07:16:36.426 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-04-22T07:16:36.427 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-04-22T07:16:36.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-04-22T07:16:36.429 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-04-22T07:16:36.430 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-04-22T07:16:36.431 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-04-22T07:16:36.432 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-04-22T07:16:36.433 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-04-22T07:16:36.434 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-04-22T07:16:36.435 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-04-22T07:16:36.436 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-04-22T07:16:36.437 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-04-22T07:16:36.438 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-04-22T07:16:36.439 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-04-22T07:16:36.440 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-04-22T07:16:36.441 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-04-22T07:16:36.442 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-04-22T07:16:36.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-04-22T07:16:36.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-04-22T07:16:36.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-04-22T07:16:36.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-04-22T07:16:36.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-04-22T07:16:36.448 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-04-22T07:16:36.449 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-04-22T07:16:36.450 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-04-22T07:16:36.451 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-04-22T07:16:36.452 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-04-22T07:16:36.453 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-04-22T07:16:36.454 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-04-22T07:16:36.455 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-04-22T07:16:36.456 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-04-22T07:16:36.457 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-04-22T07:16:36.458 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-04-22T07:16:36.459 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-04-22T07:16:36.460 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-04-22T07:16:36.461 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-04-22T07:16:36.462 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-04-22T07:16:36.463 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-04-22T07:16:36.464 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-04-22T07:16:36.465 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-04-22T07:16:36.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-04-22T07:16:36.467 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-04-22T07:16:36.468 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-04-22T07:16:36.469 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-04-22T07:16:36.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-04-22T07:16:36.471 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-04-22T07:16:36.472 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-04-22T07:16:36.473 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-04-22T07:16:36.474 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-04-22T07:16:36.475 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-04-22T07:16:36.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-04-22T07:16:36.477 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-04-22T07:16:36.478 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-04-22T07:16:36.479 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-04-22T07:16:36.480 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-04-22T07:16:36.481 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-04-22T07:16:36.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-04-22T07:16:36.483 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-04-22T07:16:36.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-04-22T07:16:36.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-04-22T07:16:36.485 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-04-22T07:16:36.486 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-04-22T07:16:36.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-04-22T07:16:36.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-04-22T07:16:36.489 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-04-22T07:16:36.490 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-04-22T07:16:36.491 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-04-22T07:16:36.492 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-04-22T07:16:36.493 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-04-22T07:16:36.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-04-22T07:16:36.495 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-04-22T07:16:36.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-04-22T07:16:36.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-04-22T07:16:36.498 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-04-22T07:16:36.500 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-04-22T07:16:36.500 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-04-22T07:16:36.500 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-04-22T07:16:36.501 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-04-22T07:16:36.502 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-04-22T07:16:36.503 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-04-22T07:16:36.504 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-04-22T07:16:36.505 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-04-22T07:16:36.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-04-22T07:16:36.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-04-22T07:16:36.507 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-04-22T07:16:36.508 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-04-22T07:16:36.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-04-22T07:16:36.510 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-04-22T07:16:36.511 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-04-22T07:16:36.511 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-04-22T07:16:36.512 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-04-22T07:16:36.513 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-04-22T07:16:36.514 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-04-22T07:16:36.515 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-04-22T07:16:36.515 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-04-22T07:16:36.516 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-04-22T07:16:36.517 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-04-22T07:16:36.518 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-04-22T07:16:36.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-04-22T07:16:36.520 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-04-22T07:16:36.521 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-04-22T07:16:36.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-04-22T07:16:36.523 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-04-22T07:16:36.524 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-04-22T07:16:36.524 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-04-22T07:16:36.525 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-04-22T07:16:36.526 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-04-22T07:16:36.527 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-04-22T07:16:36.528 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-04-22T07:16:36.529 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-04-22T07:16:36.530 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-04-22T07:16:36.531 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-04-22T07:16:36.532 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-04-22T07:16:36.533 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-04-22T07:16:36.534 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-04-22T07:16:36.535 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-04-22T07:16:36.536 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-04-22T07:16:36.536 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-04-22T07:16:36.538 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-04-22T07:16:36.539 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-04-22T07:16:36.540 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-04-22T07:16:36.541 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-04-22T07:16:36.542 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-04-22T07:16:36.543 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-04-22T07:16:36.544 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-04-22T07:16:36.545 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-04-22T07:16:36.546 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-04-22T07:16:36.547 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-04-22T07:16:36.548 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-04-22T07:16:36.549 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-04-22T07:16:36.550 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-04-22T07:16:36.551 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-04-22T07:16:36.552 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-04-22T07:16:36.553 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-04-22T07:16:36.554 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-04-22T07:16:36.555 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-04-22T07:16:36.556 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-04-22T07:16:36.557 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-04-22T07:16:36.558 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-04-22T07:16:36.559 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-04-22T07:16:36.560 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-04-22T07:16:36.561 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-04-22T07:16:36.562 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-04-22T07:16:36.563 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-04-22T07:16:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:36 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:36.564 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-04-22T07:16:36.565 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-04-22T07:16:36.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:36 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:36.566 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-04-22T07:16:36.567 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-04-22T07:16:36.568 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-04-22T07:16:36.569 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-04-22T07:16:36.570 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-04-22T07:16:36.571 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-04-22T07:16:36.572 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-04-22T07:16:36.573 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-04-22T07:16:36.574 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-04-22T07:16:36.575 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-04-22T07:16:36.576 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-04-22T07:16:36.577 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-04-22T07:16:36.578 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-04-22T07:16:36.579 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-04-22T07:16:36.580 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-04-22T07:16:36.581 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-04-22T07:16:36.582 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-04-22T07:16:36.583 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-04-22T07:16:36.584 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-04-22T07:16:36.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:36 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:36.585 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-04-22T07:16:36.585 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-04-22T07:16:36.586 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-04-22T07:16:36.587 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-04-22T07:16:36.588 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-04-22T07:16:36.589 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-04-22T07:16:36.590 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-04-22T07:16:36.591 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-04-22T07:16:36.592 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-04-22T07:16:36.593 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-04-22T07:16:36.594 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-04-22T07:16:36.595 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-04-22T07:16:36.596 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-04-22T07:16:36.597 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-04-22T07:16:36.598 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-04-22T07:16:36.599 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-04-22T07:16:36.600 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-04-22T07:16:36.601 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-04-22T07:16:36.602 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-04-22T07:16:36.603 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-04-22T07:16:36.604 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-04-22T07:16:36.605 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-04-22T07:16:36.606 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-04-22T07:16:36.607 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-04-22T07:16:36.608 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-04-22T07:16:36.609 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-04-22T07:16:36.610 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-04-22T07:16:36.611 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-04-22T07:16:36.612 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-04-22T07:16:36.613 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-04-22T07:16:36.614 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-04-22T07:16:36.615 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-04-22T07:16:36.616 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-04-22T07:16:36.617 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-04-22T07:16:36.618 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-04-22T07:16:36.619 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-04-22T07:16:36.620 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-04-22T07:16:36.621 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-04-22T07:16:36.622 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-04-22T07:16:36.623 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-04-22T07:16:36.624 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-04-22T07:16:36.625 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-04-22T07:16:36.626 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-04-22T07:16:36.627 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-04-22T07:16:36.628 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-04-22T07:16:36.629 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-04-22T07:16:36.630 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-04-22T07:16:36.631 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-04-22T07:16:36.632 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-04-22T07:16:36.633 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-04-22T07:16:36.634 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-04-22T07:16:36.635 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-04-22T07:16:36.636 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-04-22T07:16:36.637 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-04-22T07:16:36.638 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-04-22T07:16:36.639 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-04-22T07:16:36.640 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-04-22T07:16:36.641 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-04-22T07:16:36.642 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-04-22T07:16:36.643 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-04-22T07:16:36.644 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-04-22T07:16:36.645 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-04-22T07:16:36.646 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-04-22T07:16:36.647 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-04-22T07:16:36.648 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-04-22T07:16:36.649 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-04-22T07:16:36.650 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-04-22T07:16:36.650 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-04-22T07:16:36.651 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-04-22T07:16:36.652 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-04-22T07:16:36.653 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-04-22T07:16:36.654 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-04-22T07:16:36.655 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-04-22T07:16:36.656 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-04-22T07:16:36.657 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-04-22T07:16:36.658 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-04-22T07:16:36.659 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-04-22T07:16:36.660 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-04-22T07:16:36.661 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-04-22T07:16:36.662 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-04-22T07:16:36.662 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-04-22T07:16:36.663 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-04-22T07:16:36.664 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-04-22T07:16:36.665 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-04-22T07:16:36.666 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-04-22T07:16:36.667 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-04-22T07:16:36.669 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-04-22T07:16:36.670 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-04-22T07:16:36.671 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-04-22T07:16:36.672 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-04-22T07:16:36.673 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-04-22T07:16:36.674 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-04-22T07:16:36.675 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-04-22T07:16:36.676 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-04-22T07:16:36.677 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-04-22T07:16:36.678 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-04-22T07:16:36.679 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-04-22T07:16:36.680 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-04-22T07:16:36.681 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-04-22T07:16:36.682 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-04-22T07:16:36.683 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-04-22T07:16:36.684 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-04-22T07:16:36.685 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-04-22T07:16:36.686 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-04-22T07:16:36.687 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-04-22T07:16:36.688 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-04-22T07:16:36.689 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-04-22T07:16:36.690 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-04-22T07:16:36.691 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-04-22T07:16:36.692 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-04-22T07:16:36.693 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-04-22T07:16:36.693 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-04-22T07:16:36.694 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-04-22T07:16:36.695 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-04-22T07:16:36.696 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-04-22T07:16:36.697 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-04-22T07:16:36.698 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-04-22T07:16:36.699 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-04-22T07:16:36.700 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-04-22T07:16:36.701 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-04-22T07:16:36.702 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-04-22T07:16:36.703 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-04-22T07:16:36.704 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-04-22T07:16:36.705 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-04-22T07:16:36.706 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-04-22T07:16:36.707 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-04-22T07:16:36.708 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-04-22T07:16:36.709 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-04-22T07:16:36.710 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-04-22T07:16:36.711 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-04-22T07:16:36.712 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-04-22T07:16:36.713 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-04-22T07:16:36.714 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-04-22T07:16:36.715 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-04-22T07:16:36.716 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-04-22T07:16:36.717 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-04-22T07:16:36.718 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-04-22T07:16:36.719 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-04-22T07:16:36.720 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-04-22T07:16:36.721 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-04-22T07:16:36.722 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-04-22T07:16:36.723 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-04-22T07:16:36.724 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-04-22T07:16:36.724 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-04-22T07:16:36.725 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-04-22T07:16:36.726 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-04-22T07:16:36.727 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-04-22T07:16:36.728 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-04-22T07:16:36.729 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-04-22T07:16:36.730 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-04-22T07:16:36.732 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-04-22T07:16:36.733 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-04-22T07:16:36.734 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-04-22T07:16:36.735 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-04-22T07:16:36.736 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-04-22T07:16:36.737 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-04-22T07:16:36.738 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-04-22T07:16:36.739 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-04-22T07:16:36.740 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-04-22T07:16:36.741 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-04-22T07:16:36.742 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-04-22T07:16:36.743 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-04-22T07:16:36.744 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-04-22T07:16:36.745 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-04-22T07:16:36.746 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-04-22T07:16:36.747 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-04-22T07:16:36.748 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-04-22T07:16:36.749 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-04-22T07:16:36.750 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-04-22T07:16:36.750 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-04-22T07:16:36.751 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-04-22T07:16:36.752 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-04-22T07:16:36.753 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-04-22T07:16:36.754 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-04-22T07:16:36.756 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-04-22T07:16:36.757 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-04-22T07:16:36.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-04-22T07:16:36.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-04-22T07:16:36.759 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-04-22T07:16:36.761 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-04-22T07:16:36.762 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-04-22T07:16:36.762 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-04-22T07:16:36.764 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-04-22T07:16:36.765 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-04-22T07:16:36.765 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-04-22T07:16:36.767 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-04-22T07:16:36.767 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-04-22T07:16:36.769 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-04-22T07:16:36.770 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-04-22T07:16:36.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-04-22T07:16:36.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-04-22T07:16:36.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-04-22T07:16:36.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-04-22T07:16:36.775 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-04-22T07:16:36.776 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-04-22T07:16:36.777 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-04-22T07:16:36.778 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-04-22T07:16:36.779 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-04-22T07:16:36.780 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-04-22T07:16:36.781 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-04-22T07:16:36.782 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-04-22T07:16:36.783 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-04-22T07:16:36.784 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-04-22T07:16:36.784 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-04-22T07:16:36.786 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-04-22T07:16:36.787 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-04-22T07:16:36.788 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-04-22T07:16:36.789 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-04-22T07:16:36.790 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-04-22T07:16:36.791 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-04-22T07:16:36.792 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-04-22T07:16:36.793 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-04-22T07:16:36.794 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-04-22T07:16:36.795 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-04-22T07:16:36.796 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-04-22T07:16:36.797 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-04-22T07:16:36.798 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-04-22T07:16:36.799 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-04-22T07:16:36.800 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-04-22T07:16:36.801 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-04-22T07:16:36.802 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-04-22T07:16:36.803 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-04-22T07:16:36.804 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-04-22T07:16:36.805 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-04-22T07:16:36.806 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-04-22T07:16:36.807 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-04-22T07:16:36.808 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-04-22T07:16:36.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-04-22T07:16:36.810 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-04-22T07:16:36.811 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-04-22T07:16:36.812 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-04-22T07:16:36.813 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-04-22T07:16:36.814 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-04-22T07:16:36.815 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-04-22T07:16:36.816 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-04-22T07:16:36.817 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-04-22T07:16:36.818 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-04-22T07:16:36.819 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-04-22T07:16:36.820 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-04-22T07:16:36.821 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-04-22T07:16:36.822 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-04-22T07:16:36.823 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-04-22T07:16:36.824 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-04-22T07:16:36.825 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-04-22T07:16:36.826 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-04-22T07:16:36.827 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-04-22T07:16:36.828 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-04-22T07:16:36.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-04-22T07:16:36.830 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-04-22T07:16:36.831 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-04-22T07:16:36.832 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-04-22T07:16:36.833 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-04-22T07:16:36.834 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-04-22T07:16:36.835 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-04-22T07:16:36.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-04-22T07:16:36.837 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-04-22T07:16:36.838 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-04-22T07:16:36.839 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-04-22T07:16:36.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-04-22T07:16:36.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-04-22T07:16:36.841 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-04-22T07:16:36.843 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-04-22T07:16:36.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-04-22T07:16:36.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-04-22T07:16:36.845 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-04-22T07:16:36.846 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-04-22T07:16:36.847 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-04-22T07:16:36.848 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-04-22T07:16:36.849 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-04-22T07:16:36.851 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-04-22T07:16:36.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-04-22T07:16:36.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-04-22T07:16:36.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-04-22T07:16:36.854 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-04-22T07:16:36.855 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-04-22T07:16:36.856 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-04-22T07:16:36.857 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-04-22T07:16:36.858 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-04-22T07:16:36.859 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-04-22T07:16:36.860 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-04-22T07:16:36.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-04-22T07:16:36.862 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-04-22T07:16:36.863 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-04-22T07:16:36.864 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-04-22T07:16:36.865 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-04-22T07:16:36.866 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-04-22T07:16:36.867 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-04-22T07:16:36.868 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-04-22T07:16:36.869 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-04-22T07:16:36.870 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-04-22T07:16:36.871 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-04-22T07:16:36.872 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-04-22T07:16:37.067 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-04-22T07:16:37.068 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-04-22T07:16:37.069 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-04-22T07:16:37.070 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-04-22T07:16:37.071 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-04-22T07:16:37.072 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-04-22T07:16:37.073 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-04-22T07:16:37.074 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-04-22T07:16:37.075 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-04-22T07:16:37.076 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-04-22T07:16:37.077 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-04-22T07:16:37.078 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-04-22T07:16:37.079 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-04-22T07:16:37.080 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-04-22T07:16:37.081 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-04-22T07:16:37.083 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-04-22T07:16:37.083 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-04-22T07:16:37.084 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-04-22T07:16:37.085 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-04-22T07:16:37.086 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-04-22T07:16:37.088 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-04-22T07:16:37.089 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-04-22T07:16:37.090 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-04-22T07:16:37.091 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-04-22T07:16:37.092 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-04-22T07:16:37.093 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-04-22T07:16:37.094 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-04-22T07:16:37.095 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-04-22T07:16:37.096 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-04-22T07:16:37.097 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-04-22T07:16:37.098 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-04-22T07:16:37.099 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-04-22T07:16:37.100 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-04-22T07:16:37.101 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-04-22T07:16:37.102 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-04-22T07:16:37.103 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-04-22T07:16:37.104 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-04-22T07:16:37.105 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-04-22T07:16:37.106 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-04-22T07:16:37.107 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-04-22T07:16:37.108 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-04-22T07:16:37.109 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-04-22T07:16:37.110 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-04-22T07:16:37.111 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-04-22T07:16:37.111 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-04-22T07:16:37.112 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-04-22T07:16:37.113 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-04-22T07:16:37.114 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-04-22T07:16:37.116 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-04-22T07:16:37.117 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-04-22T07:16:37.118 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-04-22T07:16:37.119 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-04-22T07:16:37.120 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-04-22T07:16:37.121 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-04-22T07:16:37.122 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-04-22T07:16:37.122 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-04-22T07:16:37.123 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-04-22T07:16:37.125 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-04-22T07:16:37.126 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-04-22T07:16:37.127 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-04-22T07:16:37.128 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-04-22T07:16:37.129 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-04-22T07:16:37.130 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-04-22T07:16:37.131 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-04-22T07:16:37.132 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-04-22T07:16:37.133 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-04-22T07:16:37.134 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-04-22T07:16:37.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-04-22T07:16:37.136 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-04-22T07:16:37.137 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-04-22T07:16:37.138 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-04-22T07:16:37.138 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-04-22T07:16:37.139 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-04-22T07:16:37.140 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-04-22T07:16:37.141 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-04-22T07:16:37.142 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-04-22T07:16:37.143 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-04-22T07:16:37.144 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-04-22T07:16:37.145 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-04-22T07:16:37.146 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-04-22T07:16:37.147 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-04-22T07:16:37.148 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-04-22T07:16:37.149 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-04-22T07:16:37.150 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-04-22T07:16:37.151 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-04-22T07:16:37.152 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-04-22T07:16:37.153 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-04-22T07:16:37.154 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-04-22T07:16:37.155 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-04-22T07:16:37.156 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-04-22T07:16:37.157 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-04-22T07:16:37.158 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-04-22T07:16:37.159 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-04-22T07:16:37.160 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-04-22T07:16:37.161 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-04-22T07:16:37.162 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-04-22T07:16:37.163 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-04-22T07:16:37.164 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-04-22T07:16:37.165 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-04-22T07:16:37.166 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-04-22T07:16:37.167 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-04-22T07:16:37.168 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-04-22T07:16:37.169 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-04-22T07:16:37.170 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-04-22T07:16:37.171 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-04-22T07:16:37.172 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-04-22T07:16:37.173 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-04-22T07:16:37.173 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-04-22T07:16:37.174 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-04-22T07:16:37.175 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-04-22T07:16:37.176 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-04-22T07:16:37.177 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-04-22T07:16:37.179 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-04-22T07:16:37.180 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-04-22T07:16:37.181 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-04-22T07:16:37.182 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-04-22T07:16:37.183 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-04-22T07:16:37.184 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-04-22T07:16:37.185 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-04-22T07:16:37.186 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-04-22T07:16:37.187 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-04-22T07:16:37.188 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-04-22T07:16:37.188 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-04-22T07:16:37.190 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-04-22T07:16:37.191 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-04-22T07:16:37.192 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-04-22T07:16:37.193 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-04-22T07:16:37.194 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-04-22T07:16:37.195 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-04-22T07:16:37.196 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-04-22T07:16:37.197 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-04-22T07:16:37.198 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-04-22T07:16:37.199 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-04-22T07:16:37.200 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-04-22T07:16:37.201 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-04-22T07:16:37.202 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-04-22T07:16:37.203 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-04-22T07:16:37.204 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-04-22T07:16:37.205 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-04-22T07:16:37.206 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-04-22T07:16:37.207 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-04-22T07:16:37.208 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-04-22T07:16:37.209 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-04-22T07:16:37.210 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-04-22T07:16:37.211 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-04-22T07:16:37.213 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-04-22T07:16:37.214 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-04-22T07:16:37.215 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-04-22T07:16:37.216 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-04-22T07:16:37.217 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-04-22T07:16:37.217 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-04-22T07:16:37.218 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-04-22T07:16:37.220 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-04-22T07:16:37.221 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-04-22T07:16:37.222 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-04-22T07:16:37.223 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-04-22T07:16:37.224 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-04-22T07:16:37.225 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-04-22T07:16:37.225 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-04-22T07:16:37.226 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-04-22T07:16:37.227 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-04-22T07:16:37.228 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-04-22T07:16:37.229 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-04-22T07:16:37.230 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-04-22T07:16:37.231 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-04-22T07:16:37.232 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-04-22T07:16:37.233 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-04-22T07:16:37.234 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-04-22T07:16:37.235 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-04-22T07:16:37.236 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-04-22T07:16:37.237 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-04-22T07:16:37.238 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-04-22T07:16:37.238 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-04-22T07:16:37.239 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-04-22T07:16:37.241 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-04-22T07:16:37.242 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-04-22T07:16:37.243 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-04-22T07:16:37.243 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-04-22T07:16:37.244 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-04-22T07:16:37.245 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-04-22T07:16:37.246 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-04-22T07:16:37.247 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-04-22T07:16:37.247 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-04-22T07:16:37.249 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-04-22T07:16:37.250 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-04-22T07:16:37.251 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-04-22T07:16:37.252 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-04-22T07:16:37.253 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-04-22T07:16:37.254 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-04-22T07:16:37.258 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-04-22T07:16:37.258 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-04-22T07:16:37.258 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-04-22T07:16:37.259 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-04-22T07:16:37.259 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-04-22T07:16:37.260 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-04-22T07:16:37.260 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-04-22T07:16:37.261 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-04-22T07:16:37.263 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-04-22T07:16:37.264 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-04-22T07:16:37.265 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-04-22T07:16:37.266 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-04-22T07:16:37.266 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-04-22T07:16:37.267 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-04-22T07:16:37.268 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-04-22T07:16:37.269 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-04-22T07:16:37.270 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-04-22T07:16:37.272 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-04-22T07:16:37.273 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-04-22T07:16:37.274 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-04-22T07:16:37.275 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-04-22T07:16:37.276 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-04-22T07:16:37.277 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-04-22T07:16:37.278 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-04-22T07:16:37.279 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-04-22T07:16:37.280 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-04-22T07:16:37.281 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-04-22T07:16:37.281 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-04-22T07:16:37.282 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-04-22T07:16:37.284 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-04-22T07:16:37.285 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-04-22T07:16:37.285 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-04-22T07:16:37.286 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-04-22T07:16:37.287 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-04-22T07:16:37.288 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-04-22T07:16:37.289 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-04-22T07:16:37.290 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-04-22T07:16:37.291 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-04-22T07:16:37.292 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-04-22T07:16:37.293 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-04-22T07:16:37.294 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-04-22T07:16:37.295 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-04-22T07:16:37.296 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-04-22T07:16:37.297 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-04-22T07:16:37.298 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-04-22T07:16:37.299 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-04-22T07:16:37.301 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-04-22T07:16:37.301 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-04-22T07:16:37.303 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-04-22T07:16:37.303 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-04-22T07:16:37.304 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-04-22T07:16:37.306 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-04-22T07:16:37.306 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-04-22T07:16:37.307 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-04-22T07:16:37.308 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-04-22T07:16:37.309 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-04-22T07:16:37.310 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-04-22T07:16:37.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-04-22T07:16:37.312 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-04-22T07:16:37.313 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-04-22T07:16:37.314 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-04-22T07:16:37.315 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-04-22T07:16:37.316 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-04-22T07:16:37.317 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-04-22T07:16:37.317 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-04-22T07:16:37.318 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-04-22T07:16:37.319 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-04-22T07:16:37.320 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-04-22T07:16:37.321 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-04-22T07:16:37.322 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-04-22T07:16:37.323 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-04-22T07:16:37.324 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-04-22T07:16:37.325 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-04-22T07:16:37.326 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-04-22T07:16:37.326 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-04-22T07:16:37.327 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-04-22T07:16:37.328 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-04-22T07:16:37.330 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-04-22T07:16:37.330 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-04-22T07:16:37.331 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-04-22T07:16:37.332 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-04-22T07:16:37.333 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-04-22T07:16:37.334 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-04-22T07:16:37.335 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-04-22T07:16:37.336 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-04-22T07:16:37.337 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-04-22T07:16:37.338 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-04-22T07:16:37.339 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-04-22T07:16:37.339 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-04-22T07:16:37.341 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-04-22T07:16:37.341 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-04-22T07:16:37.342 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-04-22T07:16:37.343 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-04-22T07:16:37.345 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-04-22T07:16:37.345 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-04-22T07:16:37.346 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-04-22T07:16:37.347 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-04-22T07:16:37.348 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-04-22T07:16:37.349 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-04-22T07:16:37.350 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-04-22T07:16:37.351 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-04-22T07:16:37.352 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-04-22T07:16:37.353 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-04-22T07:16:37.354 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-04-22T07:16:37.355 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-04-22T07:16:37.356 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-04-22T07:16:37.357 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-04-22T07:16:37.357 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-04-22T07:16:37.358 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-04-22T07:16:37.360 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-04-22T07:16:37.360 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-04-22T07:16:37.361 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-04-22T07:16:37.362 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-04-22T07:16:37.363 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-04-22T07:16:37.364 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-04-22T07:16:37.365 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-04-22T07:16:37.366 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-04-22T07:16:37.367 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-04-22T07:16:37.368 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-04-22T07:16:37.369 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-04-22T07:16:37.370 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-04-22T07:16:37.372 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-04-22T07:16:37.373 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-04-22T07:16:37.375 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-04-22T07:16:37.376 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-04-22T07:16:37.376 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-04-22T07:16:37.377 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-04-22T07:16:37.378 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-04-22T07:16:37.379 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-04-22T07:16:37.380 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-04-22T07:16:37.381 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-04-22T07:16:37.382 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-04-22T07:16:37.383 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-04-22T07:16:37.384 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-04-22T07:16:37.385 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-04-22T07:16:37.386 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-04-22T07:16:37.387 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-04-22T07:16:37.388 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-04-22T07:16:37.389 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-04-22T07:16:37.389 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-04-22T07:16:37.390 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-04-22T07:16:37.391 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-04-22T07:16:37.392 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-04-22T07:16:37.393 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-04-22T07:16:37.394 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-04-22T07:16:37.395 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-04-22T07:16:37.396 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-04-22T07:16:37.397 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-04-22T07:16:37.398 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-04-22T07:16:37.398 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-04-22T07:16:37.400 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-04-22T07:16:37.401 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-04-22T07:16:37.402 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-04-22T07:16:37.402 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-04-22T07:16:37.403 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-04-22T07:16:37.404 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-04-22T07:16:37.405 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-04-22T07:16:37.406 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-04-22T07:16:37.407 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-04-22T07:16:37.408 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-04-22T07:16:37.409 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-04-22T07:16:37.410 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-04-22T07:16:37.411 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-04-22T07:16:37.412 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-04-22T07:16:37.413 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-04-22T07:16:37.414 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-04-22T07:16:37.415 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-04-22T07:16:37.416 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-04-22T07:16:37.417 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-04-22T07:16:37.418 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-04-22T07:16:37.419 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-04-22T07:16:37.420 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-04-22T07:16:37.421 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-04-22T07:16:37.422 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-04-22T07:16:37.423 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-04-22T07:16:37.424 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-04-22T07:16:37.425 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-04-22T07:16:37.426 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-04-22T07:16:37.427 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-04-22T07:16:37.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-04-22T07:16:37.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-04-22T07:16:37.429 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-04-22T07:16:37.431 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-04-22T07:16:37.432 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-04-22T07:16:37.433 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-04-22T07:16:37.434 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-04-22T07:16:37.434 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-04-22T07:16:37.435 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-04-22T07:16:37.436 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-04-22T07:16:37.437 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-04-22T07:16:37.438 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-04-22T07:16:37.439 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-04-22T07:16:37.440 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-04-22T07:16:37.441 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-04-22T07:16:37.442 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-04-22T07:16:37.444 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-04-22T07:16:37.445 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-04-22T07:16:37.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-04-22T07:16:37.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-04-22T07:16:37.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-04-22T07:16:37.448 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-04-22T07:16:37.450 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-04-22T07:16:37.450 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-04-22T07:16:37.452 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-04-22T07:16:37.452 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-04-22T07:16:37.454 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-04-22T07:16:37.455 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-04-22T07:16:37.455 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-04-22T07:16:37.457 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-04-22T07:16:37.458 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-04-22T07:16:37.459 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-04-22T07:16:37.460 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-04-22T07:16:37.461 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-04-22T07:16:37.462 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-04-22T07:16:37.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-04-22T07:16:37.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-04-22T07:16:37.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-04-22T07:16:37.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-04-22T07:16:37.467 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-04-22T07:16:37.467 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-04-22T07:16:37.468 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-04-22T07:16:37.469 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-04-22T07:16:37.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-04-22T07:16:37.472 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-04-22T07:16:37.472 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-04-22T07:16:37.473 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-04-22T07:16:37.474 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-04-22T07:16:37.475 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-04-22T07:16:37.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-04-22T07:16:37.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-04-22T07:16:37.477 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-04-22T07:16:37.478 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-04-22T07:16:37.479 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-04-22T07:16:37.480 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-04-22T07:16:37.481 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-04-22T07:16:37.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-04-22T07:16:37.483 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-04-22T07:16:37.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-04-22T07:16:37.485 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-04-22T07:16:37.486 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-04-22T07:16:37.487 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-04-22T07:16:37.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-04-22T07:16:37.489 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-04-22T07:16:37.490 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-04-22T07:16:37.491 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-04-22T07:16:37.491 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-04-22T07:16:37.492 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-04-22T07:16:37.493 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-04-22T07:16:37.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-04-22T07:16:37.495 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-04-22T07:16:37.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-04-22T07:16:37.498 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-04-22T07:16:37.499 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-04-22T07:16:37.500 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-04-22T07:16:37.501 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-04-22T07:16:37.502 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-04-22T07:16:37.503 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-04-22T07:16:37.504 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-04-22T07:16:37.505 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-04-22T07:16:37.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-04-22T07:16:37.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-04-22T07:16:37.508 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-04-22T07:16:37.508 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-04-22T07:16:37.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-04-22T07:16:37.511 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-04-22T07:16:37.512 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-04-22T07:16:37.513 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-04-22T07:16:37.514 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-04-22T07:16:37.515 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-04-22T07:16:37.516 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-04-22T07:16:37.517 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-04-22T07:16:37.518 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-04-22T07:16:37.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-04-22T07:16:37.520 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-04-22T07:16:37.521 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-04-22T07:16:37.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-04-22T07:16:37.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-04-22T07:16:37.523 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-04-22T07:16:37.524 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-04-22T07:16:37.525 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-04-22T07:16:37.526 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-04-22T07:16:37.527 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-04-22T07:16:37.528 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-04-22T07:16:37.529 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-04-22T07:16:37.530 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-04-22T07:16:37.531 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-04-22T07:16:37.532 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-04-22T07:16:37.533 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-04-22T07:16:37.534 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-04-22T07:16:37.535 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-04-22T07:16:37.536 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-04-22T07:16:37.537 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-04-22T07:16:37.538 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-04-22T07:16:37.539 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-04-22T07:16:37.540 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-04-22T07:16:37.541 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-04-22T07:16:37.542 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-04-22T07:16:37.543 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-04-22T07:16:37.544 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-04-22T07:16:37.545 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-04-22T07:16:37.546 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-04-22T07:16:37.547 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-04-22T07:16:37.548 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-04-22T07:16:37.549 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-04-22T07:16:37.550 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-04-22T07:16:37.551 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-04-22T07:16:37.552 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-04-22T07:16:37.553 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-04-22T07:16:37.554 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-04-22T07:16:37.555 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-04-22T07:16:37.556 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-04-22T07:16:37.557 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-04-22T07:16:37.558 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-04-22T07:16:37.559 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-04-22T07:16:37.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:37 smithi133 ceph-mon[31198]: pgmap v58: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.8 MiB/s wr, 32 op/s 2024-04-22T07:16:37.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:37 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:37.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:37 smithi154 ceph-mon[31862]: pgmap v58: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.8 MiB/s wr, 32 op/s 2024-04-22T07:16:37.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:37 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:37.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:37 smithi113 ceph-mon[27933]: pgmap v58: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.8 MiB/s wr, 32 op/s 2024-04-22T07:16:37.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:37 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:37.826 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-04-22T07:16:37.827 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-04-22T07:16:37.828 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-04-22T07:16:37.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-04-22T07:16:37.830 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-04-22T07:16:37.831 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-04-22T07:16:37.832 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-04-22T07:16:37.833 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-04-22T07:16:37.834 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-04-22T07:16:37.835 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-04-22T07:16:37.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-04-22T07:16:37.837 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-04-22T07:16:37.838 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-04-22T07:16:37.839 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-04-22T07:16:37.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-04-22T07:16:37.841 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-04-22T07:16:37.842 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-04-22T07:16:37.843 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-04-22T07:16:37.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-04-22T07:16:37.845 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-04-22T07:16:37.846 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-04-22T07:16:37.847 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-04-22T07:16:37.848 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-04-22T07:16:37.849 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-04-22T07:16:37.850 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-04-22T07:16:37.851 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-04-22T07:16:37.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-04-22T07:16:37.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-04-22T07:16:37.854 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-04-22T07:16:37.855 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-04-22T07:16:37.856 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-04-22T07:16:37.857 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-04-22T07:16:37.858 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-04-22T07:16:37.859 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-04-22T07:16:37.860 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-04-22T07:16:37.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-04-22T07:16:37.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-04-22T07:16:37.863 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-04-22T07:16:37.864 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-04-22T07:16:37.865 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-04-22T07:16:37.866 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-04-22T07:16:37.867 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-04-22T07:16:37.868 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-04-22T07:16:37.869 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-04-22T07:16:37.869 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-04-22T07:16:37.870 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-04-22T07:16:37.871 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-04-22T07:16:37.872 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-04-22T07:16:37.874 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-04-22T07:16:37.875 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-04-22T07:16:37.876 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-04-22T07:16:37.877 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-04-22T07:16:37.878 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-04-22T07:16:37.879 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-04-22T07:16:37.880 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-04-22T07:16:37.880 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-04-22T07:16:37.882 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-04-22T07:16:37.882 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-04-22T07:16:37.883 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-04-22T07:16:37.884 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-04-22T07:16:37.885 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-04-22T07:16:37.886 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-04-22T07:16:37.888 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-04-22T07:16:37.889 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-04-22T07:16:37.890 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-04-22T07:16:37.890 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-04-22T07:16:37.892 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-04-22T07:16:37.893 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-04-22T07:16:37.894 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-04-22T07:16:37.894 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-04-22T07:16:37.895 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-04-22T07:16:37.896 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-04-22T07:16:37.897 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-04-22T07:16:37.898 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-04-22T07:16:37.899 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-04-22T07:16:37.900 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-04-22T07:16:37.901 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-04-22T07:16:37.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-04-22T07:16:37.903 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-04-22T07:16:37.904 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-04-22T07:16:37.905 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-04-22T07:16:37.906 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-04-22T07:16:37.907 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-04-22T07:16:37.908 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-04-22T07:16:37.909 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-04-22T07:16:37.910 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-04-22T07:16:37.911 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-04-22T07:16:37.912 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-04-22T07:16:37.913 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-04-22T07:16:37.914 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-04-22T07:16:37.914 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-04-22T07:16:37.915 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-04-22T07:16:37.917 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-04-22T07:16:37.918 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-04-22T07:16:37.919 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-04-22T07:16:37.919 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-04-22T07:16:37.920 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-04-22T07:16:37.921 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-04-22T07:16:37.922 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-04-22T07:16:37.923 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-04-22T07:16:37.924 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-04-22T07:16:37.925 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-04-22T07:16:37.926 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-04-22T07:16:37.927 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-04-22T07:16:37.927 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-04-22T07:16:37.929 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-04-22T07:16:37.929 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-04-22T07:16:37.930 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-04-22T07:16:37.931 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-04-22T07:16:37.932 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-04-22T07:16:37.933 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-04-22T07:16:37.934 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-04-22T07:16:37.935 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-04-22T07:16:37.936 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-04-22T07:16:37.937 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-04-22T07:16:37.937 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-04-22T07:16:37.938 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-04-22T07:16:37.939 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-04-22T07:16:37.940 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-04-22T07:16:37.941 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-04-22T07:16:37.942 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-04-22T07:16:37.943 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-04-22T07:16:37.944 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-04-22T07:16:37.945 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-04-22T07:16:37.946 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-04-22T07:16:37.947 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-04-22T07:16:37.948 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-04-22T07:16:37.949 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-04-22T07:16:37.950 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-04-22T07:16:37.951 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-04-22T07:16:37.952 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-04-22T07:16:37.953 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-04-22T07:16:37.954 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-04-22T07:16:37.955 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-04-22T07:16:37.956 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-04-22T07:16:37.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-04-22T07:16:37.958 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-04-22T07:16:37.959 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-04-22T07:16:37.960 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-04-22T07:16:37.961 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-04-22T07:16:37.962 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-04-22T07:16:37.963 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-04-22T07:16:37.964 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-04-22T07:16:37.965 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-04-22T07:16:37.966 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-04-22T07:16:37.967 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-04-22T07:16:37.968 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-04-22T07:16:37.969 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-04-22T07:16:37.971 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-04-22T07:16:37.972 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-04-22T07:16:37.972 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-04-22T07:16:37.973 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-04-22T07:16:37.974 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-04-22T07:16:37.975 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-04-22T07:16:37.976 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-04-22T07:16:37.977 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-04-22T07:16:37.978 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-04-22T07:16:37.979 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-04-22T07:16:37.980 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-04-22T07:16:37.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-04-22T07:16:37.982 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-04-22T07:16:37.983 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-04-22T07:16:37.984 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-04-22T07:16:37.985 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-04-22T07:16:37.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-04-22T07:16:37.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-04-22T07:16:37.988 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-04-22T07:16:37.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-04-22T07:16:37.990 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-04-22T07:16:37.991 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-04-22T07:16:37.992 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-04-22T07:16:37.993 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-04-22T07:16:37.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-04-22T07:16:37.995 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-04-22T07:16:37.996 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-04-22T07:16:37.997 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-04-22T07:16:37.998 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-04-22T07:16:37.999 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-04-22T07:16:38.000 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-04-22T07:16:38.001 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-04-22T07:16:38.002 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-04-22T07:16:38.003 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-04-22T07:16:38.004 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-04-22T07:16:38.005 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-04-22T07:16:38.006 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-04-22T07:16:38.007 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-04-22T07:16:38.008 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-04-22T07:16:38.008 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-04-22T07:16:38.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-04-22T07:16:38.011 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-04-22T07:16:38.012 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-04-22T07:16:38.013 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-04-22T07:16:38.014 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-04-22T07:16:38.015 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-04-22T07:16:38.016 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-04-22T07:16:38.017 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-04-22T07:16:38.017 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-04-22T07:16:38.018 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-04-22T07:16:38.019 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-04-22T07:16:38.020 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-04-22T07:16:38.021 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-04-22T07:16:38.022 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-04-22T07:16:38.023 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-04-22T07:16:38.024 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-04-22T07:16:38.025 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-04-22T07:16:38.026 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-04-22T07:16:38.027 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-04-22T07:16:38.028 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-04-22T07:16:38.029 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-04-22T07:16:38.030 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-04-22T07:16:38.031 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-04-22T07:16:38.032 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-04-22T07:16:38.033 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-04-22T07:16:38.034 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-04-22T07:16:38.034 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-04-22T07:16:38.036 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-04-22T07:16:38.037 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-04-22T07:16:38.038 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-04-22T07:16:38.039 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-04-22T07:16:38.039 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-04-22T07:16:38.041 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-04-22T07:16:38.042 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-04-22T07:16:38.043 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-04-22T07:16:38.044 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-04-22T07:16:38.044 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-04-22T07:16:38.045 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-04-22T07:16:38.047 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-04-22T07:16:38.048 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-04-22T07:16:38.049 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-04-22T07:16:38.050 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-04-22T07:16:38.050 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-04-22T07:16:38.052 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-04-22T07:16:38.053 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-04-22T07:16:38.054 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-04-22T07:16:38.055 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-04-22T07:16:38.056 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-04-22T07:16:38.057 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-04-22T07:16:38.058 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-04-22T07:16:38.059 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-04-22T07:16:38.060 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-04-22T07:16:38.061 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-04-22T07:16:38.062 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-04-22T07:16:38.063 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-04-22T07:16:38.064 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-04-22T07:16:38.065 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-04-22T07:16:38.066 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-04-22T07:16:38.067 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-04-22T07:16:38.068 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-04-22T07:16:38.069 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-04-22T07:16:38.070 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-04-22T07:16:38.071 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-04-22T07:16:38.072 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-04-22T07:16:38.073 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-04-22T07:16:38.074 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-04-22T07:16:38.074 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-04-22T07:16:38.076 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-04-22T07:16:38.076 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-04-22T07:16:38.078 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-04-22T07:16:38.079 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-04-22T07:16:38.080 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-04-22T07:16:38.080 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-04-22T07:16:38.081 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-04-22T07:16:38.082 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-04-22T07:16:38.083 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-04-22T07:16:38.084 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-04-22T07:16:38.085 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-04-22T07:16:38.086 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-04-22T07:16:38.087 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-04-22T07:16:38.088 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-04-22T07:16:38.089 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-04-22T07:16:38.091 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-04-22T07:16:38.092 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-04-22T07:16:38.094 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-04-22T07:16:38.095 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-04-22T07:16:38.097 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-04-22T07:16:38.098 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-04-22T07:16:38.100 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-04-22T07:16:38.101 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-04-22T07:16:38.103 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-04-22T07:16:38.104 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-04-22T07:16:38.106 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-04-22T07:16:38.107 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-04-22T07:16:38.109 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-04-22T07:16:38.110 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-04-22T07:16:38.111 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-04-22T07:16:38.113 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-04-22T07:16:38.114 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-04-22T07:16:38.116 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-04-22T07:16:38.117 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-04-22T07:16:38.119 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-04-22T07:16:38.120 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-04-22T07:16:38.122 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-04-22T07:16:38.123 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-04-22T07:16:38.125 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-04-22T07:16:38.126 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-04-22T07:16:38.127 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-04-22T07:16:38.129 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-04-22T07:16:38.130 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-04-22T07:16:38.131 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-04-22T07:16:38.133 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-04-22T07:16:38.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-04-22T07:16:38.136 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-04-22T07:16:38.138 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-04-22T07:16:38.139 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-04-22T07:16:38.140 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-04-22T07:16:38.142 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-04-22T07:16:38.143 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-04-22T07:16:38.144 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-04-22T07:16:38.145 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-04-22T07:16:38.146 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-04-22T07:16:38.147 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-04-22T07:16:38.147 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-04-22T07:16:38.148 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-04-22T07:16:38.149 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-04-22T07:16:38.150 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-04-22T07:16:38.151 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-04-22T07:16:38.152 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-04-22T07:16:38.153 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-04-22T07:16:38.154 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-04-22T07:16:38.155 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-04-22T07:16:38.155 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-04-22T07:16:38.156 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-04-22T07:16:38.158 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-04-22T07:16:38.158 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-04-22T07:16:38.159 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-04-22T07:16:38.160 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-04-22T07:16:38.161 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-04-22T07:16:38.162 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-04-22T07:16:38.163 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-04-22T07:16:38.164 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-04-22T07:16:38.165 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-04-22T07:16:38.166 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-04-22T07:16:38.167 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-04-22T07:16:38.167 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-04-22T07:16:38.168 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-04-22T07:16:38.169 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-04-22T07:16:38.170 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-04-22T07:16:38.171 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-04-22T07:16:38.172 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-04-22T07:16:38.174 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-04-22T07:16:38.175 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-04-22T07:16:38.175 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-04-22T07:16:38.176 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-04-22T07:16:38.177 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-04-22T07:16:38.178 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-04-22T07:16:38.178 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-04-22T07:16:38.180 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-04-22T07:16:38.181 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-04-22T07:16:38.182 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-04-22T07:16:38.183 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-04-22T07:16:38.184 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-04-22T07:16:38.184 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-04-22T07:16:38.186 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-04-22T07:16:38.186 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-04-22T07:16:38.187 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-04-22T07:16:38.188 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-04-22T07:16:38.189 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-04-22T07:16:38.190 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-04-22T07:16:38.191 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-04-22T07:16:38.192 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-04-22T07:16:38.193 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-04-22T07:16:38.194 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-04-22T07:16:38.195 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-04-22T07:16:38.196 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-04-22T07:16:38.197 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-04-22T07:16:38.198 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-04-22T07:16:38.199 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-04-22T07:16:38.200 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-04-22T07:16:38.201 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-04-22T07:16:38.202 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-04-22T07:16:38.203 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-04-22T07:16:38.204 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-04-22T07:16:38.205 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-04-22T07:16:38.206 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-04-22T07:16:38.207 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-04-22T07:16:38.208 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-04-22T07:16:38.209 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-04-22T07:16:38.210 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-04-22T07:16:38.211 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-04-22T07:16:38.212 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-04-22T07:16:38.213 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-04-22T07:16:38.214 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-04-22T07:16:38.215 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-04-22T07:16:38.216 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-04-22T07:16:38.216 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-04-22T07:16:38.217 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-04-22T07:16:38.219 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-04-22T07:16:38.220 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-04-22T07:16:38.221 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-04-22T07:16:38.222 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-04-22T07:16:38.223 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-04-22T07:16:38.224 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-04-22T07:16:38.225 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-04-22T07:16:38.226 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-04-22T07:16:38.227 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-04-22T07:16:38.228 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-04-22T07:16:38.229 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-04-22T07:16:38.230 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-04-22T07:16:38.231 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-04-22T07:16:38.232 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-04-22T07:16:38.233 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-04-22T07:16:38.234 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-04-22T07:16:38.235 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-04-22T07:16:38.236 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-04-22T07:16:38.237 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-04-22T07:16:38.238 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-04-22T07:16:38.239 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-04-22T07:16:38.240 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-04-22T07:16:38.241 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-04-22T07:16:38.242 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-04-22T07:16:38.243 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-04-22T07:16:38.244 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-04-22T07:16:38.245 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-04-22T07:16:38.246 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-04-22T07:16:38.247 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-04-22T07:16:38.248 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-04-22T07:16:38.249 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-04-22T07:16:38.250 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-04-22T07:16:38.251 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-04-22T07:16:38.252 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-04-22T07:16:38.253 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-04-22T07:16:38.254 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-04-22T07:16:38.255 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-04-22T07:16:38.256 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-04-22T07:16:38.257 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-04-22T07:16:38.258 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-04-22T07:16:38.259 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-04-22T07:16:38.260 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-04-22T07:16:38.261 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-04-22T07:16:38.262 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-04-22T07:16:38.263 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-04-22T07:16:38.264 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-04-22T07:16:38.265 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-04-22T07:16:38.266 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-04-22T07:16:38.267 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-04-22T07:16:38.267 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-04-22T07:16:38.269 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-04-22T07:16:38.269 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-04-22T07:16:38.270 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-04-22T07:16:38.271 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-04-22T07:16:38.272 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-04-22T07:16:38.273 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-04-22T07:16:38.274 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-04-22T07:16:38.275 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-04-22T07:16:38.275 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-04-22T07:16:38.276 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-04-22T07:16:38.277 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-04-22T07:16:38.278 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-04-22T07:16:38.279 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-04-22T07:16:38.280 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-04-22T07:16:38.281 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-04-22T07:16:38.282 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-04-22T07:16:38.286 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-04-22T07:16:38.286 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-04-22T07:16:38.286 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-04-22T07:16:38.286 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-04-22T07:16:38.287 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-04-22T07:16:38.287 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-04-22T07:16:38.288 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-04-22T07:16:38.289 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-04-22T07:16:38.290 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-04-22T07:16:38.291 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-04-22T07:16:38.292 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-04-22T07:16:38.292 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-04-22T07:16:38.293 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-04-22T07:16:38.294 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-04-22T07:16:38.295 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-04-22T07:16:38.296 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-04-22T07:16:38.297 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-04-22T07:16:38.298 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-04-22T07:16:38.299 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-04-22T07:16:38.300 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-04-22T07:16:38.301 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-04-22T07:16:38.302 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-04-22T07:16:38.303 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-04-22T07:16:38.304 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-04-22T07:16:38.305 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-04-22T07:16:38.306 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-04-22T07:16:38.306 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-04-22T07:16:38.307 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-04-22T07:16:38.308 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-04-22T07:16:38.310 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-04-22T07:16:38.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-04-22T07:16:38.312 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-04-22T07:16:38.313 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-04-22T07:16:38.315 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-04-22T07:16:38.316 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-04-22T07:16:38.317 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-04-22T07:16:38.318 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-04-22T07:16:38.319 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-04-22T07:16:38.320 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-04-22T07:16:38.321 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-04-22T07:16:38.323 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-04-22T07:16:38.324 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-04-22T07:16:38.325 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/digsig.txt 2024-04-22T07:16:38.326 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-04-22T07:16:38.327 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-04-22T07:16:38.328 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-04-22T07:16:38.329 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-04-22T07:16:38.330 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-04-22T07:16:38.332 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-04-22T07:16:38.333 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dontdiff 2024-04-22T07:16:38.334 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/ 2024-04-22T07:16:38.335 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-04-22T07:16:38.336 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-04-22T07:16:38.337 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-04-22T07:16:38.338 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-04-22T07:16:38.339 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-04-22T07:16:38.341 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-04-22T07:16:38.341 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-04-22T07:16:38.342 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-04-22T07:16:38.344 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-04-22T07:16:38.345 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-04-22T07:16:38.346 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/ 2024-04-22T07:16:38.347 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-04-22T07:16:38.348 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-04-22T07:16:38.350 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-04-22T07:16:38.351 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-04-22T07:16:38.352 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-04-22T07:16:38.353 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-04-22T07:16:38.354 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-04-22T07:16:38.355 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-04-22T07:16:38.356 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-04-22T07:16:38.357 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-04-22T07:16:38.358 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-04-22T07:16:38.359 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-04-22T07:16:38.361 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-04-22T07:16:38.362 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-04-22T07:16:38.363 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-04-22T07:16:38.364 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-04-22T07:16:38.365 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-04-22T07:16:38.366 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-04-22T07:16:38.368 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/edac.txt 2024-04-22T07:16:38.369 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-04-22T07:16:38.370 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/eisa.txt 2024-04-22T07:16:38.371 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-04-22T07:16:38.374 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/extcon/ 2024-04-22T07:16:38.375 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-04-22T07:16:38.376 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-04-22T07:16:38.377 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-04-22T07:16:38.378 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-04-22T07:16:38.379 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-04-22T07:16:38.381 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/ 2024-04-22T07:16:38.382 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-04-22T07:16:38.383 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-04-22T07:16:38.384 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-04-22T07:16:38.385 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-04-22T07:16:38.386 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-04-22T07:16:38.388 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-04-22T07:16:38.389 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-04-22T07:16:38.390 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-04-22T07:16:38.391 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-04-22T07:16:38.392 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-04-22T07:16:38.393 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-04-22T07:16:38.395 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-04-22T07:16:38.396 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-04-22T07:16:38.397 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-04-22T07:16:38.398 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-04-22T07:16:38.399 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-04-22T07:16:38.400 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-04-22T07:16:38.401 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-04-22T07:16:38.402 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-04-22T07:16:38.404 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-04-22T07:16:38.405 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-04-22T07:16:38.406 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-04-22T07:16:38.407 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-04-22T07:16:38.408 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-04-22T07:16:38.409 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-04-22T07:16:38.410 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-04-22T07:16:38.412 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-04-22T07:16:38.413 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-04-22T07:16:38.414 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-04-22T07:16:38.415 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-04-22T07:16:38.416 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-04-22T07:16:38.417 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-04-22T07:16:38.418 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-04-22T07:16:38.419 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-04-22T07:16:38.421 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-04-22T07:16:38.422 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ 2024-04-22T07:16:38.423 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-04-22T07:16:38.424 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-04-22T07:16:38.425 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-04-22T07:16:38.426 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-04-22T07:16:38.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-04-22T07:16:38.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-04-22T07:16:38.430 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-04-22T07:16:38.431 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-04-22T07:16:38.432 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-04-22T07:16:38.433 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-04-22T07:16:38.434 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-04-22T07:16:38.435 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-04-22T07:16:38.437 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-04-22T07:16:38.438 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-04-22T07:16:38.439 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-04-22T07:16:38.440 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-04-22T07:16:38.441 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-04-22T07:16:38.443 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-04-22T07:16:38.444 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-04-22T07:16:38.445 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-04-22T07:16:38.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-04-22T07:16:38.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-04-22T07:16:38.448 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-04-22T07:16:38.449 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-04-22T07:16:38.451 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-04-22T07:16:38.452 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-04-22T07:16:38.453 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-04-22T07:16:38.454 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-04-22T07:16:38.455 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-04-22T07:16:38.457 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-04-22T07:16:38.458 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-04-22T07:16:38.459 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-04-22T07:16:38.460 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-04-22T07:16:38.461 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-04-22T07:16:38.463 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-04-22T07:16:38.463 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-04-22T07:16:38.465 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-04-22T07:16:38.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-04-22T07:16:38.467 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-04-22T07:16:38.468 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-04-22T07:16:38.470 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-04-22T07:16:38.471 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-04-22T07:16:38.472 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-04-22T07:16:38.473 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-04-22T07:16:38.474 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-04-22T07:16:38.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-04-22T07:16:38.477 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-04-22T07:16:38.478 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-04-22T07:16:38.479 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-04-22T07:16:38.480 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-04-22T07:16:38.481 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-04-22T07:16:38.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-04-22T07:16:38.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-04-22T07:16:38.485 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-04-22T07:16:38.486 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-04-22T07:16:38.487 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-04-22T07:16:38.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-04-22T07:16:38.490 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-04-22T07:16:38.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-04-22T07:16:38.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-04-22T07:16:38.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-04-22T07:16:38.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-04-22T07:16:38.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-04-22T07:16:38.497 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-04-22T07:16:38.498 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-04-22T07:16:38.499 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-04-22T07:16:38.501 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-04-22T07:16:38.502 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-04-22T07:16:38.503 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-04-22T07:16:38.504 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-04-22T07:16:38.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-04-22T07:16:38.507 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-04-22T07:16:38.508 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-04-22T07:16:38.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-04-22T07:16:38.510 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-04-22T07:16:38.512 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-04-22T07:16:38.513 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-04-22T07:16:38.514 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-04-22T07:16:38.515 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-04-22T07:16:38.516 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-04-22T07:16:38.518 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-04-22T07:16:38.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-04-22T07:16:38.520 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-04-22T07:16:38.521 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-04-22T07:16:38.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-04-22T07:16:38.523 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-04-22T07:16:38.525 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-04-22T07:16:38.526 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-04-22T07:16:38.527 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-04-22T07:16:38.528 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-04-22T07:16:38.529 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-04-22T07:16:38.530 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-04-22T07:16:38.532 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-04-22T07:16:38.533 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-04-22T07:16:38.534 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-04-22T07:16:38.536 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-04-22T07:16:38.537 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-04-22T07:16:38.538 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-04-22T07:16:38.539 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-04-22T07:16:38.540 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-04-22T07:16:38.542 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-04-22T07:16:38.543 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-04-22T07:16:38.544 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-04-22T07:16:38.545 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-04-22T07:16:38.546 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-04-22T07:16:38.548 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-04-22T07:16:38.549 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-04-22T07:16:38.550 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-04-22T07:16:38.551 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-04-22T07:16:38.552 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-04-22T07:16:38.554 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-04-22T07:16:38.555 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-04-22T07:16:38.556 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-04-22T07:16:38.557 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-04-22T07:16:38.558 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-04-22T07:16:38.559 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-04-22T07:16:38.561 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-04-22T07:16:38.562 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-04-22T07:16:38.563 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-04-22T07:16:38.565 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/ 2024-04-22T07:16:38.566 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-04-22T07:16:38.568 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-04-22T07:16:38.570 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-04-22T07:16:38.572 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-04-22T07:16:38.573 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-04-22T07:16:38.575 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-04-22T07:16:38.576 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-04-22T07:16:38.578 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-04-22T07:16:38.580 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-04-22T07:16:38.581 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-04-22T07:16:38.583 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-04-22T07:16:38.585 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/ 2024-04-22T07:16:38.586 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-04-22T07:16:38.588 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-04-22T07:16:38.590 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-04-22T07:16:38.591 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-04-22T07:16:38.593 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-04-22T07:16:38.595 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-04-22T07:16:38.596 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-04-22T07:16:38.598 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-04-22T07:16:38.600 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-04-22T07:16:38.601 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-04-22T07:16:38.603 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-04-22T07:16:38.605 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gcov.txt 2024-04-22T07:16:38.607 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-04-22T07:16:38.609 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/ 2024-04-22T07:16:38.610 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-04-22T07:16:38.612 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-04-22T07:16:38.613 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-04-22T07:16:38.615 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-04-22T07:16:38.617 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-04-22T07:16:38.618 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-04-22T07:16:38.620 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-04-22T07:16:38.622 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hid/ 2024-04-22T07:16:38.623 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-04-22T07:16:38.625 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-04-22T07:16:38.627 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-04-22T07:16:38.628 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-04-22T07:16:38.630 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-04-22T07:16:38.632 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/highuid.txt 2024-04-22T07:16:38.634 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hsi.txt 2024-04-22T07:16:38.635 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-04-22T07:16:38.637 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ 2024-04-22T07:16:38.639 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-04-22T07:16:38.641 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-04-22T07:16:38.642 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-04-22T07:16:38.644 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-04-22T07:16:38.645 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-04-22T07:16:38.647 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-04-22T07:16:38.648 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-04-22T07:16:38.650 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-04-22T07:16:38.652 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-04-22T07:16:38.654 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-04-22T07:16:38.655 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-04-22T07:16:38.657 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-04-22T07:16:38.659 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-04-22T07:16:38.660 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-04-22T07:16:38.662 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-04-22T07:16:38.663 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-04-22T07:16:38.665 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-04-22T07:16:38.667 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-04-22T07:16:38.668 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-04-22T07:16:38.670 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-04-22T07:16:38.672 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-04-22T07:16:38.673 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-04-22T07:16:38.675 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-04-22T07:16:38.677 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-04-22T07:16:38.678 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-04-22T07:16:38.680 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-04-22T07:16:38.682 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-04-22T07:16:38.683 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-04-22T07:16:38.685 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-04-22T07:16:38.686 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-04-22T07:16:38.689 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-04-22T07:16:38.690 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-04-22T07:16:38.692 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-04-22T07:16:38.693 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-04-22T07:16:38.695 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-04-22T07:16:38.697 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-04-22T07:16:38.698 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-04-22T07:16:38.702 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-04-22T07:16:38.702 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-04-22T07:16:38.703 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-04-22T07:16:38.705 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-04-22T07:16:38.706 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-04-22T07:16:38.708 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-04-22T07:16:38.709 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-04-22T07:16:38.711 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-04-22T07:16:38.713 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-04-22T07:16:38.715 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-04-22T07:16:38.716 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-04-22T07:16:38.718 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-04-22T07:16:38.719 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-04-22T07:16:38.721 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-04-22T07:16:38.723 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-04-22T07:16:38.724 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-04-22T07:16:38.726 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-04-22T07:16:38.728 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-04-22T07:16:38.729 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-04-22T07:16:38.731 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-04-22T07:16:38.733 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-04-22T07:16:38.734 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-04-22T07:16:38.736 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-04-22T07:16:38.738 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-04-22T07:16:38.739 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-04-22T07:16:38.741 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-04-22T07:16:38.742 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-04-22T07:16:38.744 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-04-22T07:16:38.746 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-04-22T07:16:38.747 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-04-22T07:16:38.749 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-04-22T07:16:38.751 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-04-22T07:16:38.752 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-04-22T07:16:38.754 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-04-22T07:16:38.756 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-04-22T07:16:38.757 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-04-22T07:16:38.759 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-04-22T07:16:38.761 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-04-22T07:16:38.762 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-04-22T07:16:38.765 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-04-22T07:16:38.766 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-04-22T07:16:38.767 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-04-22T07:16:38.769 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-04-22T07:16:38.771 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-04-22T07:16:38.772 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-04-22T07:16:38.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-04-22T07:16:38.775 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-04-22T07:16:38.777 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-04-22T07:16:38.779 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-04-22T07:16:38.781 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-04-22T07:16:38.782 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-04-22T07:16:38.784 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-04-22T07:16:38.786 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-04-22T07:16:38.787 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-04-22T07:16:38.789 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-04-22T07:16:38.791 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-04-22T07:16:38.792 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-04-22T07:16:38.794 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-04-22T07:16:38.796 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-04-22T07:16:38.797 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-04-22T07:16:38.799 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-04-22T07:16:38.800 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-04-22T07:16:38.802 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-04-22T07:16:38.804 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-04-22T07:16:38.805 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-04-22T07:16:38.806 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-04-22T07:16:38.808 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-04-22T07:16:38.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-04-22T07:16:38.811 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-04-22T07:16:38.812 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-04-22T07:16:38.814 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-04-22T07:16:38.815 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-04-22T07:16:38.817 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-04-22T07:16:38.818 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-04-22T07:16:38.820 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-04-22T07:16:38.821 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-04-22T07:16:38.823 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-04-22T07:16:38.824 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-04-22T07:16:38.825 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-04-22T07:16:38.827 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-04-22T07:16:38.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-04-22T07:16:38.830 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-04-22T07:16:38.832 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-04-22T07:16:38.833 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-04-22T07:16:38.835 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-04-22T07:16:38.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-04-22T07:16:38.837 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-04-22T07:16:38.839 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-04-22T07:16:38.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-04-22T07:16:38.842 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-04-22T07:16:38.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-04-22T07:16:38.845 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-04-22T07:16:38.846 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-04-22T07:16:38.848 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-04-22T07:16:38.849 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-04-22T07:16:38.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-04-22T07:16:38.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-04-22T07:16:38.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-04-22T07:16:38.855 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-04-22T07:16:38.856 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-04-22T07:16:38.858 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/ 2024-04-22T07:16:38.859 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-04-22T07:16:38.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-04-22T07:16:38.862 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-04-22T07:16:38.864 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-04-22T07:16:38.865 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-04-22T07:16:38.866 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-04-22T07:16:38.868 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-04-22T07:16:38.869 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-04-22T07:16:38.871 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-04-22T07:16:38.872 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-04-22T07:16:38.874 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-04-22T07:16:38.875 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-04-22T07:16:38.876 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-04-22T07:16:38.878 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-04-22T07:16:38.879 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-04-22T07:16:38.881 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-04-22T07:16:38.882 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-04-22T07:16:38.884 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-04-22T07:16:38.885 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-04-22T07:16:38.887 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-04-22T07:16:38.888 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-04-22T07:16:38.889 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-04-22T07:16:38.891 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-04-22T07:16:38.893 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-04-22T07:16:38.894 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-04-22T07:16:38.896 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-04-22T07:16:38.897 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-04-22T07:16:38.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-04-22T07:16:38.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-04-22T07:16:38.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-04-22T07:16:38.903 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-04-22T07:16:38.904 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-04-22T07:16:38.906 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/summary 2024-04-22T07:16:38.907 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-04-22T07:16:38.909 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-04-22T07:16:38.910 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-04-22T07:16:38.912 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2o/ 2024-04-22T07:16:38.913 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2o/README 2024-04-22T07:16:38.915 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-04-22T07:16:38.916 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/ 2024-04-22T07:16:38.918 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-04-22T07:16:38.919 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-04-22T07:16:38.920 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-04-22T07:16:38.922 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/README 2024-04-22T07:16:38.924 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-04-22T07:16:38.925 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-04-22T07:16:38.926 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-04-22T07:16:38.928 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-04-22T07:16:38.929 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-04-22T07:16:38.931 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-04-22T07:16:38.932 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-04-22T07:16:38.933 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-04-22T07:16:38.935 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/ 2024-04-22T07:16:38.937 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-04-22T07:16:38.938 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-04-22T07:16:38.939 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-04-22T07:16:38.941 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-04-22T07:16:38.942 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-04-22T07:16:38.944 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-04-22T07:16:38.945 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-04-22T07:16:38.946 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/infiniband/ 2024-04-22T07:16:38.948 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-04-22T07:16:38.949 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-04-22T07:16:38.951 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-04-22T07:16:38.952 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-04-22T07:16:38.954 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-04-22T07:16:38.955 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/init.txt 2024-04-22T07:16:38.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/initrd.txt 2024-04-22T07:16:38.958 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/ 2024-04-22T07:16:38.960 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-04-22T07:16:38.961 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-04-22T07:16:38.963 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-04-22T07:16:38.964 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-04-22T07:16:38.966 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-04-22T07:16:38.967 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-04-22T07:16:38.969 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-04-22T07:16:38.970 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-04-22T07:16:38.972 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-04-22T07:16:38.973 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-04-22T07:16:38.975 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-04-22T07:16:38.976 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-04-22T07:16:38.977 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-04-22T07:16:38.978 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-04-22T07:16:38.979 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-04-22T07:16:38.980 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-04-22T07:16:38.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-04-22T07:16:38.982 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/input.txt 2024-04-22T07:16:38.983 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-04-22T07:16:38.984 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-04-22T07:16:38.985 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-04-22T07:16:38.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-04-22T07:16:38.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-04-22T07:16:38.988 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-04-22T07:16:38.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-04-22T07:16:38.990 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-04-22T07:16:38.991 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-04-22T07:16:38.992 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-04-22T07:16:38.993 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-04-22T07:16:38.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-04-22T07:16:38.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-04-22T07:16:38.996 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-04-22T07:16:38.997 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-04-22T07:16:38.998 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-04-22T07:16:38.999 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/ 2024-04-22T07:16:39.000 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-04-22T07:16:39.001 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-04-22T07:16:39.001 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-04-22T07:16:39.003 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-04-22T07:16:39.004 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-04-22T07:16:39.004 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-04-22T07:16:39.005 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/iostats.txt 2024-04-22T07:16:39.007 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-04-22T07:16:39.008 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-04-22T07:16:39.009 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/ 2024-04-22T07:16:39.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-04-22T07:16:39.011 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-04-22T07:16:39.011 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-04-22T07:16:39.013 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-04-22T07:16:39.014 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-04-22T07:16:39.015 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-04-22T07:16:39.016 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README 2024-04-22T07:16:39.017 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-04-22T07:16:39.018 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-04-22T07:16:39.243 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-04-22T07:16:39.244 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-04-22T07:16:39.245 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-04-22T07:16:39.247 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-04-22T07:16:39.249 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-04-22T07:16:39.250 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-04-22T07:16:39.252 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-04-22T07:16:39.253 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-04-22T07:16:39.255 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-04-22T07:16:39.256 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-04-22T07:16:39.258 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-04-22T07:16:39.259 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-04-22T07:16:39.261 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-04-22T07:16:39.263 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-04-22T07:16:39.264 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-04-22T07:16:39.266 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-04-22T07:16:39.267 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-04-22T07:16:39.268 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-04-22T07:16:39.269 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-04-22T07:16:39.270 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-04-22T07:16:39.271 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-04-22T07:16:39.272 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-04-22T07:16:39.273 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/java.txt 2024-04-22T07:16:39.274 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kasan.txt 2024-04-22T07:16:39.275 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/ 2024-04-22T07:16:39.276 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-04-22T07:16:39.277 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-04-22T07:16:39.278 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-04-22T07:16:39.279 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-04-22T07:16:39.280 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-04-22T07:16:39.281 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-04-22T07:16:39.282 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-04-22T07:16:39.283 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kdump/ 2024-04-22T07:16:39.283 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-04-22T07:16:39.284 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-04-22T07:16:39.285 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-04-22T07:16:39.287 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-04-22T07:16:39.288 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-04-22T07:16:39.290 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-04-22T07:16:39.292 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-04-22T07:16:39.293 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-04-22T07:16:39.294 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-04-22T07:16:39.295 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-04-22T07:16:39.295 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-04-22T07:16:39.296 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kobject.txt 2024-04-22T07:16:39.298 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-04-22T07:16:39.299 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kref.txt 2024-04-22T07:16:39.300 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-04-22T07:16:39.301 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/ 2024-04-22T07:16:39.302 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-04-22T07:16:39.303 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-04-22T07:16:39.304 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-04-22T07:16:39.304 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-04-22T07:16:39.305 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-04-22T07:16:39.306 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-04-22T07:16:39.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-04-22T07:16:39.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-04-22T07:16:39.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-04-22T07:16:39.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-04-22T07:16:39.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-04-22T07:16:39.312 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-04-22T07:16:39.312 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ldm.txt 2024-04-22T07:16:39.313 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/ 2024-04-22T07:16:39.314 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-04-22T07:16:39.315 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-04-22T07:16:39.316 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-04-22T07:16:39.317 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-04-22T07:16:39.318 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-04-22T07:16:39.319 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-04-22T07:16:39.321 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-04-22T07:16:39.321 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-04-22T07:16:39.322 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-04-22T07:16:39.323 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-04-22T07:16:39.324 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-04-22T07:16:39.325 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-04-22T07:16:39.326 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/ 2024-04-22T07:16:39.327 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-04-22T07:16:39.327 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-04-22T07:16:39.328 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-04-22T07:16:39.329 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-04-22T07:16:39.330 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-04-22T07:16:39.331 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-04-22T07:16:39.332 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-04-22T07:16:39.333 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-04-22T07:16:39.334 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-04-22T07:16:39.335 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-04-22T07:16:39.336 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-04-22T07:16:39.337 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/logo.gif 2024-04-22T07:16:39.338 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/logo.txt 2024-04-22T07:16:39.339 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/lzo.txt 2024-04-22T07:16:39.340 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/m68k/ 2024-04-22T07:16:39.341 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-04-22T07:16:39.342 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-04-22T07:16:39.343 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-04-22T07:16:39.343 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-04-22T07:16:39.345 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-04-22T07:16:39.346 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/md.txt 2024-04-22T07:16:39.347 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-04-22T07:16:39.348 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-04-22T07:16:39.408 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-04-22T07:16:39.409 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-04-22T07:16:39.409 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-04-22T07:16:39.411 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/metag/ 2024-04-22T07:16:39.411 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-04-22T07:16:39.412 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-04-22T07:16:39.413 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/ 2024-04-22T07:16:39.414 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-04-22T07:16:39.415 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-04-22T07:16:39.416 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-04-22T07:16:39.416 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-04-22T07:16:39.418 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-04-22T07:16:39.418 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-04-22T07:16:39.419 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-04-22T07:16:39.420 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-04-22T07:16:39.421 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-04-22T07:16:39.422 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-04-22T07:16:39.422 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mips/ 2024-04-22T07:16:39.423 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-04-22T07:16:39.424 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-04-22T07:16:39.425 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-04-22T07:16:39.426 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-04-22T07:16:39.427 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-04-22T07:16:39.428 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-04-22T07:16:39.429 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-04-22T07:16:39.430 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-04-22T07:16:39.431 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-04-22T07:16:39.432 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-04-22T07:16:39.433 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-04-22T07:16:39.433 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-04-22T07:16:39.435 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-04-22T07:16:39.436 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-04-22T07:16:39.437 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-04-22T07:16:39.438 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-04-22T07:16:39.439 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-04-22T07:16:39.440 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-04-22T07:16:39.441 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-04-22T07:16:39.442 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-04-22T07:16:39.442 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-04-22T07:16:39.443 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mmc/ 2024-04-22T07:16:39.444 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-04-22T07:16:39.445 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-04-22T07:16:39.446 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-04-22T07:16:39.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-04-22T07:16:39.447 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mn10300/ 2024-04-22T07:16:39.448 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-04-22T07:16:39.449 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-04-22T07:16:39.450 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-04-22T07:16:39.451 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mono.txt 2024-04-22T07:16:39.452 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mtd/ 2024-04-22T07:16:39.453 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-04-22T07:16:39.454 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-04-22T07:16:39.455 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-04-22T07:16:39.456 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-04-22T07:16:39.456 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/namespaces/ 2024-04-22T07:16:39.457 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-04-22T07:16:39.458 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-04-22T07:16:39.459 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/netlabel/ 2024-04-22T07:16:39.460 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-04-22T07:16:39.461 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-04-22T07:16:39.462 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-04-22T07:16:39.463 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-04-22T07:16:39.464 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-04-22T07:16:39.465 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ 2024-04-22T07:16:39.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-04-22T07:16:39.466 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-04-22T07:16:39.468 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-04-22T07:16:39.468 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-04-22T07:16:39.469 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-04-22T07:16:39.471 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-04-22T07:16:39.471 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-04-22T07:16:39.473 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-04-22T07:16:39.474 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-04-22T07:16:39.475 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-04-22T07:16:39.476 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-04-22T07:16:39.477 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-04-22T07:16:39.478 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-04-22T07:16:39.479 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-04-22T07:16:39.480 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-04-22T07:16:39.481 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-04-22T07:16:39.482 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-04-22T07:16:39.483 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-04-22T07:16:39.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-04-22T07:16:39.484 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-04-22T07:16:39.486 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-04-22T07:16:39.487 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-04-22T07:16:39.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-04-22T07:16:39.488 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-04-22T07:16:39.489 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-04-22T07:16:39.490 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-04-22T07:16:39.492 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-04-22T07:16:39.492 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-04-22T07:16:39.493 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-04-22T07:16:39.494 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-04-22T07:16:39.495 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-04-22T07:16:39.496 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-04-22T07:16:39.497 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-04-22T07:16:39.498 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-04-22T07:16:39.499 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-04-22T07:16:39.500 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-04-22T07:16:39.501 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-04-22T07:16:39.502 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-04-22T07:16:39.503 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-04-22T07:16:39.504 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-04-22T07:16:39.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-04-22T07:16:39.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-04-22T07:16:39.506 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-04-22T07:16:39.507 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-04-22T07:16:39.508 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-04-22T07:16:39.509 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-04-22T07:16:39.510 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-04-22T07:16:39.511 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-04-22T07:16:39.512 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-04-22T07:16:39.512 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-04-22T07:16:39.513 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-04-22T07:16:39.514 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-04-22T07:16:39.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-04-22T07:16:39.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-04-22T07:16:39.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-04-22T07:16:39.519 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-04-22T07:16:39.520 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-04-22T07:16:39.521 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-04-22T07:16:39.522 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-04-22T07:16:39.524 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-04-22T07:16:39.525 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-04-22T07:16:39.527 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-04-22T07:16:39.528 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-04-22T07:16:39.530 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-04-22T07:16:39.531 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-04-22T07:16:39.532 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-04-22T07:16:39.534 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-04-22T07:16:39.535 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-04-22T07:16:39.537 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-04-22T07:16:39.538 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-04-22T07:16:39.540 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-04-22T07:16:39.541 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-04-22T07:16:39.543 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-04-22T07:16:39.544 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-04-22T07:16:39.546 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-04-22T07:16:39.547 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-04-22T07:16:39.548 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-04-22T07:16:39.550 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-04-22T07:16:39.551 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-04-22T07:16:39.553 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-04-22T07:16:39.554 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-04-22T07:16:39.556 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-04-22T07:16:39.557 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-04-22T07:16:39.558 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-04-22T07:16:39.559 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-04-22T07:16:39.560 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-04-22T07:16:39.560 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-04-22T07:16:39.561 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-04-22T07:16:39.562 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-04-22T07:16:39.563 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-04-22T07:16:39.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:39 smithi133 ceph-mon[31198]: pgmap v59: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 6.8 MiB/s wr, 29 op/s 2024-04-22T07:16:39.564 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-04-22T07:16:39.565 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-04-22T07:16:39.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:39 smithi154 ceph-mon[31862]: pgmap v59: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 6.8 MiB/s wr, 29 op/s 2024-04-22T07:16:39.566 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-04-22T07:16:39.568 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-04-22T07:16:39.569 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-04-22T07:16:39.571 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-04-22T07:16:39.572 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-04-22T07:16:39.574 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-04-22T07:16:39.575 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-04-22T07:16:39.577 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-04-22T07:16:39.578 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-04-22T07:16:39.579 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-04-22T07:16:39.581 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-04-22T07:16:39.582 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-04-22T07:16:39.584 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-04-22T07:16:39.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:39 smithi113 ceph-mon[27933]: pgmap v59: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 6.8 MiB/s wr, 29 op/s 2024-04-22T07:16:39.585 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-04-22T07:16:39.585 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-04-22T07:16:39.586 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-04-22T07:16:39.587 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-04-22T07:16:39.588 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-04-22T07:16:39.589 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-04-22T07:16:39.590 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-04-22T07:16:39.591 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-04-22T07:16:39.591 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-04-22T07:16:39.592 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-04-22T07:16:39.593 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-04-22T07:16:39.594 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-04-22T07:16:39.595 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-04-22T07:16:39.596 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-04-22T07:16:39.597 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-04-22T07:16:39.598 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-04-22T07:16:39.598 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-04-22T07:16:39.599 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-04-22T07:16:39.600 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-04-22T07:16:39.601 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-04-22T07:16:39.602 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-04-22T07:16:39.604 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-04-22T07:16:39.605 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-04-22T07:16:39.606 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-04-22T07:16:39.607 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-04-22T07:16:39.607 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-04-22T07:16:39.609 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-04-22T07:16:39.610 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-04-22T07:16:39.610 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-04-22T07:16:39.612 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-04-22T07:16:39.612 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/nfc/ 2024-04-22T07:16:39.613 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-04-22T07:16:39.614 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-04-22T07:16:39.615 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/nios2/ 2024-04-22T07:16:39.615 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/nios2/README 2024-04-22T07:16:39.616 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-04-22T07:16:39.618 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/numastat.txt 2024-04-22T07:16:39.619 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-04-22T07:16:39.620 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/padata.txt 2024-04-22T07:16:39.621 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/parisc/ 2024-04-22T07:16:39.622 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-04-22T07:16:39.623 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-04-22T07:16:39.623 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/parisc/registers 2024-04-22T07:16:39.624 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-04-22T07:16:39.626 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/parport.txt 2024-04-22T07:16:39.627 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-04-22T07:16:39.628 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-04-22T07:16:39.629 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-04-22T07:16:39.630 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-04-22T07:16:39.631 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-04-22T07:16:39.632 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-04-22T07:16:39.633 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-04-22T07:16:39.634 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-04-22T07:16:39.635 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-04-22T07:16:39.636 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/phy.txt 2024-04-22T07:16:39.637 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/phy/ 2024-04-22T07:16:39.638 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-04-22T07:16:39.640 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-04-22T07:16:39.641 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-04-22T07:16:39.643 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/platform/ 2024-04-22T07:16:39.644 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-04-22T07:16:39.645 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pnp.txt 2024-04-22T07:16:39.647 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/ 2024-04-22T07:16:39.648 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-04-22T07:16:39.649 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-04-22T07:16:39.651 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-04-22T07:16:39.652 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-04-22T07:16:39.654 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-04-22T07:16:39.655 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-04-22T07:16:39.657 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-04-22T07:16:39.658 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-04-22T07:16:39.660 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-04-22T07:16:39.661 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-04-22T07:16:39.662 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-04-22T07:16:39.664 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-04-22T07:16:39.665 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-04-22T07:16:39.667 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-04-22T07:16:39.668 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-04-22T07:16:39.670 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-04-22T07:16:39.671 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-04-22T07:16:39.672 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-04-22T07:16:39.674 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-04-22T07:16:39.675 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-04-22T07:16:39.677 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-04-22T07:16:39.678 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-04-22T07:16:39.679 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-04-22T07:16:39.681 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/states.txt 2024-04-22T07:16:39.682 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-04-22T07:16:39.684 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-04-22T07:16:39.685 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-04-22T07:16:39.687 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-04-22T07:16:39.688 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-04-22T07:16:39.689 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-04-22T07:16:39.691 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-04-22T07:16:39.692 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/power/video.txt 2024-04-22T07:16:39.694 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/ 2024-04-22T07:16:39.695 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-04-22T07:16:39.697 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-04-22T07:16:39.698 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-04-22T07:16:39.700 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-04-22T07:16:39.701 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-04-22T07:16:39.702 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-04-22T07:16:39.704 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-04-22T07:16:39.705 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-04-22T07:16:39.707 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-04-22T07:16:39.708 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-04-22T07:16:39.710 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-04-22T07:16:39.711 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-04-22T07:16:39.713 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-04-22T07:16:39.714 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pps/ 2024-04-22T07:16:39.716 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-04-22T07:16:39.717 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/ 2024-04-22T07:16:39.718 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-04-22T07:16:39.722 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-04-22T07:16:39.722 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-04-22T07:16:39.722 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-04-22T07:16:39.724 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-04-22T07:16:39.725 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-04-22T07:16:39.727 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-04-22T07:16:39.728 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-04-22T07:16:39.730 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-04-22T07:16:39.731 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pti/ 2024-04-22T07:16:39.733 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-04-22T07:16:39.734 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ptp/ 2024-04-22T07:16:39.735 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-04-22T07:16:39.737 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-04-22T07:16:39.738 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-04-22T07:16:39.740 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-04-22T07:16:39.741 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-04-22T07:16:39.742 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/pwm.txt 2024-04-22T07:16:39.744 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-04-22T07:16:39.746 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rapidio/ 2024-04-22T07:16:39.747 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-04-22T07:16:39.748 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-04-22T07:16:39.750 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-04-22T07:16:39.751 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-04-22T07:16:39.753 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-04-22T07:16:39.755 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-04-22T07:16:39.756 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-04-22T07:16:39.758 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-04-22T07:16:39.759 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-04-22T07:16:39.761 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/rtc.txt 2024-04-22T07:16:39.762 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/ 2024-04-22T07:16:39.763 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-04-22T07:16:39.765 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-04-22T07:16:39.766 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-04-22T07:16:39.768 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-04-22T07:16:39.769 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/DASD 2024-04-22T07:16:39.770 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-04-22T07:16:39.772 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-04-22T07:16:39.774 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-04-22T07:16:39.775 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-04-22T07:16:39.777 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-04-22T07:16:39.778 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-04-22T07:16:39.780 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-04-22T07:16:39.781 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-04-22T07:16:39.783 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-04-22T07:16:39.784 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/ 2024-04-22T07:16:39.785 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-04-22T07:16:39.787 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-04-22T07:16:39.789 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-04-22T07:16:39.790 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-04-22T07:16:39.792 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-04-22T07:16:39.793 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-04-22T07:16:39.794 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-04-22T07:16:39.796 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-04-22T07:16:39.797 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-04-22T07:16:39.799 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-04-22T07:16:39.800 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ 2024-04-22T07:16:39.802 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-04-22T07:16:39.803 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-04-22T07:16:39.804 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-04-22T07:16:39.806 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-04-22T07:16:39.807 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-04-22T07:16:39.809 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-04-22T07:16:39.810 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-04-22T07:16:39.812 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-04-22T07:16:39.814 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-04-22T07:16:39.816 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-04-22T07:16:39.817 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-04-22T07:16:39.819 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-04-22T07:16:39.820 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-04-22T07:16:39.821 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-04-22T07:16:39.823 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-04-22T07:16:39.824 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-04-22T07:16:39.826 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-04-22T07:16:39.827 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-04-22T07:16:39.829 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-04-22T07:16:39.830 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-04-22T07:16:39.832 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-04-22T07:16:39.833 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-04-22T07:16:39.835 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-04-22T07:16:39.836 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-04-22T07:16:39.838 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-04-22T07:16:39.839 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-04-22T07:16:39.840 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-04-22T07:16:39.842 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-04-22T07:16:39.843 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-04-22T07:16:39.844 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-04-22T07:16:39.846 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-04-22T07:16:39.847 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-04-22T07:16:39.849 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-04-22T07:16:39.850 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-04-22T07:16:39.852 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-04-22T07:16:39.853 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-04-22T07:16:39.855 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-04-22T07:16:39.856 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-04-22T07:16:39.858 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-04-22T07:16:39.859 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-04-22T07:16:39.861 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-04-22T07:16:39.862 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-04-22T07:16:39.864 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-04-22T07:16:39.865 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-04-22T07:16:39.867 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-04-22T07:16:39.868 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-04-22T07:16:39.870 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-04-22T07:16:39.871 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-04-22T07:16:39.872 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-04-22T07:16:39.874 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-04-22T07:16:39.876 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-04-22T07:16:39.877 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-04-22T07:16:39.879 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-04-22T07:16:39.880 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-04-22T07:16:39.881 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-04-22T07:16:39.883 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-04-22T07:16:39.886 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-04-22T07:16:39.886 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-04-22T07:16:39.888 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-04-22T07:16:39.889 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/ 2024-04-22T07:16:39.890 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-04-22T07:16:39.892 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-04-22T07:16:39.893 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-04-22T07:16:39.895 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-04-22T07:16:39.896 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-04-22T07:16:39.898 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-04-22T07:16:39.900 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-04-22T07:16:39.901 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-04-22T07:16:39.902 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-04-22T07:16:39.904 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-04-22T07:16:39.905 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-04-22T07:16:39.907 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-04-22T07:16:39.908 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-04-22T07:16:39.909 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-04-22T07:16:39.911 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/ 2024-04-22T07:16:39.913 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-04-22T07:16:39.914 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-04-22T07:16:39.915 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/driver 2024-04-22T07:16:39.917 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-04-22T07:16:39.918 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-04-22T07:16:39.920 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-04-22T07:16:39.921 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-04-22T07:16:39.923 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-04-22T07:16:39.924 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-04-22T07:16:39.926 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sh/ 2024-04-22T07:16:39.930 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-04-22T07:16:39.930 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-04-22T07:16:39.930 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-04-22T07:16:39.932 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/ 2024-04-22T07:16:39.933 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-04-22T07:16:39.935 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-04-22T07:16:39.936 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-04-22T07:16:39.937 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-04-22T07:16:39.938 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-04-22T07:16:39.939 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-04-22T07:16:39.940 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-04-22T07:16:39.941 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-04-22T07:16:39.941 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-04-22T07:16:39.942 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-04-22T07:16:39.944 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-04-22T07:16:39.945 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-04-22T07:16:39.946 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-04-22T07:16:39.947 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-04-22T07:16:39.948 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-04-22T07:16:39.949 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-04-22T07:16:39.950 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-04-22T07:16:39.951 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-04-22T07:16:39.952 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-04-22T07:16:39.953 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-04-22T07:16:39.953 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-04-22T07:16:39.955 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-04-22T07:16:39.955 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-04-22T07:16:39.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-04-22T07:16:39.957 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-04-22T07:16:39.959 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-04-22T07:16:39.960 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-04-22T07:16:39.960 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-04-22T07:16:39.961 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-04-22T07:16:39.962 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-04-22T07:16:39.963 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-04-22T07:16:39.964 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-04-22T07:16:39.966 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-04-22T07:16:39.966 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-04-22T07:16:39.967 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-04-22T07:16:39.969 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-04-22T07:16:39.969 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-04-22T07:16:39.970 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-04-22T07:16:39.971 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-04-22T07:16:39.972 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-04-22T07:16:39.973 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-04-22T07:16:39.974 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-04-22T07:16:39.974 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-04-22T07:16:39.975 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-04-22T07:16:39.977 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-04-22T07:16:39.978 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-04-22T07:16:39.979 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-04-22T07:16:39.980 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-04-22T07:16:39.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-04-22T07:16:39.981 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-04-22T07:16:39.982 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-04-22T07:16:39.983 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-04-22T07:16:39.984 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-04-22T07:16:39.985 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-04-22T07:16:39.986 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-04-22T07:16:39.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-04-22T07:16:39.987 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-04-22T07:16:39.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-04-22T07:16:39.989 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-04-22T07:16:39.990 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-04-22T07:16:39.991 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-04-22T07:16:39.992 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-04-22T07:16:39.993 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sparse.txt 2024-04-22T07:16:39.994 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/ 2024-04-22T07:16:39.995 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-04-22T07:16:39.996 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-04-22T07:16:39.997 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-04-22T07:16:39.998 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-04-22T07:16:39.999 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-04-22T07:16:40.000 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-04-22T07:16:40.001 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-04-22T07:16:40.002 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-04-22T07:16:40.003 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-04-22T07:16:40.004 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/spidev 2024-04-22T07:16:40.005 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-04-22T07:16:40.006 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-04-22T07:16:40.007 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-04-22T07:16:40.008 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-04-22T07:16:40.009 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-04-22T07:16:40.010 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/svga.txt 2024-04-22T07:16:40.011 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/ 2024-04-22T07:16:40.012 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-04-22T07:16:40.013 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/README 2024-04-22T07:16:40.014 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-04-22T07:16:40.015 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-04-22T07:16:40.016 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-04-22T07:16:40.017 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-04-22T07:16:40.018 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-04-22T07:16:40.019 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-04-22T07:16:40.020 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-04-22T07:16:40.021 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-04-22T07:16:40.022 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/target/ 2024-04-22T07:16:40.023 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-04-22T07:16:40.024 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-04-22T07:16:40.026 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-04-22T07:16:40.027 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/ 2024-04-22T07:16:40.028 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-04-22T07:16:40.029 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-04-22T07:16:40.030 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-04-22T07:16:40.031 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-04-22T07:16:40.032 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-04-22T07:16:40.033 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-04-22T07:16:40.034 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-04-22T07:16:40.035 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-04-22T07:16:40.036 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/ 2024-04-22T07:16:40.037 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-04-22T07:16:40.038 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-04-22T07:16:40.039 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-04-22T07:16:40.040 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-04-22T07:16:40.041 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-04-22T07:16:40.042 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-04-22T07:16:40.043 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-04-22T07:16:40.044 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-04-22T07:16:40.045 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-04-22T07:16:40.046 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-04-22T07:16:40.047 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-04-22T07:16:40.048 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/tpm/ 2024-04-22T07:16:40.049 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-04-22T07:16:40.050 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/ 2024-04-22T07:16:40.051 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-04-22T07:16:40.052 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-04-22T07:16:40.053 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-04-22T07:16:40.053 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-04-22T07:16:40.055 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-04-22T07:16:40.056 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-04-22T07:16:40.057 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-04-22T07:16:40.058 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-04-22T07:16:40.060 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-04-22T07:16:40.061 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-04-22T07:16:40.062 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-04-22T07:16:40.063 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-04-22T07:16:40.064 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-04-22T07:16:40.065 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-04-22T07:16:40.066 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-04-22T07:16:40.067 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-04-22T07:16:40.068 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-04-22T07:16:40.069 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-04-22T07:16:40.070 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/unicode.txt 2024-04-22T07:16:40.071 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/unshare.txt 2024-04-22T07:16:40.072 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/ 2024-04-22T07:16:40.073 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-04-22T07:16:40.075 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-04-22T07:16:40.076 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-04-22T07:16:40.076 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-04-22T07:16:40.078 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-04-22T07:16:40.079 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-04-22T07:16:40.080 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-04-22T07:16:40.081 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-04-22T07:16:40.082 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-04-22T07:16:40.083 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-04-22T07:16:40.084 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-04-22T07:16:40.085 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-04-22T07:16:40.086 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-04-22T07:16:40.087 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-04-22T07:16:40.088 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-04-22T07:16:40.089 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-04-22T07:16:40.089 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-04-22T07:16:40.091 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-04-22T07:16:40.092 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-04-22T07:16:40.093 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-04-22T07:16:40.094 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-04-22T07:16:40.095 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-04-22T07:16:40.096 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-04-22T07:16:40.097 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-04-22T07:16:40.098 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-04-22T07:16:40.099 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-04-22T07:16:40.100 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-04-22T07:16:40.101 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-04-22T07:16:40.102 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-04-22T07:16:40.103 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-04-22T07:16:40.104 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-04-22T07:16:40.105 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-04-22T07:16:40.105 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-04-22T07:16:40.106 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-04-22T07:16:40.108 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-04-22T07:16:40.109 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-04-22T07:16:40.110 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vDSO/ 2024-04-22T07:16:40.111 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-04-22T07:16:40.112 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-04-22T07:16:40.113 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-04-22T07:16:40.114 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-04-22T07:16:40.115 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-04-22T07:16:40.116 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vfio.txt 2024-04-22T07:16:40.117 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-04-22T07:16:40.118 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video-output.txt 2024-04-22T07:16:40.119 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/ 2024-04-22T07:16:40.120 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-04-22T07:16:40.121 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-04-22T07:16:40.122 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-04-22T07:16:40.123 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-04-22T07:16:40.124 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-04-22T07:16:40.125 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-04-22T07:16:40.125 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-04-22T07:16:40.127 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-04-22T07:16:40.128 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-04-22T07:16:40.128 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-04-22T07:16:40.129 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-04-22T07:16:40.134 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-04-22T07:16:40.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-04-22T07:16:40.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-04-22T07:16:40.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-04-22T07:16:40.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-04-22T07:16:40.135 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-04-22T07:16:40.136 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-04-22T07:16:40.137 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-04-22T07:16:40.138 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-04-22T07:16:40.139 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-04-22T07:16:40.139 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-04-22T07:16:40.140 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-04-22T07:16:40.142 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-04-22T07:16:40.142 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-04-22T07:16:40.143 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-04-22T07:16:40.144 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-04-22T07:16:40.145 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-04-22T07:16:40.146 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-04-22T07:16:40.146 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-04-22T07:16:40.147 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-04-22T07:16:40.149 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-04-22T07:16:40.150 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-04-22T07:16:40.150 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-04-22T07:16:40.152 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-04-22T07:16:40.153 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-04-22T07:16:40.153 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-04-22T07:16:40.154 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-04-22T07:16:40.155 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-04-22T07:16:40.156 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-04-22T07:16:40.157 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-04-22T07:16:40.158 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-04-22T07:16:40.158 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-04-22T07:16:40.159 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-04-22T07:16:40.160 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-04-22T07:16:40.161 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-04-22T07:16:40.162 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-04-22T07:16:40.163 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-04-22T07:16:40.164 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-04-22T07:16:40.165 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-04-22T07:16:40.166 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-04-22T07:16:40.166 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-04-22T07:16:40.168 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-04-22T07:16:40.168 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-04-22T07:16:40.169 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-04-22T07:16:40.170 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-04-22T07:16:40.171 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-04-22T07:16:40.173 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-04-22T07:16:40.174 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-04-22T07:16:40.175 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-04-22T07:16:40.176 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-04-22T07:16:40.177 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-04-22T07:16:40.178 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-04-22T07:16:40.178 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-04-22T07:16:40.180 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-04-22T07:16:40.180 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-04-22T07:16:40.182 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-04-22T07:16:40.183 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-04-22T07:16:40.184 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-04-22T07:16:40.185 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-04-22T07:16:40.186 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-04-22T07:16:40.187 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-04-22T07:16:40.188 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-04-22T07:16:40.189 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-04-22T07:16:40.189 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-04-22T07:16:40.191 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-04-22T07:16:40.191 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-04-22T07:16:40.193 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-04-22T07:16:40.193 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/ 2024-04-22T07:16:40.195 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-04-22T07:16:40.195 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-04-22T07:16:40.196 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-04-22T07:16:40.197 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-04-22T07:16:40.199 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-04-22T07:16:40.200 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-04-22T07:16:40.200 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-04-22T07:16:40.201 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-04-22T07:16:40.202 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-04-22T07:16:40.203 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-04-22T07:16:40.204 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-04-22T07:16:40.205 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-04-22T07:16:40.206 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-04-22T07:16:40.207 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-04-22T07:16:40.208 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-04-22T07:16:40.209 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-04-22T07:16:40.210 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-04-22T07:16:40.211 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-04-22T07:16:40.212 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-04-22T07:16:40.213 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-04-22T07:16:40.214 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-04-22T07:16:40.215 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-04-22T07:16:40.216 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-04-22T07:16:40.217 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-04-22T07:16:40.218 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-04-22T07:16:40.220 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/ 2024-04-22T07:16:40.221 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-04-22T07:16:40.221 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-04-22T07:16:40.222 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-04-22T07:16:40.223 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/balance 2024-04-22T07:16:40.224 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-04-22T07:16:40.225 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-04-22T07:16:40.227 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-04-22T07:16:40.228 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-04-22T07:16:40.229 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-04-22T07:16:40.230 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-04-22T07:16:40.231 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/numa 2024-04-22T07:16:40.232 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-04-22T07:16:40.233 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-04-22T07:16:40.235 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-04-22T07:16:40.235 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-04-22T07:16:40.236 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-04-22T07:16:40.237 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-04-22T07:16:40.238 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-04-22T07:16:40.239 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-04-22T07:16:40.240 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-04-22T07:16:40.241 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-04-22T07:16:40.242 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-04-22T07:16:40.243 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-04-22T07:16:40.244 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-04-22T07:16:40.245 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-04-22T07:16:40.246 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/ 2024-04-22T07:16:40.247 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-04-22T07:16:40.247 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-04-22T07:16:40.248 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-04-22T07:16:40.249 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-04-22T07:16:40.250 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-04-22T07:16:40.251 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-04-22T07:16:40.252 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-04-22T07:16:40.252 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-04-22T07:16:40.253 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-04-22T07:16:40.254 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-04-22T07:16:40.255 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-04-22T07:16:40.256 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-04-22T07:16:40.256 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-04-22T07:16:40.257 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-04-22T07:16:40.258 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-04-22T07:16:40.259 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-04-22T07:16:40.260 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/ 2024-04-22T07:16:40.260 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-04-22T07:16:40.261 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-04-22T07:16:40.262 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-04-22T07:16:40.263 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-04-22T07:16:40.264 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-04-22T07:16:40.265 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-04-22T07:16:40.266 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-04-22T07:16:40.267 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-04-22T07:16:40.267 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-04-22T07:16:40.268 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-04-22T07:16:40.269 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-04-22T07:16:40.270 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-04-22T07:16:40.271 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-04-22T07:16:40.272 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/wimax/ 2024-04-22T07:16:40.273 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-04-22T07:16:40.274 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-04-22T07:16:40.275 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-04-22T07:16:40.276 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/ 2024-04-22T07:16:40.277 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-04-22T07:16:40.278 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-04-22T07:16:40.279 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-04-22T07:16:40.279 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-04-22T07:16:40.280 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-04-22T07:16:40.281 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-04-22T07:16:40.282 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-04-22T07:16:40.283 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-04-22T07:16:40.284 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-04-22T07:16:40.285 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-04-22T07:16:40.286 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-04-22T07:16:40.287 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-04-22T07:16:40.288 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-04-22T07:16:40.289 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-04-22T07:16:40.290 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-04-22T07:16:40.291 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-04-22T07:16:40.292 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-04-22T07:16:40.293 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-04-22T07:16:40.295 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-04-22T07:16:40.296 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-04-22T07:16:40.297 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-04-22T07:16:40.298 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-04-22T07:16:40.299 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-04-22T07:16:40.300 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-04-22T07:16:40.302 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/xtensa/ 2024-04-22T07:16:40.303 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-04-22T07:16:40.304 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-04-22T07:16:40.304 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/xz.txt 2024-04-22T07:16:40.306 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-04-22T07:16:40.307 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-04-22T07:16:40.308 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-04-22T07:16:40.309 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-04-22T07:16:40.310 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-04-22T07:16:40.311 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-04-22T07:16:40.312 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-04-22T07:16:40.313 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-04-22T07:16:40.314 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-04-22T07:16:40.314 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-04-22T07:16:40.315 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-04-22T07:16:40.316 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-04-22T07:16:40.318 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-04-22T07:16:40.319 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-04-22T07:16:40.320 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-04-22T07:16:40.322 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-04-22T07:16:40.322 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-04-22T07:16:40.323 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-04-22T07:16:40.324 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-04-22T07:16:40.324 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-04-22T07:16:40.325 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-04-22T07:16:40.326 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-04-22T07:16:40.328 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-04-22T07:16:40.329 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-04-22T07:16:40.329 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-04-22T07:16:40.331 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-04-22T07:16:40.332 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-04-22T07:16:40.332 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-04-22T07:16:40.333 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-04-22T07:16:40.334 INFO:tasks.workunit.client.0.smithi113.stdout:linux-4.0.5/Documentation/zorro.txt 2024-04-22T07:16:41.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:41 smithi154 ceph-mon[31862]: pgmap v60: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 6.8 MiB/s wr, 29 op/s 2024-04-22T07:16:41.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:41 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:41.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:41 smithi133 ceph-mon[31198]: pgmap v60: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 6.8 MiB/s wr, 29 op/s 2024-04-22T07:16:41.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:41 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:41.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:41 smithi113 ceph-mon[27933]: pgmap v60: 97 pgs: 97 active+clean; 83 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 6.8 MiB/s wr, 29 op/s 2024-04-22T07:16:41.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:41 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:16:42.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:42 smithi133 ceph-mon[31198]: pgmap v61: 97 pgs: 97 active+clean; 89 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 7.3 MiB/s wr, 42 op/s 2024-04-22T07:16:42.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:42 smithi154 ceph-mon[31862]: pgmap v61: 97 pgs: 97 active+clean; 89 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 7.3 MiB/s wr, 42 op/s 2024-04-22T07:16:42.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:42 smithi113 ceph-mon[27933]: pgmap v61: 97 pgs: 97 active+clean; 89 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 7.3 MiB/s wr, 42 op/s 2024-04-22T07:16:44.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:44 smithi133 ceph-mon[31198]: pgmap v62: 97 pgs: 97 active+clean; 89 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 7.3 MiB/s wr, 38 op/s 2024-04-22T07:16:44.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:44 smithi154 ceph-mon[31862]: pgmap v62: 97 pgs: 97 active+clean; 89 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 7.3 MiB/s wr, 38 op/s 2024-04-22T07:16:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:44 smithi113 ceph-mon[27933]: pgmap v62: 97 pgs: 97 active+clean; 89 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 7.3 MiB/s wr, 38 op/s 2024-04-22T07:16:45.162 INFO:tasks.workunit.client.0.smithi113.stderr:+ tar cf doc.tar linux-4.0.5 2024-04-22T07:16:46.214 INFO:tasks.workunit.client.0.smithi113.stderr:+ tar xf doc.tar 2024-04-22T07:16:46.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:46 smithi133 ceph-mon[31198]: pgmap v63: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 7.6 MiB/s wr, 58 op/s 2024-04-22T07:16:46.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:46 smithi154 ceph-mon[31862]: pgmap v63: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 7.6 MiB/s wr, 58 op/s 2024-04-22T07:16:46.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:46 smithi113 ceph-mon[27933]: pgmap v63: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 7.6 MiB/s wr, 58 op/s 2024-04-22T07:16:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:48 smithi133 ceph-mon[31198]: pgmap v64: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 800 KiB/s wr, 32 op/s 2024-04-22T07:16:48.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:48 smithi154 ceph-mon[31862]: pgmap v64: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 800 KiB/s wr, 32 op/s 2024-04-22T07:16:48.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:48 smithi113 ceph-mon[27933]: pgmap v64: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 800 KiB/s wr, 32 op/s 2024-04-22T07:16:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:50 smithi133 ceph-mon[31198]: pgmap v65: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 800 KiB/s wr, 32 op/s 2024-04-22T07:16:50.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:50 smithi154 ceph-mon[31862]: pgmap v65: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 800 KiB/s wr, 32 op/s 2024-04-22T07:16:50.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:50 smithi113 ceph-mon[27933]: pgmap v65: 97 pgs: 97 active+clean; 91 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 800 KiB/s wr, 32 op/s 2024-04-22T07:16:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:52 smithi133 ceph-mon[31198]: pgmap v66: 97 pgs: 97 active+clean; 143 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 418 op/s 2024-04-22T07:16:52.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:52 smithi154 ceph-mon[31862]: pgmap v66: 97 pgs: 97 active+clean; 143 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 418 op/s 2024-04-22T07:16:52.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:52 smithi113 ceph-mon[27933]: pgmap v66: 97 pgs: 97 active+clean; 143 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 418 op/s 2024-04-22T07:16:54.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:54 smithi133 ceph-mon[31198]: pgmap v67: 97 pgs: 97 active+clean; 143 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.8 MiB/s wr, 406 op/s 2024-04-22T07:16:54.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:54 smithi154 ceph-mon[31862]: pgmap v67: 97 pgs: 97 active+clean; 143 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.8 MiB/s wr, 406 op/s 2024-04-22T07:16:54.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:54 smithi113 ceph-mon[27933]: pgmap v67: 97 pgs: 97 active+clean; 143 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 4.8 MiB/s wr, 406 op/s 2024-04-22T07:16:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:56 smithi133 ceph-mon[31198]: pgmap v68: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 493 op/s 2024-04-22T07:16:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:16:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:56 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:56 smithi154 ceph-mon[31862]: pgmap v68: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 493 op/s 2024-04-22T07:16:56.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:16:56.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:56 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:56 smithi113 ceph-mon[27933]: pgmap v68: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 493 op/s 2024-04-22T07:16:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:16:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:56 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:57.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:57 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:16:57.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:57 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:16:57.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:57 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:57.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:57 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:16:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:57 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:16:57.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:57 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:57 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:16:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:57 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:16:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:57 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:16:59.313 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:16:59 smithi133 ceph-mon[31198]: pgmap v69: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-04-22T07:16:59.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:16:59 smithi154 ceph-mon[31862]: pgmap v69: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-04-22T07:16:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:16:59 smithi113 ceph-mon[27933]: pgmap v69: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-04-22T07:17:01.119 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.1 2024-04-22T07:17:01.120 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:17:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 -- 172.21.15.113:0/246051615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 msgr2=0x7f197410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 --2- 172.21.15.113:0/246051615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197410ee30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 -- 172.21.15.113:0/246051615 shutdown_connections 2024-04-22T07:17:01.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 --2- 172.21.15.113:0/246051615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197410ee30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 --2- 172.21.15.113:0/246051615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102b40 0x7f1974106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 --2- 172.21.15.113:0/246051615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1974102190 0x7f1974102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 -- 172.21.15.113:0/246051615 >> 172.21.15.113:0/246051615 conn(0x7f19740793e0 msgr2=0x7f19740797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 -- 172.21.15.113:0/246051615 shutdown_connections 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 -- 172.21.15.113:0/246051615 wait complete. 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 Processor -- start 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.225+0000 7f197afef640 1 -- start start 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102190 0x7f197419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1974198620 0x7f1974198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112300 con 0x7f1974107260 2024-04-22T07:17:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112180 con 0x7f1974198620 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112480 con 0x7f1974102190 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43340/0 (socket says 172.21.15.113:43340) 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 -- 172.21.15.113:0/2121175339 learned_addr learned my addr 172.21.15.113:0/2121175339 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1978d8b640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102190 0x7f197419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197958c640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1974198620 0x7f1974198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 -- 172.21.15.113:0/2121175339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102190 msgr2=0x7f197419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102190 0x7f197419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 -- 172.21.15.113:0/2121175339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1974198620 msgr2=0x7f1974198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1974198620 0x7f1974198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 -- 172.21.15.113:0/2121175339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1974199140 con 0x7f1974107260 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1978d8b640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102190 0x7f197419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1973fff640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197419e9b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9f0 tx=0x7f196000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196000ccc0 con 0x7f1974107260 2024-04-22T07:17:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1974199430 con 0x7f1974107260 2024-04-22T07:17:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19600054f0 con 0x7f1974107260 2024-04-22T07:17:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.226+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960010690 con 0x7f1974107260 2024-04-22T07:17:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.227+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741a5680 con 0x7f1974107260 2024-04-22T07:17:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.227+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1974069230 con 0x7f1974107260 2024-04-22T07:17:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.229+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f19600046c0 con 0x7f1974107260 2024-04-22T07:17:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.229+0000 7f1971ffb640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f19640a3d20 0x7f19640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.229+0000 7f1978d8b640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f19640a3d20 0x7f19640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.230+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f19600d0ee0 con 0x7f1974107260 2024-04-22T07:17:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.230+0000 7f1978d8b640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f19640a3d20 0x7f19640a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f196800bfd0 tx=0x7f196800b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.233+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1960053ff0 con 0x7f1974107260 2024-04-22T07:17:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:01 smithi133 ceph-mon[31198]: pgmap v70: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-04-22T07:17:01.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:01 smithi154 ceph-mon[31862]: pgmap v70: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-04-22T07:17:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:01 smithi113 ceph-mon[27933]: pgmap v70: 97 pgs: 97 active+clean; 152 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-04-22T07:17:01.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.463+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f197410a550 con 0x7f1974107260 2024-04-22T07:17:01.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.464+0000 7f1971ffb640 1 -- 172.21.15.113:0/2121175339 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11890 (secure 0 0 0) 0x7f196009d500 con 0x7f1974107260 2024-04-22T07:17:01.465 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:17:01.465 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:16:39.519389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24445,"mds_2":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":26,"state":"up:active","state_seq":54,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":24,"state":"up:active","state_seq":53,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24433,24445,24506]},"id":1}]} 2024-04-22T07:17:01.467 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 34 2024-04-22T07:17:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f19640a3d20 msgr2=0x7f19640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f19640a3d20 0x7f19640a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f196800bfd0 tx=0x7f196800b390 comp rx=0 tx=0).stop 2024-04-22T07:17:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 msgr2=0x7f197419e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197419e9b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9f0 tx=0x7f196000eeb0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 shutdown_connections 2024-04-22T07:17:01.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f19640a3d20 0x7f19640a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1974198620 0x7f1974198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1974107260 0x7f197419e9b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 --2- 172.21.15.113:0/2121175339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1974102190 0x7f197419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 >> 172.21.15.113:0/2121175339 conn(0x7f19740793e0 msgr2=0x7f19740ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:01.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 shutdown_connections 2024-04-22T07:17:01.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.466+0000 7f197afef640 1 -- 172.21.15.113:0/2121175339 wait complete. 2024-04-22T07:17:01.475 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 84.1 --await' 2024-04-22T07:17:01.475 DEBUG:teuthology.orchestra.run.smithi113:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24506 quiesce db / --timeout 60 --expiration 84.1 --await 2024-04-22T07:17:01.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 -- 172.21.15.113:0/303621773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff154101a50 msgr2=0x7ff154105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 --2- 172.21.15.113:0/303621773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff154101a50 0x7ff154105a90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff140009900 tx=0x7ff14002bc40 comp rx=0 tx=0).stop 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 -- 172.21.15.113:0/303621773 shutdown_connections 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 --2- 172.21.15.113:0/303621773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 0x7ff15410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 --2- 172.21.15.113:0/303621773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff154101a50 0x7ff154105a90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 --2- 172.21.15.113:0/303621773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.592+0000 7ff15ac55640 1 -- 172.21.15.113:0/303621773 >> 172.21.15.113:0/303621773 conn(0x7ff154077f50 msgr2=0x7ff154078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.593+0000 7ff15ac55640 1 -- 172.21.15.113:0/303621773 shutdown_connections 2024-04-22T07:17:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.593+0000 7ff15ac55640 1 -- 172.21.15.113:0/303621773 wait complete. 2024-04-22T07:17:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.593+0000 7ff15ac55640 1 Processor -- start 2024-04-22T07:17:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.593+0000 7ff15ac55640 1 -- start start 2024-04-22T07:17:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff15ac55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff15ac55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 0x7ff154196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff15ac55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff15419aeb0 0x7ff15419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff15ac55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff154110100 con 0x7ff1541010a0 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff15ac55640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff15410ff80 con 0x7ff154106170 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff15ac55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff154110280 con 0x7ff15419aeb0 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff1589f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff1589f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43364/0 (socket says 172.21.15.113:43364) 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff1589f1640 1 -- 172.21.15.113:0/1474521362 learned_addr learned my addr 172.21.15.113:0/1474521362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.594+0000 7ff153fff640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 0x7ff154196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff153fff640 1 -- 172.21.15.113:0/1474521362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff15419aeb0 msgr2=0x7ff15419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff153fff640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff15419aeb0 0x7ff15419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff153fff640 1 -- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 msgr2=0x7ff154196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff153fff640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff153fff640 1 -- 172.21.15.113:0/1474521362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15419b9a0 con 0x7ff154106170 2024-04-22T07:17:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff1589f1640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff153fff640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 0x7ff154196b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff140036040 tx=0x7ff14002ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff151ffb640 1 -- 172.21.15.113:0/1474521362 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140041070 con 0x7ff154106170 2024-04-22T07:17:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.595+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15419bc30 con 0x7ff154106170 2024-04-22T07:17:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.596+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1541a33f0 con 0x7ff154106170 2024-04-22T07:17:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.596+0000 7ff151ffb640 1 -- 172.21.15.113:0/1474521362 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff140004450 con 0x7ff154106170 2024-04-22T07:17:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.596+0000 7ff151ffb640 1 -- 172.21.15.113:0/1474521362 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14004a060 con 0x7ff154106170 2024-04-22T07:17:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 Processor -- start 2024-04-22T07:17:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 -- start start 2024-04-22T07:17:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff151ffb640 1 -- 172.21.15.113:0/1474521362 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff1400035f0 con 0x7ff154106170 2024-04-22T07:17:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546db9c0 0x5582546dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 0x5582546e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55825463c0b0 con 0x5582546dc2e0 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582545ed680 con 0x5582546e4ec0 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.599+0000 7ff15d84a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5582545f9c60 con 0x5582546db9c0 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 0x5582546e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1591f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546db9c0 0x5582546dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 0x5582546e4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43378/0 (socket says 172.21.15.113:43378) 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff153fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 -- 172.21.15.113:0/967171631 learned_addr learned my addr 172.21.15.113:0/967171631 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 -- 172.21.15.113:0/967171631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546db9c0 msgr2=0x5582546dbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546db9c0 0x5582546dbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 -- 172.21.15.113:0/967171631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 msgr2=0x5582546e72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff151ffb640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1440a8130 0x7ff1440aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1589f1640 1 -- 172.21.15.113:0/967171631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55825441ec90 con 0x5582546dc2e0 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff1591f2640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1440a8130 0x7ff1440aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.600+0000 7ff151ffb640 1 -- 172.21.15.113:0/1474521362 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7ff1400f2080 con 0x7ff154106170 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.601+0000 7ff1589f1640 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 0x5582546e4980 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff148011650 tx=0x7ff148009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae905bea54fe7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.601+0000 7ff13effd640 1 -- 172.21.15.113:0/967171631 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14801e070 con 0x5582546dc2e0 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.601+0000 7ff13effd640 1 -- 172.21.15.113:0/967171631 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff14800fab0 con 0x5582546dc2e0 2024-04-22T07:17:01.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.601+0000 7ff13effd640 1 -- 172.21.15.113:0/967171631 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148019c30 con 0x5582546dc2e0 2024-04-22T07:17:01.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 -- 172.21.15.113:0/967171631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 msgr2=0x5582546e4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 0x5582546e4980 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff148011650 tx=0x7ff148009f90 comp rx=0 tx=0).stop 2024-04-22T07:17:01.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff1591f2640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1440a8130 0x7ff1440aa5f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005fd0 tx=0x7ff14c0098e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 -- 172.21.15.113:0/967171631 shutdown_connections 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546dc2e0 0x5582546e4980 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 --2- 172.21.15.113:0/967171631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546db9c0 0x5582546dbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 -- 172.21.15.113:0/967171631 >> 172.21.15.113:0/967171631 conn(0x5582546d60f0 msgr2=0x5582546d64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 -- 172.21.15.113:0/967171631 shutdown_connections 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 -- 172.21.15.113:0/967171631 wait complete. 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 -- start start 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.602+0000 7ff15d84a740 1 Processor -- start 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55825463f400 con 0x5582546e4ec0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582545fb840 con 0x5582546db9c0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558254644720 con 0x5582546dc2e0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff15d84a740 10 client.0 _ll_register_callbacks cb 0x55825464cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43384/0 (socket says 172.21.15.113:43384) 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 -- 172.21.15.113:0/3420586735 learned_addr learned my addr 172.21.15.113:0/3420586735 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1589f1640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff153fff640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 -- 172.21.15.113:0/3420586735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 msgr2=0x5582546e2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 -- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 msgr2=0x5582546e1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546e1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.603+0000 7ff1591f2640 1 -- 172.21.15.113:0/3420586735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55825447f420 con 0x5582546e4ec0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff1591f2640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e2a40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009aa0 tx=0x7ff14c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f15b7b099470db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff133fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff133fff640 1 -- 172.21.15.113:0/3420586735 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c029060 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff133fff640 1 -- 172.21.15.113:0/3420586735 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7ff14c01cb60 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14800fab0 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff133fff640 1 -- 172.21.15.113:0/3420586735 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff14c004820 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff15d84a740 10 client.16140 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558254312580 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.604+0000 7ff133fff640 1 -- 172.21.15.113:0/3420586735 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff14800fab0 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff133fff640 1 -- 172.21.15.113:0/3420586735 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff14c018040 con 0x5582546e4ec0 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff15d84a740 10 client.16140 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff15d84a740 10 client.16140 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff15d84a740 20 client.16140 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x5582546f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff15d84a740 4 client.16140 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:01.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55825464db10 con 0x5582546f4d70 2024-04-22T07:17:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.605+0000 7ff1589f1640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x5582546f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.606+0000 7ff1589f1640 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x5582546f71d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3afcf6d1ff6d8e82 server_cookie=630eb0bb26b074e8 in_seq=0 out_seq=0 2024-04-22T07:17:01.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.606+0000 7ff133fff640 10 client.16140 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff133fff640 1 -- 172.21.15.113:0/3420586735 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55825464db10 con 0x5582546f4d70 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff133fff640 10 client.16140 handle_command_reply: tid=0 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 client.16140 shutdown 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 msgr2=0x5582546e2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e2a40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009aa0 tx=0x7ff14c004a40 comp rx=0 tx=0).stop 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 shutdown_connections 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x5582546f71d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e2a40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff133fff640 0 client.16140 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3420586735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546e1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.607+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 >> 172.21.15.113:0/3420586735 conn(0x5582546d60f0 msgr2=0x5582546d7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.608+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 shutdown_connections 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.608+0000 7ff15d84a740 1 -- 172.21.15.113:0/3420586735 wait complete. 2024-04-22T07:17:01.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.608+0000 7ff15d84a740 20 client.16140 trim_cache size 0 max 16384 2024-04-22T07:17:01.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.636+0000 7ff15d84a740 1 Processor -- start 2024-04-22T07:17:01.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.636+0000 7ff15d84a740 1 -- start start 2024-04-22T07:17:01.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff15d84a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800e9d0 con 0x5582546e4ec0 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff15d84a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800c820 con 0x5582546db9c0 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff15d84a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800e300 con 0x5582546dc2e0 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43390/0 (socket says 172.21.15.113:43390) 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 -- 172.21.15.113:0/3017839675 learned_addr learned my addr 172.21.15.113:0/3017839675 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff153fff640 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1589f1640 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 -- 172.21.15.113:0/3017839675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 msgr2=0x5582546d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 -- 172.21.15.113:0/3017839675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 msgr2=0x5582546f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 -- 172.21.15.113:0/3017839675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582546dc110 con 0x5582546e4ec0 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff1591f2640 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e0890 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff14c016040 tx=0x7ff14c003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65c13fa74da51de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.637+0000 7ff13e7fc640 1 -- 172.21.15.113:0/3017839675 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c01cd60 con 0x5582546e4ec0 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff13e7fc640 1 -- 172.21.15.113:0/3017839675 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff14c0246e0 con 0x5582546e4ec0 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff13e7fc640 1 -- 172.21.15.113:0/3017839675 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c005280 con 0x5582546e4ec0 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 -- 172.21.15.113:0/3017839675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 msgr2=0x5582546e0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e0890 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff14c016040 tx=0x7ff14c003fe0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 -- 172.21.15.113:0/3017839675 shutdown_connections 2024-04-22T07:17:01.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546e4ec0 0x5582546e0890 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 --2- 172.21.15.113:0/3017839675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546db9c0 0x5582546f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 -- 172.21.15.113:0/3017839675 >> 172.21.15.113:0/3017839675 conn(0x5582546d60f0 msgr2=0x5582546d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 -- 172.21.15.113:0/3017839675 shutdown_connections 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.638+0000 7ff15d84a740 1 -- 172.21.15.113:0/3017839675 wait complete. 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 -- start start 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 Processor -- start 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 0x5582546ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800d560 con 0x5582546db9c0 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800ce90 con 0x5582546e4ec0 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800dc30 con 0x5582546dc2e0 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff1589f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 0x5582546ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff153fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff153fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:59226/0 (socket says 172.21.15.113:59226) 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff15d84a740 10 client.0 _ll_register_callbacks cb 0x7ff14c0099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff153fff640 1 -- 172.21.15.113:0/1347008110 learned_addr learned my addr 172.21.15.113:0/1347008110 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff1589f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 0x5582546ed0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43400/0 (socket says 172.21.15.113:43400) 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.639+0000 7ff1591f2640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff153fff640 1 -- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 msgr2=0x5582546e99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff153fff640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff153fff640 1 -- 172.21.15.113:0/1347008110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 msgr2=0x5582546ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff153fff640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 0x5582546ed0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff153fff640 1 -- 172.21.15.113:0/1347008110 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582546ea110 con 0x5582546dc2e0 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff1589f1640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 0x5582546ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff1591f2640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff153fff640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e9490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1400ef8c0 tx=0x7ff1400bdf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580a2020a15f8d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff133fff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-22T07:17:01.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff133fff640 1 -- 172.21.15.113:0/1347008110 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400be7b0 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff133fff640 1 -- 172.21.15.113:0/1347008110 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7ff1400c6890 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14c0246e0 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff133fff640 1 -- 172.21.15.113:0/1347008110 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff1400be0b0 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff15d84a740 10 client.25441 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55825475c230 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.640+0000 7ff133fff640 1 -- 172.21.15.113:0/1347008110 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff14c0246e0 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff133fff640 1 -- 172.21.15.113:0/1347008110 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff1400c20d0 con 0x5582546dc2e0 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff15d84a740 10 client.25441 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff15d84a740 10 client.25441 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff15d84a740 20 client.25441 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff15d84a740 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x55825475ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff15d84a740 4 client.25441 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.1, "await": true}] 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.1, "await": true}) -- 0x558254647980 con 0x5582546f4d70 2024-04-22T07:17:01.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff1589f1640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x55825475ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:01.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff1589f1640 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x55825475ea40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e05e11e2ff21c4 server_cookie=2cd8fe4200d5f661 in_seq=0 out_seq=0 2024-04-22T07:17:01.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:01.641+0000 7ff133fff640 10 client.25441 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:02.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2121175339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:02.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2121175339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2121175339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:03.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:03 smithi133 ceph-mon[31198]: pgmap v71: 97 pgs: 97 active+clean; 157 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-04-22T07:17:03.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:03 smithi154 ceph-mon[31862]: pgmap v71: 97 pgs: 97 active+clean; 157 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-04-22T07:17:03.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:03 smithi113 ceph-mon[27933]: pgmap v71: 97 pgs: 97 active+clean; 157 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-04-22T07:17:03.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:17:03.766 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:17:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 -- 172.21.15.113:0/4215897927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70100fd0 msgr2=0x7f0d7010d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- 172.21.15.113:0/4215897927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70100fd0 0x7f0d7010d440 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800adb0 tx=0x7f0d6802cff0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 -- 172.21.15.113:0/4215897927 shutdown_connections 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- 172.21.15.113:0/4215897927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70100fd0 0x7f0d7010d440 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- 172.21.15.113:0/4215897927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70100610 0x7f0d70100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- 172.21.15.113:0/4215897927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d70106680 0x7f0d70106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 -- 172.21.15.113:0/4215897927 >> 172.21.15.113:0/4215897927 conn(0x7f0d70077360 msgr2=0x7f0d700fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 -- 172.21.15.113:0/4215897927 shutdown_connections 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 -- 172.21.15.113:0/4215897927 wait complete. 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 Processor -- start 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 -- start start 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70100610 0x7f0d70198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d70100fd0 0x7f0d70198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.881+0000 7f0d78031640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 0x7f0d7019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d78031640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701100c0 con 0x7f0d70106680 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d78031640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010ff40 con 0x7f0d70100610 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d78031640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70110240 con 0x7f0d70100fd0 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 0x7f0d7019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 0x7f0d7019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43404/0 (socket says 172.21.15.113:43404) 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 -- 172.21.15.113:0/72084800 learned_addr learned my addr 172.21.15.113:0/72084800 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70100610 0x7f0d70198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d755cc640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d70100fd0 0x7f0d70198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 -- 172.21.15.113:0/72084800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d70100fd0 msgr2=0x7f0d70198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d70100fd0 0x7f0d70198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 -- 172.21.15.113:0/72084800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70100610 msgr2=0x7f0d70198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70100610 0x7f0d70198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 -- 172.21.15.113:0/72084800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7019d730 con 0x7f0d70106680 2024-04-22T07:17:03.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d765ce640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 0x7f0d7019d000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0d6802d600 tx=0x7f0d6800a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d66ffd640 1 -- 172.21.15.113:0/72084800 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6803a040 con 0x7f0d70106680 2024-04-22T07:17:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d66ffd640 1 -- 172.21.15.113:0/72084800 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d68043070 con 0x7f0d70106680 2024-04-22T07:17:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7019d9c0 con 0x7f0d70106680 2024-04-22T07:17:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d66ffd640 1 -- 172.21.15.113:0/72084800 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680356a0 con 0x7f0d70106680 2024-04-22T07:17:03.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.882+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a52b0 con 0x7f0d70106680 2024-04-22T07:17:03.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.884+0000 7f0d66ffd640 1 -- 172.21.15.113:0/72084800 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0d68004040 con 0x7f0d70106680 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 Processor -- start 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 -- start start 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d66ffd640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0d5c0a3d20 0x7f0d5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10cbe90 0x563bc10ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563bc10231e0 con 0x563bc10be2b0 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563bc0fc0390 con 0x563bc10bebd0 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563bc101f1a0 con 0x563bc10cbe90 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d755cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0d5c0a3d20 0x7f0d5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d66ffd640 1 -- 172.21.15.113:0/72084800 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f0d680f26c0 con 0x7f0d70106680 2024-04-22T07:17:03.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d755cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10be690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43420/0 (socket says 172.21.15.113:43420) 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d755cc640 1 -- 172.21.15.113:0/1432843169 learned_addr learned my addr 172.21.15.113:0/1432843169 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.885+0000 7f0d765ce640 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10cbe90 0x563bc10ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 -- 172.21.15.113:0/1432843169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10cbe90 msgr2=0x563bc10ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10cbe90 0x563bc10ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 -- 172.21.15.113:0/1432843169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 msgr2=0x563bc10cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 -- 172.21.15.113:0/1432843169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc0dff7d0 con 0x563bc10be2b0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0d5c0a3d20 0x7f0d5c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005fd0 tx=0x7f0d6c005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d755cc640 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10be690 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0d600058c0 tx=0x7f0d6000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ac10633c2103b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d5bfff640 1 -- 172.21.15.113:0/1432843169 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60019030 con 0x563bc10be2b0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d5bfff640 1 -- 172.21.15.113:0/1432843169 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d60011990 con 0x563bc10be2b0 2024-04-22T07:17:03.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.886+0000 7f0d5bfff640 1 -- 172.21.15.113:0/1432843169 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6001d8f0 con 0x563bc10be2b0 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1432843169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 msgr2=0x563bc10be690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10be690 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0d600058c0 tx=0x7f0d6000b6b0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1432843169 shutdown_connections 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10cbe90 0x563bc10ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1432843169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10be690 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1432843169 >> 172.21.15.113:0/1432843169 conn(0x563bc10b8cb0 msgr2=0x563bc10b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1432843169 shutdown_connections 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1432843169 wait complete. 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 -- start start 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.887+0000 7f0d7ac27740 1 Processor -- start 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 0x563bc10c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563bc10cec50 con 0x563bc10cbe90 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563bc0fdc2f0 con 0x563bc10bebd0 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563bc10cedd0 con 0x563bc10be2b0 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 10 client.0 _ll_register_callbacks cb 0x563bc0fe9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 0x563bc10c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 0x563bc10c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43430/0 (socket says 172.21.15.113:43430) 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 -- 172.21.15.113:0/1500844941 learned_addr learned my addr 172.21.15.113:0/1500844941 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d755cc640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 -- 172.21.15.113:0/1500844941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 msgr2=0x563bc10c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 -- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 msgr2=0x563bc10c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 -- 172.21.15.113:0/1500844941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc0d353f0 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d765ce640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 0x563bc10c98b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0d680bb0c0 tx=0x7f0d680be2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bca9a453588f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d597fa640 1 -- 172.21.15.113:0/1500844941 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6802dda0 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d597fa640 1 -- 172.21.15.113:0/1500844941 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f0d680bad60 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.888+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d60011990 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d597fa640 1 -- 172.21.15.113:0/1500844941 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d68003b70 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 10 client.16158 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563bc0cf50f0 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d597fa640 1 -- 172.21.15.113:0/1500844941 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d60011990 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d597fa640 1 -- 172.21.15.113:0/1500844941 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0d6803d070 con 0x563bc10cbe90 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 10 client.16158 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 20 client.16158 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:03.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc10dbd90 0x563bc10de1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 4 client.16158 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc1030c00 con 0x563bc10dbd90 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.889+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc10dbd90 0x563bc10de1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.890+0000 7f0d597fa640 10 client.16158 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.890+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc10dbd90 0x563bc10de1d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb864f386f98081 server_cookie=6e1866334f5b9e90 in_seq=0 out_seq=0 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.890+0000 7f0d597fa640 1 -- 172.21.15.113:0/1500844941 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bc1030c00 con 0x563bc10dbd90 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.890+0000 7f0d597fa640 10 client.16158 handle_command_reply: tid=0 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.890+0000 7f0d7ac27740 1 client.16158 shutdown 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 msgr2=0x563bc10c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 0x563bc10c98b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0d680bb0c0 tx=0x7f0d680be2f0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 shutdown_connections 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc10dbd90 0x563bc10de1d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10cbe90 0x563bc10c98b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10bebd0 0x563bc10c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/1500844941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d597fa640 0 client.16158 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:03.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 >> 172.21.15.113:0/1500844941 conn(0x563bc10b8cb0 msgr2=0x563bc10baa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:03.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 shutdown_connections 2024-04-22T07:17:03.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 1 -- 172.21.15.113:0/1500844941 wait complete. 2024-04-22T07:17:03.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.891+0000 7f0d7ac27740 20 client.16158 trim_cache size 0 max 16384 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 Processor -- start 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 -- start start 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10bebd0 0x563bc10d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc10c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ae340 con 0x563bc10be2b0 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701abb50 con 0x563bc10cbe90 2024-04-22T07:17:03.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701a99c0 con 0x563bc10bebd0 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d75dcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d75dcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43438/0 (socket says 172.21.15.113:43438) 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.908+0000 7f0d75dcd640 1 -- 172.21.15.113:0/973852295 learned_addr learned my addr 172.21.15.113:0/973852295 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d75dcd640 1 -- 172.21.15.113:0/973852295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10bebd0 msgr2=0x563bc10d7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10bebd0 0x563bc10d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d765ce640 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc10c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d75dcd640 1 -- 172.21.15.113:0/973852295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 msgr2=0x563bc10c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc10c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d75dcd640 1 -- 172.21.15.113:0/973852295 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc10de850 con 0x563bc10be2b0 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d765ce640 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc10c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10cb560 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00fea0 tx=0x7f0d6c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851e253de7c7905d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:03.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d5b7fe640 1 -- 172.21.15.113:0/973852295 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c02d070 con 0x563bc10be2b0 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d5b7fe640 1 -- 172.21.15.113:0/973852295 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d6c01f430 con 0x563bc10be2b0 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d5b7fe640 1 -- 172.21.15.113:0/973852295 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c01e680 con 0x563bc10be2b0 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 -- 172.21.15.113:0/973852295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 msgr2=0x563bc10cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10cb560 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00fea0 tx=0x7f0d6c015000 comp rx=0 tx=0).stop 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 -- 172.21.15.113:0/973852295 shutdown_connections 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc10c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10bebd0 0x563bc10d7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/973852295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10be2b0 0x563bc10cb560 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.909+0000 7f0d7ac27740 1 -- 172.21.15.113:0/973852295 >> 172.21.15.113:0/973852295 conn(0x563bc10b8cb0 msgr2=0x563bc10b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 -- 172.21.15.113:0/973852295 shutdown_connections 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 -- 172.21.15.113:0/973852295 wait complete. 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 -- start start 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 Processor -- start 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 0x563bc10c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc1141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701aa010 con 0x563bc10bebd0 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701ab480 con 0x563bc10cbe90 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701aa6e0 con 0x563bc10be2b0 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d7ac27740 10 client.0 _ll_register_callbacks cb 0x7f0d6800aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d755cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 0x563bc10c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d755cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 0x563bc10c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43452/0 (socket says 172.21.15.113:43452) 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.910+0000 7f0d755cc640 1 -- 172.21.15.113:0/4009419796 learned_addr learned my addr 172.21.15.113:0/4009419796 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d765ce640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc1141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d755cc640 1 -- 172.21.15.113:0/4009419796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 msgr2=0x563bc10c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d755cc640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d755cc640 1 -- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 msgr2=0x563bc1141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d755cc640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc1141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d755cc640 1 -- 172.21.15.113:0/4009419796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc10b8ae0 con 0x563bc10bebd0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d755cc640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 0x563bc10c6760 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x563bc10c8690 tx=0x7f0d6000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33165cf53609372b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d597fa640 1 -- 172.21.15.113:0/4009419796 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60019050 con 0x563bc10bebd0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d597fa640 1 -- 172.21.15.113:0/4009419796 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f0d60005450 con 0x563bc10bebd0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c01f430 con 0x563bc10bebd0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d597fa640 1 -- 172.21.15.113:0/4009419796 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d6000b410 con 0x563bc10bebd0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 10 client.16164 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563bc11428d0 con 0x563bc10bebd0 2024-04-22T07:17:03.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d597fa640 1 -- 172.21.15.113:0/4009419796 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0d6c01f430 con 0x563bc10bebd0 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d597fa640 1 -- 172.21.15.113:0/4009419796 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0d60026020 con 0x563bc10bebd0 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 10 client.16164 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 20 client.16164 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc1142f00 0x563bc1145320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 4 client.16164 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.911+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bc102aa70 con 0x563bc1142f00 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc1142f00 0x563bc1145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d597fa640 10 client.16164 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d75dcd640 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc1142f00 0x563bc1145320 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda1e0de26e0b21e server_cookie=1b2533fbe25b5857 in_seq=0 out_seq=0 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d597fa640 1 -- 172.21.15.113:0/4009419796 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563bc102aa70 con 0x563bc1142f00 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d597fa640 10 client.16164 handle_command_reply: tid=0 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 client.16164 shutdown 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 msgr2=0x563bc10c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 0x563bc10c6760 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x563bc10c8690 tx=0x7f0d6000beb0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 shutdown_connections 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc10cbe90 0x563bc1141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x563bc1142f00 0x563bc1145320 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563bc10bebd0 0x563bc10c6760 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.912+0000 7f0d7ac27740 1 --2- 172.21.15.113:0/4009419796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563bc10be2b0 0x563bc10c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d597fa640 0 client.16164 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 >> 172.21.15.113:0/4009419796 conn(0x563bc10b8cb0 msgr2=0x563bc10b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 shutdown_connections 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d7ac27740 1 -- 172.21.15.113:0/4009419796 wait complete. 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d7ac27740 20 client.16164 trim_cache size 0 max 16384 2024-04-22T07:17:03.917 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:17:03.918 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:17:03.918 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:17:03.918 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0d5c0a3d20 msgr2=0x7f0d5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0d5c0a3d20 0x7f0d5c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005fd0 tx=0x7f0d6c005850 comp rx=0 tx=0).stop 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 msgr2=0x7f0d7019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 0x7f0d7019d000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0d6802d600 tx=0x7f0d6800a4e0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 shutdown_connections 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0d5c0a3d20 0x7f0d5c0a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d70106680 0x7f0d7019d000 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d70100fd0 0x7f0d70198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 --2- 172.21.15.113:0/72084800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70100610 0x7f0d70198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 >> 172.21.15.113:0/72084800 conn(0x7f0d70077360 msgr2=0x7f0d701049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 shutdown_connections 2024-04-22T07:17:03.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:03.913+0000 7f0d78031640 1 -- 172.21.15.113:0/72084800 wait complete. 2024-04-22T07:17:03.921 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:17:03.921 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:17:04.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 -- 172.21.15.113:0/1922813236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 msgr2=0x7f75d810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 --2- 172.21.15.113:0/1922813236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d810e9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009ca0 tx=0x7f75d002c180 comp rx=0 tx=0).stop 2024-04-22T07:17:04.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 -- 172.21.15.113:0/1922813236 shutdown_connections 2024-04-22T07:17:04.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 --2- 172.21.15.113:0/1922813236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d810e9d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 --2- 172.21.15.113:0/1922813236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d8103af0 0x7f75d810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 --2- 172.21.15.113:0/1922813236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 0x7f75d81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 -- 172.21.15.113:0/1922813236 >> 172.21.15.113:0/1922813236 conn(0x7f75d80771d0 msgr2=0x7f75d80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 -- 172.21.15.113:0/1922813236 shutdown_connections 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.021+0000 7f75dccfd640 1 -- 172.21.15.113:0/1922813236 wait complete. 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 Processor -- start 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 -- start start 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 0x7f75d819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d8103af0 0x7f75d819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8112300 con 0x7f75d810c5a0 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8112180 con 0x7f75d81031d0 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75dccfd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8112480 con 0x7f75d8103af0 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d81985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43470/0 (socket says 172.21.15.113:43470) 2024-04-22T07:17:04.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75d6d9d640 1 -- 172.21.15.113:0/4145991751 learned_addr learned my addr 172.21.15.113:0/4145991751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75d659c640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 0x7f75d819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.022+0000 7f75d5d9b640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d8103af0 0x7f75d819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d6d9d640 1 -- 172.21.15.113:0/4145991751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d8103af0 msgr2=0x7f75d819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d6d9d640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d8103af0 0x7f75d819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d6d9d640 1 -- 172.21.15.113:0/4145991751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 msgr2=0x7f75d819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d6d9d640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 0x7f75d819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d6d9d640 1 -- 172.21.15.113:0/4145991751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d8198d60 con 0x7f75d810c5a0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d659c640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 0x7f75d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75d6d9d640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d81985e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f75d002c790 tx=0x7f75d0035e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0039040 con 0x7f75d810c5a0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f75d0044070 con 0x7f75d810c5a0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d8198ff0 con 0x7f75d810c5a0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0032940 con 0x7f75d810c5a0 2024-04-22T07:17:04.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.023+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81a52a0 con 0x7f75d810c5a0 2024-04-22T07:17:04.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.024+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8069230 con 0x7f75d810c5a0 2024-04-22T07:17:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.025+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f75d0032ae0 con 0x7f75d810c5a0 2024-04-22T07:17:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.026+0000 7f75c77fe640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75c00a3d20 0x7f75c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.026+0000 7f75d659c640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75c00a3d20 0x7f75c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.026+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f75d00f3440 con 0x7f75d810c5a0 2024-04-22T07:17:04.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.026+0000 7f75d659c640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75c00a3d20 0x7f75c00a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009b40 tx=0x7f75c8005e50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.029+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75d003e3a0 con 0x7f75d810c5a0 2024-04-22T07:17:04.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.245+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d8199ac0 con 0x7f75d810c5a0 2024-04-22T07:17:04.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.246+0000 7f75c77fe640 1 -- 172.21.15.113:0/4145991751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11890 (secure 0 0 0) 0x7f75d00bfb70 con 0x7f75d810c5a0 2024-04-22T07:17:04.244 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:17:04.244 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:16:39.519389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24445,"mds_2":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":26,"state":"up:active","state_seq":54,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":24,"state":"up:active","state_seq":53,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24433,24445,24506]},"id":1}]} 2024-04-22T07:17:04.245 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 34 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75c00a3d20 msgr2=0x7f75c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75c00a3d20 0x7f75c00a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009b40 tx=0x7f75c8005e50 comp rx=0 tx=0).stop 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 msgr2=0x7f75d81985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d81985e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f75d002c790 tx=0x7f75d0035e40 comp rx=0 tx=0).stop 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 shutdown_connections 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75c00a3d20 0x7f75c00a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75d810c5a0 0x7f75d81985e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75d8103af0 0x7f75d819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 --2- 172.21.15.113:0/4145991751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75d81031d0 0x7f75d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 >> 172.21.15.113:0/4145991751 conn(0x7f75d80771d0 msgr2=0x7f75d80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 shutdown_connections 2024-04-22T07:17:04.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.248+0000 7f75dccfd640 1 -- 172.21.15.113:0/4145991751 wait complete. 2024-04-22T07:17:04.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.359+0000 7f98f88d9640 1 -- 172.21.15.113:0/3169014364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 msgr2=0x7f98f410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.359+0000 7f98f88d9640 1 --2- 172.21.15.113:0/3169014364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f410dc90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009ca0 tx=0x7f98ec02c140 comp rx=0 tx=0).stop 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.359+0000 7f98f88d9640 1 -- 172.21.15.113:0/3169014364 shutdown_connections 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 --2- 172.21.15.113:0/3169014364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f410dc90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 --2- 172.21.15.113:0/3169014364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f41015c0 0x7f98f4101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 --2- 172.21.15.113:0/3169014364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 0x7f98f4106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- 172.21.15.113:0/3169014364 >> 172.21.15.113:0/3169014364 conn(0x7f98f40773e0 msgr2=0x7f98f40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- 172.21.15.113:0/3169014364 shutdown_connections 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- 172.21.15.113:0/3169014364 wait complete. 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 Processor -- start 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- start start 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f41015c0 0x7f98f41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 0x7f98f419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98f4112410 con 0x7f98f4101f60 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98f4112290 con 0x7f98f4105e20 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.360+0000 7f98f88d9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98f4112590 con 0x7f98f41015c0 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f41a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43494/0 (socket says 172.21.15.113:43494) 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 -- 172.21.15.113:0/990469184 learned_addr learned my addr 172.21.15.113:0/990469184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98f259c640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f41015c0 0x7f98f41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 0x7f98f419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 -- 172.21.15.113:0/990469184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f41015c0 msgr2=0x7f98f41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f41015c0 0x7f98f41a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 -- 172.21.15.113:0/990469184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 msgr2=0x7f98f419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 0x7f98f419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 -- 172.21.15.113:0/990469184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f419aff0 con 0x7f98f4101f60 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 0x7f98f419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98ebfff640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f41a0c00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00d950 tx=0x7f98dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98eb7fe640 1 -- 172.21.15.113:0/990469184 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc017030 con 0x7f98f4101f60 2024-04-22T07:17:04.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f419b2e0 con 0x7f98f4101f60 2024-04-22T07:17:04.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98eb7fe640 1 -- 172.21.15.113:0/990469184 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98dc004020 con 0x7f98f4101f60 2024-04-22T07:17:04.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98eb7fe640 1 -- 172.21.15.113:0/990469184 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0047a0 con 0x7f98f4101f60 2024-04-22T07:17:04.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.361+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f41a7580 con 0x7f98f4101f60 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98eb7fe640 1 -- 172.21.15.113:0/990469184 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f98dc0041d0 con 0x7f98f4101f60 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 Processor -- start 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 -- start start 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98eb7fe640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f98e00a3d20 0x7f98e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c63430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c65e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5629f2bbf1f0 con 0x5629f2c63a00 2024-04-22T07:17:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5629f2b5c4b0 con 0x5629f2c5a4d0 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5629f2bbb1b0 con 0x5629f2c5ae80 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.364+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f98e00a3d20 0x7f98e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98eb7fe640 1 -- 172.21.15.113:0/990469184 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f98dc0d02d0 con 0x7f98f4101f60 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c65e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c65e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43508/0 (socket says 172.21.15.113:43508) 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 -- 172.21.15.113:0/3943513085 learned_addr learned my addr 172.21.15.113:0/3943513085 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f259c640 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98ebfff640 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c63430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 -- 172.21.15.113:0/3943513085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 msgr2=0x5629f2c63430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c63430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 -- 172.21.15.113:0/3943513085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 msgr2=0x5629f2c5a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 -- 172.21.15.113:0/3943513085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f2b4e1b0 con 0x5629f2c63a00 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f98e00a3d20 0x7f98e00a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036000 tx=0x7f98ec0324f0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c65e50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f98ec032780 tx=0x7f98ec045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1261def17670 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98e8ff9640 1 -- 172.21.15.113:0/3943513085 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec050070 con 0x5629f2c63a00 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98e8ff9640 1 -- 172.21.15.113:0/3943513085 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98ec03a040 con 0x5629f2c63a00 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.365+0000 7f98e8ff9640 1 -- 172.21.15.113:0/3943513085 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec039070 con 0x5629f2c63a00 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3943513085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 msgr2=0x5629f2c65e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c65e50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f98ec032780 tx=0x7f98ec045000 comp rx=0 tx=0).stop 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3943513085 shutdown_connections 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c65e50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c63430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3943513085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3943513085 >> 172.21.15.113:0/3943513085 conn(0x5629f2c54ba0 msgr2=0x5629f2c54f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3943513085 shutdown_connections 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3943513085 wait complete. 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 -- start start 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.366+0000 7f98fb5a9740 1 Processor -- start 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c63280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c60610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c60b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5629f2c66910 con 0x5629f2c63a00 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5629f2b78500 con 0x5629f2c5a4d0 2024-04-22T07:17:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5629f2c66a90 con 0x5629f2c5ae80 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 10 client.0 _ll_register_callbacks cb 0x5629f2b46b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c60b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c60b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43510/0 (socket says 172.21.15.113:43510) 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 -- 172.21.15.113:0/1847041910 learned_addr learned my addr 172.21.15.113:0/1847041910 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98ebfff640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c60610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f259c640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c63280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 -- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 msgr2=0x5629f2c60610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c60610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 -- 172.21.15.113:0/1847041910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 msgr2=0x5629f2c63280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c63280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 -- 172.21.15.113:0/1847041910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f28d13f0 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98f2d9d640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c60b50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f98ec004e30 tx=0x7f98ec003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5016df32db07bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98d9ffb640 1 -- 172.21.15.113:0/1847041910 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec059020 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98d9ffb640 1 -- 172.21.15.113:0/1847041910 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f98ec005030 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ec03a040 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98d9ffb640 1 -- 172.21.15.113:0/1847041910 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f98ec064050 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 10 client.16182 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.367+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5629f299b7d0 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98d9ffb640 1 -- 172.21.15.113:0/1847041910 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98ec03a040 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98d9ffb640 1 -- 172.21.15.113:0/1847041910 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f98ec005800 con 0x5629f2c63a00 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98fb5a9740 10 client.16182 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98fb5a9740 10 client.16182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T07:17:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98fb5a9740 20 client.16182 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2c739b0 0x5629f2c75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98fb5a9740 4 client.16182 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 --> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f2b5e930 con 0x5629f2c739b0 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98f259c640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2c739b0 0x5629f2c75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98d9ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.154:6834/217490889 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.368+0000 7f98f259c640 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2c739b0 0x5629f2c75da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36f4bc789f30e000 server_cookie=760e4429ed02f37b in_seq=0 out_seq=0 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98d9ffb640 1 -- 172.21.15.113:0/1847041910 <== mds.2 v2:172.21.15.154:6834/217490889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629f2b5e930 con 0x5629f2c739b0 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98d9ffb640 10 client.16182 handle_command_reply: tid=0 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 client.16182 shutdown 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 msgr2=0x5629f2c60b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c60b50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f98ec004e30 tx=0x7f98ec003f70 comp rx=0 tx=0).stop 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 shutdown_connections 2024-04-22T07:17:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2c739b0 0x5629f2c75da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c63a00 0x5629f2c60b50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5ae80 0x5629f2c60610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98d9ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.154:6834/217490889 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/1847041910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c5a4d0 0x5629f2c63280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 >> 172.21.15.113:0/1847041910 conn(0x5629f2c54ba0 msgr2=0x5629f2c568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 shutdown_connections 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.369+0000 7f98fb5a9740 1 -- 172.21.15.113:0/1847041910 wait complete. 2024-04-22T07:17:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.370+0000 7f98fb5a9740 20 client.16182 trim_cache size 0 max 16384 2024-04-22T07:17:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.382+0000 7f98fb5a9740 1 Processor -- start 2024-04-22T07:17:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.382+0000 7f98fb5a9740 1 -- start start 2024-04-22T07:17:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c53370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 0x5629f2c62d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41b2830 con 0x5629f2c5ae80 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41b0040 con 0x5629f2c63a00 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41ade90 con 0x5629f2c5a4d0 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c53370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c53370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43512/0 (socket says 172.21.15.113:43512) 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 -- 172.21.15.113:0/2055627919 learned_addr learned my addr 172.21.15.113:0/2055627919 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 -- 172.21.15.113:0/2055627919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 msgr2=0x5629f2c6f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 -- 172.21.15.113:0/2055627919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 msgr2=0x5629f2c62d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 0x5629f2c62d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 -- 172.21.15.113:0/2055627919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f2c54220 con 0x5629f2c5ae80 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98ebfff640 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c53370 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0cdfa0 tx=0x7f98dc00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554b86eb4de0e0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98dbfff640 1 -- 172.21.15.113:0/2055627919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0b5b50 con 0x5629f2c5ae80 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98dbfff640 1 -- 172.21.15.113:0/2055627919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98dc09c1f0 con 0x5629f2c5ae80 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.383+0000 7f98dbfff640 1 -- 172.21.15.113:0/2055627919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0bbb30 con 0x5629f2c5ae80 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- 172.21.15.113:0/2055627919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 msgr2=0x5629f2c53370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c53370 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0cdfa0 tx=0x7f98dc00d8d0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- 172.21.15.113:0/2055627919 shutdown_connections 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 0x5629f2c62d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c53370 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/2055627919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c6f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- 172.21.15.113:0/2055627919 >> 172.21.15.113:0/2055627919 conn(0x5629f2c54ba0 msgr2=0x5629f2c5f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- 172.21.15.113:0/2055627919 shutdown_connections 2024-04-22T07:17:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- 172.21.15.113:0/2055627919 wait complete. 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- start start 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 Processor -- start 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c5d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 0x5629f2c5dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41ae500 con 0x5629f2c5ae80 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41af970 con 0x5629f2c63a00 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f98f41aebd0 con 0x5629f2c5a4d0 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98fb5a9740 10 client.0 _ll_register_callbacks cb 0x7f98dc005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.384+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c5d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c5d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43520/0 (socket says 172.21.15.113:43520) 2024-04-22T07:17:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 -- 172.21.15.113:0/3302593681 learned_addr learned my addr 172.21.15.113:0/3302593681 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98f259c640 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 -- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 msgr2=0x5629f2c5d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 -- 172.21.15.113:0/3302593681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 msgr2=0x5629f2c5dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 0x5629f2c5dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 -- 172.21.15.113:0/3302593681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f2cd8db0 con 0x5629f2c5ae80 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98ebfff640 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c5d9a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f98dc098bc0 tx=0x7f98dc0a06b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa25bf5effd9aa2e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98d9ffb640 1 -- 172.21.15.113:0/3302593681 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc062020 con 0x5629f2c5ae80 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98d9ffb640 1 -- 172.21.15.113:0/3302593681 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f98dc0a0160 con 0x5629f2c5ae80 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc09c1f0 con 0x5629f2c5ae80 2024-04-22T07:17:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98d9ffb640 1 -- 172.21.15.113:0/3302593681 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f98dc0a0360 con 0x5629f2c5ae80 2024-04-22T07:17:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98fb5a9740 10 client.16188 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.385+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5629f2cda070 con 0x5629f2c5ae80 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98d9ffb640 1 -- 172.21.15.113:0/3302593681 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98dc09c1f0 con 0x5629f2c5ae80 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98d9ffb640 1 -- 172.21.15.113:0/3302593681 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f98dc097f40 con 0x5629f2c5ae80 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98fb5a9740 10 client.16188 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98fb5a9740 10 client.16188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98fb5a9740 20 client.16188 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2cda700 0x5629f2cdcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98fb5a9740 4 client.16188 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 --> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629f2c60ee0 con 0x5629f2cda700 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.388+0000 7f98f259c640 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2cda700 0x5629f2cdcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98f259c640 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2cda700 0x5629f2cdcb20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8764e6edc58437d server_cookie=3645d07569dbe27 in_seq=0 out_seq=0 2024-04-22T07:17:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98d9ffb640 10 client.16188 ms_handle_connect on v2:172.21.15.154:6834/217490889 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98d9ffb640 1 -- 172.21.15.113:0/3302593681 <== mds.2 v2:172.21.15.154:6834/217490889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629f2c60ee0 con 0x5629f2cda700 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98d9ffb640 10 client.16188 handle_command_reply: tid=0 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 client.16188 shutdown 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 msgr2=0x5629f2c5d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c5d9a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f98dc098bc0 tx=0x7f98dc0a06b0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 shutdown_connections 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:6834/217490889,v1:172.21.15.154:6835/217490889] conn(0x5629f2cda700 0x5629f2cdcb20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629f2c63a00 0x5629f2c5dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5629f2c5ae80 0x5629f2c5d9a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 --2- 172.21.15.113:0/3302593681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5629f2c5a4d0 0x5629f2c5d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98d9ffb640 0 client.16188 ms_handle_reset on v2:172.21.15.154:6834/217490889 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 >> 172.21.15.113:0/3302593681 conn(0x5629f2c54ba0 msgr2=0x5629f2c5b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 shutdown_connections 2024-04-22T07:17:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 1 -- 172.21.15.113:0/3302593681 wait complete. 2024-04-22T07:17:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.389+0000 7f98fb5a9740 20 client.16188 trim_cache size 0 max 16384 2024-04-22T07:17:04.393 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:17:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f98e00a3d20 msgr2=0x7f98e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f98e00a3d20 0x7f98e00a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036000 tx=0x7f98ec0324f0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 msgr2=0x7f98f41a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f41a0c00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00d950 tx=0x7f98dc00de10 comp rx=0 tx=0).stop 2024-04-22T07:17:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 shutdown_connections 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f98e00a3d20 0x7f98e00a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98f4105e20 0x7f98f419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98f4101f60 0x7f98f41a0c00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 --2- 172.21.15.113:0/990469184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f41015c0 0x7f98f41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 >> 172.21.15.113:0/990469184 conn(0x7f98f40773e0 msgr2=0x7f98f4104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 shutdown_connections 2024-04-22T07:17:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.390+0000 7f98f88d9640 1 -- 172.21.15.113:0/990469184 wait complete. 2024-04-22T07:17:04.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:17:04.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/584324906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc102b40 msgr2=0x7f9abc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/584324906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc102b40 0x7f9abc106b80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009900 tx=0x7f9aa402bc30 comp rx=0 tx=0).stop 2024-04-22T07:17:04.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/584324906 shutdown_connections 2024-04-22T07:17:04.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/584324906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9abc107260 0x7f9abc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/584324906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc102b40 0x7f9abc106b80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/584324906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9abc102190 0x7f9abc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/584324906 >> 172.21.15.113:0/584324906 conn(0x7f9abc0793a0 msgr2=0x7f9abc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/584324906 shutdown_connections 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/584324906 wait complete. 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 Processor -- start 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.501+0000 7f9ac0a9a640 1 -- start start 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9abc102190 0x7f9abc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9abc107260 0x7f9abc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 0x7f9abc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1122e0 con 0x7f9abc198680 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc112160 con 0x7f9abc102190 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc112460 con 0x7f9abc107260 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 0x7f9abc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 0x7f9abc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43542/0 (socket says 172.21.15.113:43542) 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 -- 172.21.15.113:0/2140106553 learned_addr learned my addr 172.21.15.113:0/2140106553 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9aba59c640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9abc102190 0x7f9abc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9abc107260 0x7f9abc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 -- 172.21.15.113:0/2140106553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9abc107260 msgr2=0x7f9abc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9abc107260 0x7f9abc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 -- 172.21.15.113:0/2140106553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9abc102190 msgr2=0x7f9abc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9abc102190 0x7f9abc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 -- 172.21.15.113:0/2140106553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc199280 con 0x7f9abc198680 2024-04-22T07:17:04.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9abad9d640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 0x7f9abc198b00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400eaa0 tx=0x7f9ab400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ab37fe640 1 -- 172.21.15.113:0/2140106553 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400cdb0 con 0x7f9abc198680 2024-04-22T07:17:04.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ab37fe640 1 -- 172.21.15.113:0/2140106553 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ab4005020 con 0x7f9abc198680 2024-04-22T07:17:04.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ab37fe640 1 -- 172.21.15.113:0/2140106553 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40116d0 con 0x7f9abc198680 2024-04-22T07:17:04.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc1a5360 con 0x7f9abc198680 2024-04-22T07:17:04.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.502+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc1a5710 con 0x7f9abc198680 2024-04-22T07:17:04.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.504+0000 7f9ab37fe640 1 -- 172.21.15.113:0/2140106553 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9ab40051d0 con 0x7f9abc198680 2024-04-22T07:17:04.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 Processor -- start 2024-04-22T07:17:04.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 -- start start 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 0x558b75a398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a42a00 0x558b75a44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558b7599e1f0 con 0x558b75a394d0 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558b7593b4b0 con 0x558b75a42a00 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ac376c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558b7599a1b0 con 0x558b75a39e80 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 0x558b75a398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ab37fe640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9aa80a3d20 0x7f9aa80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 0x558b75a398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43554/0 (socket says 172.21.15.113:43554) 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.505+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4083352707 learned_addr learned my addr 172.21.15.113:0/4083352707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9abad9d640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9aa80a3d20 0x7f9aa80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a42a00 0x558b75a44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9aba59c640 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4083352707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 msgr2=0x558b75a42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a42430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4083352707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a42a00 msgr2=0x558b75a44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a42a00 0x558b75a44e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4083352707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7592d1b0 con 0x558b75a394d0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab37fe640 1 -- 172.21.15.113:0/2140106553 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f9ab40d0ad0 con 0x7f9abc198680 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 0x558b75a398b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4005940 tx=0x7f9aa40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e532129ce2d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9abad9d640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9aa80a3d20 0x7f9aa80a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ab406b310 tx=0x7f9ab40c7aa0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab0ff9640 1 -- 172.21.15.113:0/4083352707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4047070 con 0x558b75a394d0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab0ff9640 1 -- 172.21.15.113:0/4083352707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9aa4005040 con 0x558b75a394d0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.506+0000 7f9ab0ff9640 1 -- 172.21.15.113:0/4083352707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004920 con 0x558b75a394d0 2024-04-22T07:17:04.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 -- 172.21.15.113:0/4083352707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 msgr2=0x558b75a398b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 0x558b75a398b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4005940 tx=0x7f9aa40058b0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 -- 172.21.15.113:0/4083352707 shutdown_connections 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a42a00 0x558b75a44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4083352707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a394d0 0x558b75a398b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 -- 172.21.15.113:0/4083352707 >> 172.21.15.113:0/4083352707 conn(0x558b75a33ba0 msgr2=0x558b75a33f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 -- 172.21.15.113:0/4083352707 shutdown_connections 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 -- 172.21.15.113:0/4083352707 wait complete. 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 -- start start 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.507+0000 7f9ac376c740 1 Processor -- start 2024-04-22T07:17:04.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a3f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75a3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558b75a45910 con 0x558b75a42a00 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558b75957500 con 0x558b75a394d0 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558b75a45a90 con 0x558b75a39e80 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 10 client.0 _ll_register_callbacks cb 0x558b75925b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75a3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75a3fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43562/0 (socket says 172.21.15.113:43562) 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4202247353 learned_addr learned my addr 172.21.15.113:0/4202247353 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9aba59c640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a3f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 msgr2=0x558b75a3f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a3f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4202247353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 msgr2=0x558b75a423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 -- 172.21.15.113:0/4202247353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b756b03f0 con 0x558b75a42a00 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9aba59c640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a3f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75a3fe70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9aa402ce40 tx=0x7f9aa402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b67fb6f01bad5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/4202247353 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4053060 con 0x558b75a42a00 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/4202247353 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f9aa4004980 con 0x558b75a42a00 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa4005040 con 0x558b75a42a00 2024-04-22T07:17:04.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/4202247353 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9aa4047040 con 0x558b75a42a00 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 10 client.16200 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.508+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b7577a7d0 con 0x558b75a42a00 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/4202247353 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9aa4005040 con 0x558b75a42a00 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/4202247353 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9aa404d040 con 0x558b75a42a00 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9ac376c740 10 client.16200 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9ac376c740 10 client.16200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9ac376c740 20 client.16200 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75a52ae0 0x558b75a54ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9ac376c740 4 client.16200 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 --> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b7593d930 con 0x558b75a52ae0 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9aba59c640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75a52ae0 0x558b75a54ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9aba59c640 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75a52ae0 0x558b75a54ed0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c28944de7a13445 server_cookie=3de7a6776f2607d7 in_seq=0 out_seq=0 2024-04-22T07:17:04.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.509+0000 7f9aa1ffb640 10 client.16200 ms_handle_connect on v2:172.21.15.154:6836/1615298521 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/4202247353 <== mds.1 v2:172.21.15.154:6836/1615298521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b7593d930 con 0x558b75a52ae0 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9aa1ffb640 10 client.16200 handle_command_reply: tid=0 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 client.16200 shutdown 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 msgr2=0x558b75a3fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75a3fe70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9aa402ce40 tx=0x7f9aa402bc30 comp rx=0 tx=0).stop 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 shutdown_connections 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75a52ae0 0x558b75a54ed0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75a3fe70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a39e80 0x558b75a3f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 --2- 172.21.15.113:0/4202247353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9aa1ffb640 0 client.16200 ms_handle_reset on v2:172.21.15.154:6836/1615298521 2024-04-22T07:17:04.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 >> 172.21.15.113:0/4202247353 conn(0x558b75a33ba0 msgr2=0x558b75a358f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 shutdown_connections 2024-04-22T07:17:04.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.510+0000 7f9ac376c740 1 -- 172.21.15.113:0/4202247353 wait complete. 2024-04-22T07:17:04.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.511+0000 7f9ac376c740 20 client.16200 trim_cache size 0 max 16384 2024-04-22T07:17:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.523+0000 7f9ac376c740 1 Processor -- start 2024-04-22T07:17:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 -- start start 2024-04-22T07:17:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 0x558b75a4e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a42a00 0x558b75a41e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b0670 con 0x558b75a39e80 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1ade80 con 0x558b75a394d0 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ac376c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1abcf0 con 0x558b75a42a00 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 0x558b75a4e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 0x558b75a4e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43568/0 (socket says 172.21.15.113:43568) 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 -- 172.21.15.113:0/3426481716 learned_addr learned my addr 172.21.15.113:0/3426481716 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a42a00 0x558b75a41e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 -- 172.21.15.113:0/3426481716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a42a00 msgr2=0x558b75a41e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a42a00 0x558b75a41e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 -- 172.21.15.113:0/3426481716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 msgr2=0x558b75a582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 -- 172.21.15.113:0/3426481716 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b75a336d0 con 0x558b75a39e80 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 0x558b75a4e420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x558b75a3a280 tx=0x7f9aac004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41b83e76477017a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aa3fff640 1 -- 172.21.15.113:0/3426481716 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac021030 con 0x558b75a39e80 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aa3fff640 1 -- 172.21.15.113:0/3426481716 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9aac01d8f0 con 0x558b75a39e80 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.524+0000 7f9aa3fff640 1 -- 172.21.15.113:0/3426481716 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0047a0 con 0x558b75a39e80 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 -- 172.21.15.113:0/3426481716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 msgr2=0x558b75a4e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 0x558b75a4e420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x558b75a3a280 tx=0x7f9aac004140 comp rx=0 tx=0).stop 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 -- 172.21.15.113:0/3426481716 shutdown_connections 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a42a00 0x558b75a41e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a39e80 0x558b75a4e420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3426481716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a394d0 0x558b75a582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 -- 172.21.15.113:0/3426481716 >> 172.21.15.113:0/3426481716 conn(0x558b75a33ba0 msgr2=0x558b75a4dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 -- 172.21.15.113:0/3426481716 shutdown_connections 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 -- 172.21.15.113:0/3426481716 wait complete. 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 -- start start 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.525+0000 7f9ac376c740 1 Processor -- start 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a394d0 0x558b75ab5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a39e80 0x558b75ab5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75aafd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1ac340 con 0x558b75a42a00 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1ad7b0 con 0x558b75a39e80 2024-04-22T07:17:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1aca10 con 0x558b75a394d0 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 10 client.0 _ll_register_callbacks cb 0x7f9aa40052c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75aafd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75aafd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43582/0 (socket says 172.21.15.113:43582) 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 -- 172.21.15.113:0/3702977111 learned_addr learned my addr 172.21.15.113:0/3702977111 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a394d0 0x558b75ab5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a39e80 0x558b75ab5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 -- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a394d0 msgr2=0x558b75ab5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a394d0 0x558b75ab5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 -- 172.21.15.113:0/3702977111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a39e80 msgr2=0x558b75ab5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a39e80 0x558b75ab5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 -- 172.21.15.113:0/3702977111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b75ab0470 con 0x558b75a42a00 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a39e80 0x558b75ab5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:04.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ab3fff640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75aafd80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009850 tx=0x7f9aa402dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba69eff818774d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/3702977111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4053040 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/3702977111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f9aa4004230 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aac01d8f0 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/3702977111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9aa4004430 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 10 client.16206 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.526+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b75abd170 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/3702977111 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9aac01d8f0 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/3702977111 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9aa4067070 con 0x558b75a42a00 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9ac376c740 10 client.16206 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9ac376c740 10 client.16206 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9ac376c740 20 client.16206 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75abd7b0 0x558b75abfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9ac376c740 4 client.16206 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 --> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b759abc10 con 0x558b75abd7b0 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75abd7b0 0x558b75abfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9aa1ffb640 10 client.16206 ms_handle_connect on v2:172.21.15.154:6836/1615298521 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.527+0000 7f9aba59c640 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75abd7b0 0x558b75abfbd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=664a3a2b0d3a8299 server_cookie=87ae488a67822e41 in_seq=0 out_seq=0 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9aa1ffb640 1 -- 172.21.15.113:0/3702977111 <== mds.1 v2:172.21.15.154:6836/1615298521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b759abc10 con 0x558b75abd7b0 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9aa1ffb640 10 client.16206 handle_command_reply: tid=0 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 client.16206 shutdown 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 msgr2=0x558b75aafd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75aafd80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009850 tx=0x7f9aa402dfb0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 shutdown_connections 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:6836/1615298521,v1:172.21.15.154:6837/1615298521] conn(0x558b75abd7b0 0x558b75abfbd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558b75a42a00 0x558b75aafd80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9aa1ffb640 0 client.16206 ms_handle_reset on v2:172.21.15.154:6836/1615298521 2024-04-22T07:17:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b75a39e80 0x558b75ab5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 --2- 172.21.15.113:0/3702977111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b75a394d0 0x558b75ab5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 >> 172.21.15.113:0/3702977111 conn(0x558b75a33ba0 msgr2=0x558b75a4e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 shutdown_connections 2024-04-22T07:17:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 1 -- 172.21.15.113:0/3702977111 wait complete. 2024-04-22T07:17:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac376c740 20 client.16206 trim_cache size 0 max 16384 2024-04-22T07:17:04.537 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:17:04.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9aa80a3d20 msgr2=0x7f9aa80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9aa80a3d20 0x7f9aa80a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ab406b310 tx=0x7f9ab40c7aa0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 msgr2=0x7f9abc198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 0x7f9abc198b00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400eaa0 tx=0x7f9ab400ef60 comp rx=0 tx=0).stop 2024-04-22T07:17:04.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 shutdown_connections 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9aa80a3d20 0x7f9aa80a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9abc198680 0x7f9abc198b00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9abc107260 0x7f9abc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 --2- 172.21.15.113:0/2140106553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9abc102190 0x7f9abc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 >> 172.21.15.113:0/2140106553 conn(0x7f9abc0793a0 msgr2=0x7f9abc0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 shutdown_connections 2024-04-22T07:17:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.528+0000 7f9ac0a9a640 1 -- 172.21.15.113:0/2140106553 wait complete. 2024-04-22T07:17:04.539 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:04 smithi113 ceph-mon[27933]: pgmap v72: 97 pgs: 97 active+clean; 157 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-04-22T07:17:04.539 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:04 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4145991751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:04.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:17:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 -- 172.21.15.113:0/2021486149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc107260 msgr2=0x7f59fc10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 --2- 172.21.15.113:0/2021486149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc107260 0x7f59fc10ee30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ca0 tx=0x7f59f402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 -- 172.21.15.113:0/2021486149 shutdown_connections 2024-04-22T07:17:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 --2- 172.21.15.113:0/2021486149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc107260 0x7f59fc10ee30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 --2- 172.21.15.113:0/2021486149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc102b40 0x7f59fc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 --2- 172.21.15.113:0/2021486149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59fc102190 0x7f59fc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 -- 172.21.15.113:0/2021486149 >> 172.21.15.113:0/2021486149 conn(0x7f59fc0793a0 msgr2=0x7f59fc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 -- 172.21.15.113:0/2021486149 shutdown_connections 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.652+0000 7f5a0305c640 1 -- 172.21.15.113:0/2021486149 wait complete. 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 Processor -- start 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 -- start start 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 0x7f59fc19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc107260 0x7f59fc19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59fc198600 0x7f59fc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112300 con 0x7f59fc102190 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112180 con 0x7f59fc198600 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a0305c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112480 con 0x7f59fc107260 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 0x7f59fc19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 0x7f59fc19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43608/0 (socket says 172.21.15.113:43608) 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 -- 172.21.15.113:0/846774304 learned_addr learned my addr 172.21.15.113:0/846774304 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f59fbfff640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc107260 0x7f59fc19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a015f9640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59fc198600 0x7f59fc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 -- 172.21.15.113:0/846774304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc107260 msgr2=0x7f59fc19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc107260 0x7f59fc19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 -- 172.21.15.113:0/846774304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59fc198600 msgr2=0x7f59fc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59fc198600 0x7f59fc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.653+0000 7f5a00df8640 1 -- 172.21.15.113:0/846774304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc199120 con 0x7f59fc102190 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.654+0000 7f5a00df8640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 0x7f59fc19e450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e9f0 tx=0x7f59f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.654+0000 7f59f9ffb640 1 -- 172.21.15.113:0/846774304 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f000cd80 con 0x7f59fc102190 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.654+0000 7f59f9ffb640 1 -- 172.21.15.113:0/846774304 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59f0010900 con 0x7f59fc102190 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.654+0000 7f59f9ffb640 1 -- 172.21.15.113:0/846774304 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0018920 con 0x7f59fc102190 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.654+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc199410 con 0x7f59fc102190 2024-04-22T07:17:04.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.654+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a56b0 con 0x7f59fc102190 2024-04-22T07:17:04.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f59f9ffb640 1 -- 172.21.15.113:0/846774304 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f59f0010ab0 con 0x7f59fc102190 2024-04-22T07:17:04.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 Processor -- start 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 -- start start 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e85790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e8e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea59dea1f0 con 0x55ea59e853b0 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ea59d874b0 con 0x55ea59e85d60 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a05c55740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ea59de61b0 con 0x55ea59e8ee60 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a015f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e85790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f59f9ffb640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f59ec0a3da0 0x7f59ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a015f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e85790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43614/0 (socket says 172.21.15.113:43614) 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f59fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a015f9640 1 -- 172.21.15.113:0/609251672 learned_addr learned my addr 172.21.15.113:0/609251672 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a00df8640 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e8e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.657+0000 7f5a015f9640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f59ec0a3da0 0x7f59ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59fbfff640 1 -- 172.21.15.113:0/609251672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 msgr2=0x55ea59e8e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59fbfff640 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e8e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59fbfff640 1 -- 172.21.15.113:0/609251672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 msgr2=0x55ea59e85790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59fbfff640 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e85790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59fbfff640 1 -- 172.21.15.113:0/609251672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea59d791b0 con 0x55ea59e8ee60 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59f9ffb640 1 -- 172.21.15.113:0/846774304 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f59f00d0b90 con 0x7f59fc102190 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f5a015f9640 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e85790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59fbfff640 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e8f2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59fc199740 tx=0x7f59e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea36507d7f0aa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f5a015f9640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f59ec0a3da0 0x7f59ec0a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009430 tx=0x7f59f402c0f0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59e6ffd640 1 -- 172.21.15.113:0/609251672 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8004050 con 0x55ea59e8ee60 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59e6ffd640 1 -- 172.21.15.113:0/609251672 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59e80041f0 con 0x55ea59e8ee60 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.658+0000 7f59e6ffd640 1 -- 172.21.15.113:0/609251672 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8019670 con 0x55ea59e8ee60 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 -- 172.21.15.113:0/609251672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 msgr2=0x55ea59e8f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e8f2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59fc199740 tx=0x7f59e8011810 comp rx=0 tx=0).stop 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 -- 172.21.15.113:0/609251672 shutdown_connections 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e8f2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e8e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 --2- 172.21.15.113:0/609251672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e85790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 -- 172.21.15.113:0/609251672 >> 172.21.15.113:0/609251672 conn(0x55ea59e7fba0 msgr2=0x55ea59e7ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 -- 172.21.15.113:0/609251672 shutdown_connections 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 -- 172.21.15.113:0/609251672 wait complete. 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 -- start start 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 Processor -- start 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e95280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.659+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e92610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e92b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ea59e95da0 con 0x55ea59e853b0 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ea59da3500 con 0x55ea59e85d60 2024-04-22T07:17:04.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ea59e95f20 con 0x55ea59e8ee60 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 10 client.0 _ll_register_callbacks cb 0x55ea59d71b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e95280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e95280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43630/0 (socket says 172.21.15.113:43630) 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 -- 172.21.15.113:0/2402980103 learned_addr learned my addr 172.21.15.113:0/2402980103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a015f9640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e92b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59fbfff640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e92610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 -- 172.21.15.113:0/2402980103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 msgr2=0x55ea59e92b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e92b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 -- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 msgr2=0x55ea59e92610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e92610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 -- 172.21.15.113:0/2402980103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea59afc3f0 con 0x55ea59e853b0 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59fbfff640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a00df8640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e95280 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e9c0 tx=0x7f59f0010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acb3e87ffc06aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59dbfff640 1 -- 172.21.15.113:0/2402980103 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a5720 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59dbfff640 1 -- 172.21.15.113:0/2402980103 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f59f0010430 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e80041f0 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59dbfff640 1 -- 172.21.15.113:0/2402980103 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f59f0010630 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 10 client.16218 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ea59bc67d0 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.660+0000 7f59dbfff640 1 -- 172.21.15.113:0/2402980103 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59e80041f0 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f59dbfff640 1 -- 172.21.15.113:0/2402980103 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f59f0098920 con 0x55ea59e853b0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f5a05c55740 10 client.16218 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f5a05c55740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f5a05c55740 20 client.16218 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f5a05c55740 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59ea30d0 0x55ea59ea54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f5a05c55740 4 client.16218 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea59d89930 con 0x55ea59ea30d0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f59fbfff640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59ea30d0 0x55ea59ea54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f59dbfff640 10 client.16218 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.661+0000 7f59fbfff640 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59ea30d0 0x55ea59ea54c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67641abd9ca06593 server_cookie=ad66082455abc71b in_seq=0 out_seq=0 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f59dbfff640 1 -- 172.21.15.113:0/2402980103 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea59d89930 con 0x55ea59ea30d0 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f59dbfff640 10 client.16218 handle_command_reply: tid=0 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 client.16218 shutdown 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 msgr2=0x55ea59e95280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e95280 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e9c0 tx=0x7f59f0010e20 comp rx=0 tx=0).stop 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 shutdown_connections 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e8ee60 0x55ea59e92b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e92610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59ea30d0 0x55ea59ea54c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 --2- 172.21.15.113:0/2402980103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e95280 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f59dbfff640 0 client.16218 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:04.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 >> 172.21.15.113:0/2402980103 conn(0x55ea59e7fba0 msgr2=0x55ea59e818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 shutdown_connections 2024-04-22T07:17:04.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 1 -- 172.21.15.113:0/2402980103 wait complete. 2024-04-22T07:17:04.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.662+0000 7f5a05c55740 20 client.16218 trim_cache size 0 max 16384 2024-04-22T07:17:04.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.675+0000 7f5a05c55740 1 Processor -- start 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 -- start start 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e90250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e85d60 0x55ea59e9eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e8ee60 0x55ea59e7ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1ae490 con 0x55ea59e853b0 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1abca0 con 0x55ea59e8ee60 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a05c55740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1a9b10 con 0x55ea59e85d60 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e90250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e90250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43642/0 (socket says 172.21.15.113:43642) 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 -- 172.21.15.113:0/3659414086 learned_addr learned my addr 172.21.15.113:0/3659414086 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f59fbfff640 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e85d60 0x55ea59e9eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a015f9640 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e8ee60 0x55ea59e7ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 -- 172.21.15.113:0/3659414086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e85d60 msgr2=0x55ea59e9eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e85d60 0x55ea59e9eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 -- 172.21.15.113:0/3659414086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e8ee60 msgr2=0x55ea59e7ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e8ee60 0x55ea59e7ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 -- 172.21.15.113:0/3659414086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea59e84fd0 con 0x55ea59e853b0 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f5a00df8640 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e90250 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f59f0098fc0 tx=0x7f59f009b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821f20b68abf2945 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f59e67fc640 1 -- 172.21.15.113:0/3659414086 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a5720 con 0x55ea59e853b0 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f59e67fc640 1 -- 172.21.15.113:0/3659414086 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59f00991d0 con 0x55ea59e853b0 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.676+0000 7f59e67fc640 1 -- 172.21.15.113:0/3659414086 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0018d40 con 0x55ea59e853b0 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 -- 172.21.15.113:0/3659414086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 msgr2=0x55ea59e90250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e90250 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f59f0098fc0 tx=0x7f59f009b070 comp rx=0 tx=0).stop 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 -- 172.21.15.113:0/3659414086 shutdown_connections 2024-04-22T07:17:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e8ee60 0x55ea59e7ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e85d60 0x55ea59e9eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3659414086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e853b0 0x55ea59e90250 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 -- 172.21.15.113:0/3659414086 >> 172.21.15.113:0/3659414086 conn(0x55ea59e7fba0 msgr2=0x55ea59e7e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 -- 172.21.15.113:0/3659414086 shutdown_connections 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 -- 172.21.15.113:0/3659414086 wait complete. 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 -- start start 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.677+0000 7f5a05c55740 1 Processor -- start 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e853b0 0x55ea59e94300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e94840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 0x55ea59e94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1aa160 con 0x55ea59e8ee60 2024-04-22T07:17:04.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1ab5d0 con 0x55ea59e85d60 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1aa830 con 0x55ea59e853b0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 10 client.0 _ll_register_callbacks cb 0x7f59f009c770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 0x55ea59e94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 0x55ea59e94d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43648/0 (socket says 172.21.15.113:43648) 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 -- 172.21.15.113:0/3166790509 learned_addr learned my addr 172.21.15.113:0/3166790509 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a00df8640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e853b0 0x55ea59e94300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f59fbfff640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e94840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 -- 172.21.15.113:0/3166790509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e853b0 msgr2=0x55ea59e94300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e853b0 0x55ea59e94300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 -- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 msgr2=0x55ea59e94840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e94840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 -- 172.21.15.113:0/3166790509 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea59f08630 con 0x55ea59e8ee60 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a015f9640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 0x55ea59e94d80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ca0 tx=0x7f59f40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402270e429bb9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f59dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f59dbfff640 1 -- 172.21.15.113:0/3166790509 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4039040 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f00991d0 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.678+0000 7f59dbfff640 1 -- 172.21.15.113:0/3166790509 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f59f4004510 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f59dbfff640 1 -- 172.21.15.113:0/3166790509 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f59f405e050 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 10 client.16224 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ea59f09910 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f59dbfff640 1 -- 172.21.15.113:0/3166790509 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f59f00991d0 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f59dbfff640 1 -- 172.21.15.113:0/3166790509 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f59f403f070 con 0x55ea59e8ee60 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 10 client.16224 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 20 client.16224 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59f0a1a0 0x55ea59f0c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 4 client.16224 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea59e92ee0 con 0x55ea59f0a1a0 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.679+0000 7f5a00df8640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59f0a1a0 0x55ea59f0c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f59dbfff640 10 client.16224 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a00df8640 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59f0a1a0 0x55ea59f0c5c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2309bc559c30d874 server_cookie=10f0f87a34ce944b in_seq=0 out_seq=0 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f59dbfff640 1 -- 172.21.15.113:0/3166790509 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea59e92ee0 con 0x55ea59f0a1a0 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f59dbfff640 10 client.16224 handle_command_reply: tid=0 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 client.16224 shutdown 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 msgr2=0x55ea59e94d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 0x55ea59e94d80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ca0 tx=0x7f59f40095c0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 shutdown_connections 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ea59f0a1a0 0x55ea59f0c5c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ea59e8ee60 0x55ea59e94d80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f59dbfff640 0 client.16224 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea59e85d60 0x55ea59e94840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 --2- 172.21.15.113:0/3166790509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ea59e853b0 0x55ea59e94300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 >> 172.21.15.113:0/3166790509 conn(0x55ea59e7fba0 msgr2=0x55ea59e7ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 shutdown_connections 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 1 -- 172.21.15.113:0/3166790509 wait complete. 2024-04-22T07:17:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.680+0000 7f5a05c55740 20 client.16224 trim_cache size 0 max 16384 2024-04-22T07:17:04.686 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f59ec0a3da0 msgr2=0x7f59ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f59ec0a3da0 0x7f59ec0a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009430 tx=0x7f59f402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 msgr2=0x7f59fc19e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 0x7f59fc19e450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e9f0 tx=0x7f59f000eeb0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 shutdown_connections 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f59ec0a3da0 0x7f59ec0a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59fc198600 0x7f59fc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc107260 0x7f59fc19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 --2- 172.21.15.113:0/846774304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59fc102190 0x7f59fc19e450 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 >> 172.21.15.113:0/846774304 conn(0x7f59fc0793a0 msgr2=0x7f59fc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 shutdown_connections 2024-04-22T07:17:04.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:04.681+0000 7f5a0305c640 1 -- 172.21.15.113:0/846774304 wait complete. 2024-04-22T07:17:04.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:04 smithi133 ceph-mon[31198]: pgmap v72: 97 pgs: 97 active+clean; 157 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-04-22T07:17:04.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:04 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4145991751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:04.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:04 smithi154 ceph-mon[31862]: pgmap v72: 97 pgs: 97 active+clean; 157 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-04-22T07:17:04.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:04 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4145991751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:05.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:17:05.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:17:05.690 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:17:05.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 -- 172.21.15.113:0/2722948056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a24107260 msgr2=0x7f6a2410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 --2- 172.21.15.113:0/2722948056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a24107260 0x7f6a2410ee30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009ca0 tx=0x7f6a1c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 -- 172.21.15.113:0/2722948056 shutdown_connections 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 --2- 172.21.15.113:0/2722948056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a24107260 0x7f6a2410ee30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 --2- 172.21.15.113:0/2722948056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102b40 0x7f6a24106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 --2- 172.21.15.113:0/2722948056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24102190 0x7f6a24102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 -- 172.21.15.113:0/2722948056 >> 172.21.15.113:0/2722948056 conn(0x7f6a240793a0 msgr2=0x7f6a240797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.798+0000 7f6a2c455640 1 -- 172.21.15.113:0/2722948056 shutdown_connections 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 -- 172.21.15.113:0/2722948056 wait complete. 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 Processor -- start 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 -- start start 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 0x7f6a2419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 0x7f6a2419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 0x7f6a24198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241122e0 con 0x7f6a241985e0 2024-04-22T07:17:05.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24112160 con 0x7f6a24102190 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2c455640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24112460 con 0x7f6a24107260 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2a9f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 0x7f6a24198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 0x7f6a2419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a299f0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 0x7f6a2419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a299f0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 0x7f6a2419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:60226/0 (socket says 172.21.15.113:60226) 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2a9f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 0x7f6a24198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56046/0 (socket says 172.21.15.113:56046) 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a299f0640 1 -- 172.21.15.113:0/3426836013 learned_addr learned my addr 172.21.15.113:0/3426836013 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.799+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 0x7f6a2419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:36838/0 (socket says 172.21.15.113:36838) 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a299f0640 1 -- 172.21.15.113:0/3426836013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 msgr2=0x7f6a2419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a299f0640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 0x7f6a2419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a299f0640 1 -- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 msgr2=0x7f6a24198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a299f0640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 0x7f6a24198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a299f0640 1 -- 172.21.15.113:0/3426836013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a24199100 con 0x7f6a24107260 2024-04-22T07:17:05.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 0x7f6a2419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 0x7f6a24198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a299f0640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 0x7f6a2419e970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400e9c0 tx=0x7f6a1400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a1b7fe640 1 -- 172.21.15.113:0/3426836013 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400cd80 con 0x7f6a24107260 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241993f0 con 0x7f6a24107260 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a1b7fe640 1 -- 172.21.15.113:0/3426836013 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6a140054f0 con 0x7f6a24107260 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a1b7fe640 1 -- 172.21.15.113:0/3426836013 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14010690 con 0x7f6a24107260 2024-04-22T07:17:05.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.800+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a5640 con 0x7f6a24107260 2024-04-22T07:17:05.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a2f049740 1 Processor -- start 2024-04-22T07:17:05.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a2f049740 1 -- start start 2024-04-22T07:17:05.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a1b7fe640 1 -- 172.21.15.113:0/3426836013 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6a140046c0 con 0x7f6a24107260 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b10c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b10ce10 0x55dc1b10f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.803+0000 7f6a2f049740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc1b067f00 con 0x55dc1b10ce10 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2f049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55dc1b005590 con 0x55dc1b1038e0 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2f049740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc1b063ec0 con 0x55dc1b104290 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a299f0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b10ce10 0x55dc1b10f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b10c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a9f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b10c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:60230/0 (socket says 172.21.15.113:60230) 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a9f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:36848/0 (socket says 172.21.15.113:36848) 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/957754204 learned_addr learned my addr 172.21.15.113:0/957754204 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/957754204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 msgr2=0x55dc1b103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/957754204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b10ce10 msgr2=0x55dc1b10f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b10ce10 0x55dc1b10f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/957754204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1ae427e0 con 0x55dc1b104290 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a299f0640 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b10ce10 0x55dc1b10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b10c840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a200115f0 tx=0x7f6a20011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7755f0f4f926c879 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a1b7fe640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6a100a3d20 0x7f6a100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a18bfe640 1 -- 172.21.15.113:0/957754204 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2001e070 con 0x55dc1b104290 2024-04-22T07:17:05.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6a100a3d20 0x7f6a100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a18bfe640 1 -- 172.21.15.113:0/957754204 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6a2000fb10 con 0x55dc1b104290 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a18bfe640 1 -- 172.21.15.113:0/957754204 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20019bd0 con 0x55dc1b104290 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.804+0000 7f6a1b7fe640 1 -- 172.21.15.113:0/3426836013 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f6a140d0d10 con 0x7f6a24107260 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 -- 172.21.15.113:0/957754204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 msgr2=0x55dc1b10c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6a100a3d20 0x7f6a100a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004af0 tx=0x7f6a1c005c50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b10c840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a200115f0 tx=0x7f6a20011ab0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 -- 172.21.15.113:0/957754204 shutdown_connections 2024-04-22T07:17:05.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b10ce10 0x55dc1b10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b10c840 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 --2- 172.21.15.113:0/957754204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 -- 172.21.15.113:0/957754204 >> 172.21.15.113:0/957754204 conn(0x55dc1b0fdfb0 msgr2=0x55dc1b0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 -- 172.21.15.113:0/957754204 shutdown_connections 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.805+0000 7f6a2f049740 1 -- 172.21.15.113:0/957754204 wait complete. 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 -- start start 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 Processor -- start 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b109e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b104290 0x55dc1b10a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b10a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55dc1b10fd80 con 0x55dc1b1038e0 2024-04-22T07:17:05.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55dc1b021580 con 0x55dc1b104290 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55dc1b10ff00 con 0x55dc1b10ce10 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2f049740 10 client.0 _ll_register_callbacks cb 0x55dc1b027650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56058/0 (socket says 172.21.15.113:56058) 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/2354972836 learned_addr learned my addr 172.21.15.113:0/2354972836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a299f0640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b104290 0x55dc1b10a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b10a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/2354972836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 msgr2=0x55dc1b10a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b10a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b104290 msgr2=0x55dc1b10a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b104290 0x55dc1b10a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 -- 172.21.15.113:0/2354972836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1aea2f70 con 0x55dc1b1038e0 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a299f0640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b104290 0x55dc1b10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:05.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b109e70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6a2001c040 tx=0x7f6a2000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc45350cd629cb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.806+0000 7f6a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2354972836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2001e040 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2354972836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f6a2000f700 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2000fb10 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2354972836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6a20030050 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 10 client.16236 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dc1ad3a170 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2354972836 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6a2000fb10 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2354972836 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6a20021070 con 0x55dc1b1038e0 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 10 client.16236 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 20 client.16236 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:05.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b11d2b0 0x55dc1b11f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 4 client.16236 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc1b075920 con 0x55dc1b11d2b0 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.807+0000 7f6a299f0640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b11d2b0 0x55dc1b11f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.808+0000 7f6a0dffb640 10 client.16236 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.808+0000 7f6a299f0640 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b11d2b0 0x55dc1b11f6a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8a8f643b6e4188 server_cookie=df93778f16d273ce in_seq=0 out_seq=0 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.808+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2354972836 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc1b075920 con 0x55dc1b11d2b0 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.808+0000 7f6a0dffb640 10 client.16236 handle_command_reply: tid=0 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 client.16236 shutdown 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 msgr2=0x55dc1b109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b109e70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6a2001c040 tx=0x7f6a2000ff70 comp rx=0 tx=0).stop 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 shutdown_connections 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b104290 0x55dc1b10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b11d2b0 0x55dc1b11f6a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2354972836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b109e70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 >> 172.21.15.113:0/2354972836 conn(0x55dc1b0fdfb0 msgr2=0x55dc1b0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:05.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a0dffb640 0 client.16236 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:05.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 shutdown_connections 2024-04-22T07:17:05.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 1 -- 172.21.15.113:0/2354972836 wait complete. 2024-04-22T07:17:05.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.809+0000 7f6a2f049740 20 client.16236 trim_cache size 0 max 16384 2024-04-22T07:17:05.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 Processor -- start 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 -- start start 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b10c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 0x55dc1b118c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b124eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241ae570 con 0x55dc1b104290 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241abd80 con 0x55dc1b1038e0 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a2f049740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241a9bd0 con 0x55dc1b10ce10 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.833+0000 7f6a299f0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 0x55dc1b118c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 0x55dc1b118c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56062/0 (socket says 172.21.15.113:56062) 2024-04-22T07:17:05.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 -- 172.21.15.113:0/514354929 learned_addr learned my addr 172.21.15.113:0/514354929 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b124eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b10c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 -- 172.21.15.113:0/514354929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 msgr2=0x55dc1b124eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b124eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 -- 172.21.15.113:0/514354929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 msgr2=0x55dc1b10c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b10c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 -- 172.21.15.113:0/514354929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1b1083c0 con 0x55dc1b104290 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b124eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a299f0640 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 0x55dc1b118c40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6a14099090 tx=0x7f6a140dbb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8969faf3f40fbf07 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a0ffff640 1 -- 172.21.15.113:0/514354929 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14087040 con 0x55dc1b104290 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a0ffff640 1 -- 172.21.15.113:0/514354929 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6a1409d330 con 0x55dc1b104290 2024-04-22T07:17:05.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.834+0000 7f6a0ffff640 1 -- 172.21.15.113:0/514354929 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a140a58a0 con 0x55dc1b104290 2024-04-22T07:17:05.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 -- 172.21.15.113:0/514354929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 msgr2=0x55dc1b118c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 0x55dc1b118c40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6a14099090 tx=0x7f6a140dbb20 comp rx=0 tx=0).stop 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 -- 172.21.15.113:0/514354929 shutdown_connections 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b10ce10 0x55dc1b124eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b104290 0x55dc1b118c40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 --2- 172.21.15.113:0/514354929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b1038e0 0x55dc1b10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 -- 172.21.15.113:0/514354929 >> 172.21.15.113:0/514354929 conn(0x55dc1b0fdfb0 msgr2=0x55dc1b108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 -- 172.21.15.113:0/514354929 shutdown_connections 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.835+0000 7f6a2f049740 1 -- 172.21.15.113:0/514354929 wait complete. 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 -- start start 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 Processor -- start 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b115150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b115690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 0x55dc1b17eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241aa240 con 0x55dc1b1038e0 2024-04-22T07:17:05.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241ab6b0 con 0x55dc1b10ce10 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241aa910 con 0x55dc1b104290 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2f049740 10 client.0 _ll_register_callbacks cb 0x7f6a2001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2a9f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 0x55dc1b17eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2a1f1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2a9f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 0x55dc1b17eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:36876/0 (socket says 172.21.15.113:36876) 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a2a9f2640 1 -- 172.21.15.113:0/2874273293 learned_addr learned my addr 172.21.15.113:0/2874273293 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.836+0000 7f6a299f0640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b115690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a9f2640 1 -- 172.21.15.113:0/2874273293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 msgr2=0x55dc1b115690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b115690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a9f2640 1 -- 172.21.15.113:0/2874273293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 msgr2=0x55dc1b115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b115150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a9f2640 1 -- 172.21.15.113:0/2874273293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1b17f3b0 con 0x55dc1b10ce10 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2a9f2640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 0x55dc1b17eb80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c037040 tx=0x7f6a1c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b3c8a35da66d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2874273293 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c039040 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2874273293 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f6a1c04f440 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a1409d330 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2874273293 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c04f640 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2f049740 10 client.25664 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.837+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dc1b194730 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2874273293 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6a1409d330 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2874273293 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f6a1c05d070 con 0x55dc1b10ce10 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2f049740 10 client.25664 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2f049740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2f049740 20 client.25664 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b194e00 0x55dc1b197220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2f049740 4 client.25664 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dc1b06f790 con 0x55dc1b194e00 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.838+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b194e00 0x55dc1b197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.839+0000 7f6a2a1f1640 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b194e00 0x55dc1b197220 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e127ea48849f96 server_cookie=ffe78f64f753fad3 in_seq=0 out_seq=0 2024-04-22T07:17:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.839+0000 7f6a0dffb640 10 client.25664 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.840+0000 7f6a0dffb640 1 -- 172.21.15.113:0/2874273293 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dc1b06f790 con 0x55dc1b194e00 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.840+0000 7f6a0dffb640 10 client.25664 handle_command_reply: tid=0 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 client.25664 shutdown 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 msgr2=0x55dc1b17eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 0x55dc1b17eb80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c037040 tx=0x7f6a1c0095c0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 shutdown_connections 2024-04-22T07:17:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc1b10ce10 0x55dc1b17eb80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55dc1b104290 0x55dc1b115690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55dc1b194e00 0x55dc1b197220 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 --2- 172.21.15.113:0/2874273293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55dc1b1038e0 0x55dc1b115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a0dffb640 0 client.25664 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 >> 172.21.15.113:0/2874273293 conn(0x55dc1b0fdfb0 msgr2=0x55dc1b109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 shutdown_connections 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 1 -- 172.21.15.113:0/2874273293 wait complete. 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.841+0000 7f6a2f049740 20 client.25664 trim_cache size 0 max 16384 2024-04-22T07:17:05.842 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8", 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6a100a3d20 msgr2=0x7f6a100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6a100a3d20 0x7f6a100a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004af0 tx=0x7f6a1c005c50 comp rx=0 tx=0).stop 2024-04-22T07:17:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 msgr2=0x7f6a2419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 0x7f6a2419e970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400e9c0 tx=0x7f6a1400ee80 comp rx=0 tx=0).stop 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 shutdown_connections 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6a100a3d20 0x7f6a100a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a241985e0 0x7f6a24198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a24107260 0x7f6a2419e970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 --2- 172.21.15.113:0/3426836013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24102190 0x7f6a2419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 >> 172.21.15.113:0/3426836013 conn(0x7f6a240793a0 msgr2=0x7f6a240ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 shutdown_connections 2024-04-22T07:17:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.842+0000 7f6a2c455640 1 -- 172.21.15.113:0/3426836013 wait complete. 2024-04-22T07:17:05.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:17:05.856 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:17:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 -- 172.21.15.113:0/1513258512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 msgr2=0x7fc24410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 --2- 172.21.15.113:0/1513258512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc24410dd20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009900 tx=0x7fc23c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 -- 172.21.15.113:0/1513258512 shutdown_connections 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 --2- 172.21.15.113:0/1513258512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc24410dd20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 --2- 172.21.15.113:0/1513258512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101a30 0x7fc244105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 --2- 172.21.15.113:0/1513258512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc244101080 0x7fc244101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 -- 172.21.15.113:0/1513258512 >> 172.21.15.113:0/1513258512 conn(0x7fc244078000 msgr2=0x7fc244078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 -- 172.21.15.113:0/1513258512 shutdown_connections 2024-04-22T07:17:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 -- 172.21.15.113:0/1513258512 wait complete. 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.975+0000 7fc24baba640 1 Processor -- start 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 -- start start 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101080 0x7fc2441995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc244199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24419a0b0 0x7fc2441a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc244111240 con 0x7fc244106150 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441110c0 con 0x7fc24419a0b0 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24baba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441113c0 con 0x7fc244101080 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc244199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249856640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101080 0x7fc2441995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc244199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56088/0 (socket says 172.21.15.113:56088) 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 -- 172.21.15.113:0/2078943142 learned_addr learned my addr 172.21.15.113:0/2078943142 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc24a057640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24419a0b0 0x7fc2441a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 -- 172.21.15.113:0/2078943142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101080 msgr2=0x7fc2441995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101080 0x7fc2441995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 -- 172.21.15.113:0/2078943142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24419a0b0 msgr2=0x7fc2441a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24419a0b0 0x7fc2441a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 -- 172.21.15.113:0/2078943142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2441a4e60 con 0x7fc244106150 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249856640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101080 0x7fc2441995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc249055640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc244199b20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc23400e9f0 tx=0x7fc23400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.976+0000 7fc23affd640 1 -- 172.21.15.113:0/2078943142 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23400ccc0 con 0x7fc244106150 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.977+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2441a5150 con 0x7fc244106150 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.977+0000 7fc23affd640 1 -- 172.21.15.113:0/2078943142 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc2340054f0 con 0x7fc244106150 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.977+0000 7fc23affd640 1 -- 172.21.15.113:0/2078943142 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234010690 con 0x7fc244106150 2024-04-22T07:17:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.977+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441a5650 con 0x7fc244106150 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.979+0000 7fc23affd640 1 -- 172.21.15.113:0/2078943142 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fc2340046c0 con 0x7fc244106150 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.979+0000 7fc24e789740 1 Processor -- start 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.979+0000 7fc23affd640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2300a3d20 0x7fc2300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24a057640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2300a3d20 0x7fc2300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc23affd640 1 -- 172.21.15.113:0/2078943142 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fc2340d0c10 con 0x7fc244106150 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 -- start start 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51d580 0x55ad9f51d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f526540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f528e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ad9f4821e0 con 0x55ad9f526a80 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad9f41f390 con 0x55ad9f51d580 2024-04-22T07:17:06.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24e789740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ad9f47e1a0 con 0x55ad9f51dea0 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f526540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249856640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51d580 0x55ad9f51d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249856640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51d580 0x55ad9f51d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:36902/0 (socket says 172.21.15.113:36902) 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f526540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:60260/0 (socket says 172.21.15.113:60260) 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249856640 1 -- 172.21.15.113:0/127163355 learned_addr learned my addr 172.21.15.113:0/127163355 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24a057640 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f528e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24a057640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2300a3d20 0x7fc2300a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005ab0 tx=0x7fc23c005a40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 -- 172.21.15.113:0/127163355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51d580 msgr2=0x55ad9f51d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51d580 0x55ad9f51d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 -- 172.21.15.113:0/127163355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 msgr2=0x55ad9f528e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f528e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc249055640 1 -- 172.21.15.113:0/127163355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9f25e7d0 con 0x55ad9f51dea0 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.980+0000 7fc24a057640 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f528e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:17:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.981+0000 7fc249055640 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f526540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2340ce2e0 tx=0x7fc23409c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd662bf7280b7d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.981+0000 7fc22ffff640 1 -- 172.21.15.113:0/127163355 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23409de60 con 0x55ad9f51dea0 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.981+0000 7fc22ffff640 1 -- 172.21.15.113:0/127163355 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc2340a0ee0 con 0x55ad9f51dea0 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.981+0000 7fc22ffff640 1 -- 172.21.15.113:0/127163355 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340a5310 con 0x55ad9f51dea0 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 -- 172.21.15.113:0/127163355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 msgr2=0x55ad9f526540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f526540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2340ce2e0 tx=0x7fc23409c730 comp rx=0 tx=0).stop 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 -- 172.21.15.113:0/127163355 shutdown_connections 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f528e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f526540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 --2- 172.21.15.113:0/127163355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51d580 0x55ad9f51d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 -- 172.21.15.113:0/127163355 >> 172.21.15.113:0/127163355 conn(0x55ad9f517cb0 msgr2=0x55ad9f518090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 -- 172.21.15.113:0/127163355 shutdown_connections 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 -- 172.21.15.113:0/127163355 wait complete. 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 -- start start 2024-04-22T07:17:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 Processor -- start 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51d580 0x55ad9f523b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.982+0000 7fc24e789740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f5240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f5245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ad9f529870 con 0x55ad9f526a80 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad9f43b2f0 con 0x55ad9f51dea0 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ad9f5299f0 con 0x55ad9f51d580 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 10 client.0 _ll_register_callbacks cb 0x55ad9f448580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f5245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f5245f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56108/0 (socket says 172.21.15.113:56108) 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 -- 172.21.15.113:0/2727568740 learned_addr learned my addr 172.21.15.113:0/2727568740 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc249856640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51d580 0x55ad9f523b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc249055640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f5240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 -- 172.21.15.113:0/2727568740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51d580 msgr2=0x55ad9f523b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51d580 0x55ad9f523b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 -- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 msgr2=0x55ad9f5240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f5240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 -- 172.21.15.113:0/2727568740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9f1943f0 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24a057640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f5245f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc23c02c240 tx=0x7fc23c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657cb9efa934fe13 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc22d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2727568740 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c04f030 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2340a0ee0 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2727568740 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fc23c03a040 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2727568740 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc23c02cc10 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 10 client.16254 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.983+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad9f1540f0 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2727568740 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc2340a0ee0 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2727568740 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fc23c05a070 con 0x55ad9f526a80 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc24e789740 10 client.16254 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:06.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc24e789740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc24e789740 20 client.16254 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc24e789740 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f536b20 0x55ad9f538f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc24e789740 4 client.16254 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad9f48fc00 con 0x55ad9f536b20 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.984+0000 7fc249856640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f536b20 0x55ad9f538f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.985+0000 7fc249856640 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f536b20 0x55ad9f538f40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678e403a39fdf13b server_cookie=362281764e39ca95 in_seq=0 out_seq=0 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.985+0000 7fc22d7fa640 10 client.16254 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2727568740 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad9f48fc00 con 0x55ad9f536b20 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc22d7fa640 10 client.16254 handle_command_reply: tid=0 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 client.16254 shutdown 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 msgr2=0x55ad9f5245f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f5245f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc23c02c240 tx=0x7fc23c009260 comp rx=0 tx=0).stop 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 shutdown_connections 2024-04-22T07:17:06.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f536b20 0x55ad9f538f40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f526a80 0x55ad9f5245f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f5240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc22d7fa640 0 client.16254 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 --2- 172.21.15.113:0/2727568740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51d580 0x55ad9f523b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 >> 172.21.15.113:0/2727568740 conn(0x55ad9f517cb0 msgr2=0x55ad9f519a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 shutdown_connections 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 1 -- 172.21.15.113:0/2727568740 wait complete. 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:05.986+0000 7fc24e789740 20 client.16254 trim_cache size 0 max 16384 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 Processor -- start 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 -- start start 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f5262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f522ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f526a80 0x55ad9f532730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441b0700 con 0x55ad9f51d580 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441adf10 con 0x55ad9f526a80 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24e789740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441abd60 con 0x55ad9f51dea0 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f5262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f5262a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56118/0 (socket says 172.21.15.113:56118) 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 -- 172.21.15.113:0/1185321292 learned_addr learned my addr 172.21.15.113:0/1185321292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249055640 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f522ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc24a057640 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f526a80 0x55ad9f532730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 -- 172.21.15.113:0/1185321292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 msgr2=0x55ad9f522ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f522ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 -- 172.21.15.113:0/1185321292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f526a80 msgr2=0x55ad9f532730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f526a80 0x55ad9f532730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 -- 172.21.15.113:0/1185321292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9f51dcd0 con 0x55ad9f51d580 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.004+0000 7fc249856640 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f5262a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc2400129b0 tx=0x7fc240009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba2fad00377ad5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc22f7fe640 1 -- 172.21.15.113:0/1185321292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240012110 con 0x55ad9f51d580 2024-04-22T07:17:06.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc22f7fe640 1 -- 172.21.15.113:0/1185321292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc240003970 con 0x55ad9f51d580 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc22f7fe640 1 -- 172.21.15.113:0/1185321292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400225d0 con 0x55ad9f51d580 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 -- 172.21.15.113:0/1185321292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 msgr2=0x55ad9f5262a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f5262a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc2400129b0 tx=0x7fc240009500 comp rx=0 tx=0).stop 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 -- 172.21.15.113:0/1185321292 shutdown_connections 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f526a80 0x55ad9f532730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f51dea0 0x55ad9f522ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 --2- 172.21.15.113:0/1185321292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f5262a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 -- 172.21.15.113:0/1185321292 >> 172.21.15.113:0/1185321292 conn(0x55ad9f517cb0 msgr2=0x55ad9f519970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 -- 172.21.15.113:0/1185321292 shutdown_connections 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.005+0000 7fc24e789740 1 -- 172.21.15.113:0/1185321292 wait complete. 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 -- start start 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 Processor -- start 2024-04-22T07:17:06.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f52ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f52f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f526a80 0x55ad9f52bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441ac3d0 con 0x55ad9f51d580 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441ad840 con 0x55ad9f51dea0 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441acaa0 con 0x55ad9f526a80 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24e789740 10 client.0 _ll_register_callbacks cb 0x7fc234005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc249856640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f52ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc249856640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f52ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56132/0 (socket says 172.21.15.113:56132) 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc249856640 1 -- 172.21.15.113:0/2604169956 learned_addr learned my addr 172.21.15.113:0/2604169956 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc24a057640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f526a80 0x55ad9f52bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.006+0000 7fc249055640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f52f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc249856640 1 -- 172.21.15.113:0/2604169956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f526a80 msgr2=0x55ad9f52bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc249856640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f526a80 0x55ad9f52bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc249856640 1 -- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 msgr2=0x55ad9f52f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc249856640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f52f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc249856640 1 -- 172.21.15.113:0/2604169956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9f52c3c0 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc249856640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f52ec40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc24000a010 tx=0x7fc240005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5779557d5fd34f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc22d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2604169956 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24002e020 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2604169956 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fc240004a40 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc240003970 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2604169956 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc240039050 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 10 client.16260 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad9f59dac0 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2604169956 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc240003970 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2604169956 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fc240029040 con 0x55ad9f51d580 2024-04-22T07:17:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 10 client.16260 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 20 client.16260 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f59e200 0x55ad9f5a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 4 client.16260 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.007+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad9f489a70 con 0x55ad9f59e200 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc249055640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f59e200 0x55ad9f5a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc249055640 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f59e200 0x55ad9f5a0620 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78ba849858011cd server_cookie=179017d40bd8b91a in_seq=0 out_seq=0 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc22d7fa640 10 client.16260 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc22d7fa640 1 -- 172.21.15.113:0/2604169956 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ad9f489a70 con 0x55ad9f59e200 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc22d7fa640 10 client.16260 handle_command_reply: tid=0 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc24e789740 1 client.16260 shutdown 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 msgr2=0x55ad9f52ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.008+0000 7fc24e789740 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f52ec40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc24000a010 tx=0x7fc240005af0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 shutdown_connections 2024-04-22T07:17:06.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ad9f526a80 0x55ad9f52bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad9f51dea0 0x55ad9f52f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55ad9f59e200 0x55ad9f5a0620 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 --2- 172.21.15.113:0/2604169956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ad9f51d580 0x55ad9f52ec40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc22d7fa640 0 client.16260 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 >> 172.21.15.113:0/2604169956 conn(0x55ad9f517cb0 msgr2=0x55ad9f523b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 shutdown_connections 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 1 -- 172.21.15.113:0/2604169956 wait complete. 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24e789740 20 client.16260 trim_cache size 0 max 16384 2024-04-22T07:17:06.030 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: "a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8": { 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8", 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:06.031 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:17:06.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2300a3d20 msgr2=0x7fc2300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2300a3d20 0x7fc2300a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005ab0 tx=0x7fc23c005a40 comp rx=0 tx=0).stop 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 msgr2=0x7fc244199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc244199b20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc23400e9f0 tx=0x7fc23400eeb0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 shutdown_connections 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc2300a3d20 0x7fc2300a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24419a0b0 0x7fc2441a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc244106150 0x7fc244199b20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 --2- 172.21.15.113:0/2078943142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc244101080 0x7fc2441995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.009+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 >> 172.21.15.113:0/2078943142 conn(0x7fc244078000 msgr2=0x7fc2440fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.010+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 shutdown_connections 2024-04-22T07:17:06.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:06.010+0000 7fc24baba640 1 -- 172.21.15.113:0/2078943142 wait complete. 2024-04-22T07:17:06.034 INFO:tasks.cephfs.filesystem:scrub status for tag:a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8 - {'path': '/', 'tag': 'a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8', 'options': 'recursive,force'} 2024-04-22T07:17:06.813 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:06 smithi133 ceph-mon[31198]: pgmap v73: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-04-22T07:17:06.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:06 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:06.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:06 smithi154 ceph-mon[31862]: pgmap v73: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-04-22T07:17:06.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:06 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:06.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:06 smithi113 ceph-mon[27933]: pgmap v73: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-04-22T07:17:06.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:06 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:07.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:07 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:17:07.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:07.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:07.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:07.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:07.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:07 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:17:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:07.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:07.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:07.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:07.836 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:07.836 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:07 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:08.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:08 smithi133 ceph-mon[31198]: pgmap v74: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-22T07:17:08.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:08 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:08.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:08 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:08.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:08 smithi154 ceph-mon[31862]: pgmap v74: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-22T07:17:08.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:08 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:08.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:08 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:08 smithi113 ceph-mon[27933]: pgmap v74: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-22T07:17:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:08 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:08 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:09 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:09.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:09 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:09 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:09.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:09 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:09 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:09.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:09 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.716 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:09 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:09.716 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:09 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.717 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:09 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:09.717 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:09 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.717 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:09 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:09.717 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:09 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:09 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:09.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:09 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:09 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:09.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:09 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:09.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:09 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:09.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:09 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:10.730 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:10 smithi133 ceph-mon[31198]: pgmap v75: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-22T07:17:10.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:10 smithi154 ceph-mon[31862]: pgmap v75: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-22T07:17:10.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:10 smithi113 ceph-mon[27933]: pgmap v75: 97 pgs: 97 active+clean; 161 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff133fff640 1 -- 172.21.15.113:0/1347008110 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x558254647980 con 0x5582546f4d70 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff133fff640 10 client.25441 handle_command_reply: tid=0 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 client.25441 shutdown 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 msgr2=0x5582546e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e9490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1400ef8c0 tx=0x7ff1400bdf40 comp rx=0 tx=0).stop 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 shutdown_connections 2024-04-22T07:17:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582546e4ec0 0x5582546e99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582546dc2e0 0x5582546e9490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5582546f4d70 0x55825475ea40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 --2- 172.21.15.113:0/1347008110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5582546db9c0 0x5582546ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff133fff640 0 client.25441 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.186+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 >> 172.21.15.113:0/1347008110 conn(0x5582546d60f0 msgr2=0x5582546d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 shutdown_connections 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15d84a740 1 -- 172.21.15.113:0/1347008110 wait complete. 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15d84a740 20 client.25441 trim_cache size 0 max 16384 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:17:12.187 INFO:teuthology.orchestra.run.smithi113.stdout: "epoch": 34, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "leader": 24506, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "set_version": 8, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "sets": { 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "f75ef567": { 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "version": 8, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "age_ref": 0.0, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "QUIESCED", 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "timeout": 60.0, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "expiration": 84.1, 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "members": { 2024-04-22T07:17:12.188 INFO:teuthology.orchestra.run.smithi113.stdout: "file:/": { 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: "excluded": false, 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: "state": { 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "QUIESCED", 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.0 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:12.189 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:17:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1440a8130 msgr2=0x7ff1440aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1440a8130 0x7ff1440aa5f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005fd0 tx=0x7ff14c0098e0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 msgr2=0x7ff154196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 0x7ff154196b80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff140036040 tx=0x7ff14002ce00 comp rx=0 tx=0).stop 2024-04-22T07:17:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 shutdown_connections 2024-04-22T07:17:12.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff1440a8130 0x7ff1440aa5f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff15419aeb0 0x7ff15419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff154106170 0x7ff154196b80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 --2- 172.21.15.113:0/1474521362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1541010a0 0x7ff154196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 >> 172.21.15.113:0/1474521362 conn(0x7ff154077f50 msgr2=0x7ff1540fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 shutdown_connections 2024-04-22T07:17:12.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:12.187+0000 7ff15ac55640 1 -- 172.21.15.113:0/1474521362 wait complete. 2024-04-22T07:17:12.198 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: f75ef567, quiesce took 11.1 seconds. Will release after: 24.1 2024-04-22T07:17:12.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:12 smithi133 ceph-mon[31198]: pgmap v76: 97 pgs: 97 active+clean; 175 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 590 op/s 2024-04-22T07:17:12.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:12 smithi154 ceph-mon[31862]: pgmap v76: 97 pgs: 97 active+clean; 175 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 590 op/s 2024-04-22T07:17:12.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:12 smithi113 ceph-mon[27933]: pgmap v76: 97 pgs: 97 active+clean; 175 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 590 op/s 2024-04-22T07:17:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:13.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:13.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:13.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:13.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:14 smithi133 ceph-mon[31198]: pgmap v77: 97 pgs: 97 active+clean; 175 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 471 op/s 2024-04-22T07:17:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:17:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:14 smithi154 ceph-mon[31862]: pgmap v77: 97 pgs: 97 active+clean; 175 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 471 op/s 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:17:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:14 smithi113 ceph-mon[27933]: pgmap v77: 97 pgs: 97 active+clean; 175 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 471 op/s 2024-04-22T07:17:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:16 smithi133 ceph-mon[31198]: pgmap v78: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 554 op/s 2024-04-22T07:17:16.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:16 smithi154 ceph-mon[31862]: pgmap v78: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 554 op/s 2024-04-22T07:17:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:16 smithi113 ceph-mon[27933]: pgmap v78: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 554 op/s 2024-04-22T07:17:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:18 smithi133 ceph-mon[31198]: pgmap v79: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 432 op/s 2024-04-22T07:17:18.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:18 smithi154 ceph-mon[31862]: pgmap v79: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 432 op/s 2024-04-22T07:17:18.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:18 smithi113 ceph-mon[27933]: pgmap v79: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 432 op/s 2024-04-22T07:17:20.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:20 smithi133 ceph-mon[31198]: pgmap v80: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 432 op/s 2024-04-22T07:17:20.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:20 smithi154 ceph-mon[31862]: pgmap v80: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 432 op/s 2024-04-22T07:17:20.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:20 smithi113 ceph-mon[27933]: pgmap v80: 97 pgs: 97 active+clean; 183 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 432 op/s 2024-04-22T07:17:22.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:22 smithi133 ceph-mon[31198]: pgmap v81: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 440 op/s 2024-04-22T07:17:22.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:22 smithi154 ceph-mon[31862]: pgmap v81: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 440 op/s 2024-04-22T07:17:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:22 smithi113 ceph-mon[27933]: pgmap v81: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 440 op/s 2024-04-22T07:17:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:24 smithi133 ceph-mon[31198]: pgmap v82: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 854 KiB/s wr, 90 op/s 2024-04-22T07:17:24.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:24 smithi154 ceph-mon[31862]: pgmap v82: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 854 KiB/s wr, 90 op/s 2024-04-22T07:17:24.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:24 smithi113 ceph-mon[27933]: pgmap v82: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 854 KiB/s wr, 90 op/s 2024-04-22T07:17:26.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:26 smithi133 ceph-mon[31198]: pgmap v83: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 863 KiB/s wr, 95 op/s 2024-04-22T07:17:26.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:26 smithi154 ceph-mon[31862]: pgmap v83: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 863 KiB/s wr, 95 op/s 2024-04-22T07:17:26.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:26 smithi113 ceph-mon[27933]: pgmap v83: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 863 KiB/s wr, 95 op/s 2024-04-22T07:17:28.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:28 smithi133 ceph-mon[31198]: pgmap v84: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-04-22T07:17:28.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:28 smithi154 ceph-mon[31862]: pgmap v84: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-04-22T07:17:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:28 smithi113 ceph-mon[27933]: pgmap v84: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-04-22T07:17:30.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:30 smithi133 ceph-mon[31198]: pgmap v85: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-04-22T07:17:30.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:30 smithi154 ceph-mon[31862]: pgmap v85: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-04-22T07:17:30.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:30 smithi113 ceph-mon[27933]: pgmap v85: 97 pgs: 97 active+clean; 184 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 KiB/s wr, 12 op/s 2024-04-22T07:17:32.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:32 smithi133 ceph-mon[31198]: pgmap v86: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 284 KiB/s wr, 41 op/s 2024-04-22T07:17:32.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:32 smithi154 ceph-mon[31862]: pgmap v86: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 284 KiB/s wr, 41 op/s 2024-04-22T07:17:32.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:32 smithi113 ceph-mon[27933]: pgmap v86: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 284 KiB/s wr, 41 op/s 2024-04-22T07:17:34.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:34 smithi133 ceph-mon[31198]: pgmap v87: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 KiB/s wr, 33 op/s 2024-04-22T07:17:34.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:34 smithi154 ceph-mon[31862]: pgmap v87: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 KiB/s wr, 33 op/s 2024-04-22T07:17:34.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:34 smithi113 ceph-mon[27933]: pgmap v87: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 KiB/s wr, 33 op/s 2024-04-22T07:17:36.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:17:36.035 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- 172.21.15.113:0/2124678743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29181031b0 msgr2=0x7f2918103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- 172.21.15.113:0/2124678743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29181031b0 0x7f2918103590 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2908009900 tx=0x7f290802bcd0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- 172.21.15.113:0/2124678743 shutdown_connections 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- 172.21.15.113:0/2124678743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f291810c580 0x7f291810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- 172.21.15.113:0/2124678743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2918103ad0 0x7f291810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- 172.21.15.113:0/2124678743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29181031b0 0x7f2918103590 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- 172.21.15.113:0/2124678743 >> 172.21.15.113:0/2124678743 conn(0x7f29180771d0 msgr2=0x7f29180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- 172.21.15.113:0/2124678743 shutdown_connections 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- 172.21.15.113:0/2124678743 wait complete. 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 Processor -- start 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- start start 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29181031b0 0x7f291819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2918103ad0 0x7f291819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 0x7f29181987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f29181123e0 con 0x7f291810c580 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2918112260 con 0x7f2918103ad0 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.150+0000 7f291ee81640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2918112560 con 0x7f29181031b0 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 0x7f29181987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 0x7f29181987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40692/0 (socket says 172.21.15.113:40692) 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 -- 172.21.15.113:0/1881659415 learned_addr learned my addr 172.21.15.113:0/1881659415 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f2917fff640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2918103ad0 0x7f291819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291cc1d640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29181031b0 0x7f291819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 -- 172.21.15.113:0/1881659415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29181031b0 msgr2=0x7f291819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29181031b0 0x7f291819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 -- 172.21.15.113:0/1881659415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2918103ad0 msgr2=0x7f291819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2918103ad0 0x7f291819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 -- 172.21.15.113:0/1881659415 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2918198fb0 con 0x7f291810c580 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291d41e640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 0x7f29181987e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f291000b9e0 tx=0x7f291000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f2915ffb640 1 -- 172.21.15.113:0/1881659415 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910019070 con 0x7f291810c580 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f2915ffb640 1 -- 172.21.15.113:0/1881659415 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2910014440 con 0x7f291810c580 2024-04-22T07:17:36.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29181992a0 con 0x7f291810c580 2024-04-22T07:17:36.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.151+0000 7f2915ffb640 1 -- 172.21.15.113:0/1881659415 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910012580 con 0x7f291810c580 2024-04-22T07:17:36.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.152+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29181a54c0 con 0x7f291810c580 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.154+0000 7f2915ffb640 1 -- 172.21.15.113:0/1881659415 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f291001c070 con 0x7f291810c580 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.154+0000 7f2921a74740 1 Processor -- start 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.154+0000 7f2921a74740 1 -- start start 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2921a74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2921a74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2921a74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2921a74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5616a5eca1e0 con 0x5616a5f6ea80 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2921a74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5616a5e67390 con 0x5616a5f65580 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2921a74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5616a5ec61a0 con 0x5616a5f65ea0 2024-04-22T07:17:36.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2915ffb640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f29000a3d20 0x7f29000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f291d41e640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f29000a3d20 0x7f29000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f70e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40700/0 (socket says 172.21.15.113:40700) 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 -- 172.21.15.113:0/885740595 learned_addr learned my addr 172.21.15.113:0/885740595 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f291cc1d640 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2915ffb640 1 -- 172.21.15.113:0/1881659415 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f29100cfca0 con 0x7f291810c580 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 -- 172.21.15.113:0/885740595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 msgr2=0x5616a5f6e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f6e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 -- 172.21.15.113:0/885740595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 msgr2=0x5616a5f65960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f65960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 -- 172.21.15.113:0/885740595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a5ca67d0 con 0x5616a5f6ea80 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f2917fff640 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f70e70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f290c009f80 tx=0x7f290c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1aa599e2746a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f291d41e640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f29000a3d20 0x7f29000a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29100980d0 tx=0x7f291006e760 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.155+0000 7f28feffd640 1 -- 172.21.15.113:0/885740595 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00f040 con 0x5616a5f6ea80 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f28feffd640 1 -- 172.21.15.113:0/885740595 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f290c01c070 con 0x5616a5f6ea80 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f28feffd640 1 -- 172.21.15.113:0/885740595 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c015640 con 0x5616a5f6ea80 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 -- 172.21.15.113:0/885740595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 msgr2=0x5616a5f70e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f70e70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f290c009f80 tx=0x7f290c009580 comp rx=0 tx=0).stop 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 -- 172.21.15.113:0/885740595 shutdown_connections 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f70e70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 --2- 172.21.15.113:0/885740595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 -- 172.21.15.113:0/885740595 >> 172.21.15.113:0/885740595 conn(0x5616a5f5fcb0 msgr2=0x5616a5f60090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 -- 172.21.15.113:0/885740595 shutdown_connections 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.156+0000 7f2921a74740 1 -- 172.21.15.113:0/885740595 wait complete. 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 -- start start 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 Processor -- start 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f6c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f6c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5616a5f71870 con 0x5616a5f65ea0 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5616a5e832f0 con 0x5616a5f6ea80 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5616a5f719f0 con 0x5616a5f65580 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2921a74740 10 client.0 _ll_register_callbacks cb 0x5616a5e90580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f291cc1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f6c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f291cc1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f6c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40716/0 (socket says 172.21.15.113:40716) 2024-04-22T07:17:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f291cc1d640 1 -- 172.21.15.113:0/761362189 learned_addr learned my addr 172.21.15.113:0/761362189 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2917fff640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f6c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.157+0000 7f2917fff640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f291cc1d640 1 -- 172.21.15.113:0/761362189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 msgr2=0x5616a5f6bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f291cc1d640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f291cc1d640 1 -- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 msgr2=0x5616a5f6c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f291cc1d640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f6c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f291cc1d640 1 -- 172.21.15.113:0/761362189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a5bdc3f0 con 0x5616a5f65ea0 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2917fff640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f291cc1d640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f6c080 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f290802c210 tx=0x7f29080058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18f6f47396b4970 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f28f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f28f7fff640 1 -- 172.21.15.113:0/761362189 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908005230 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f28f7fff640 1 -- 172.21.15.113:0/761362189 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f29080053d0 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c01c070 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f28f7fff640 1 -- 172.21.15.113:0/761362189 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2908004490 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 10 client.16272 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5616a5b9c0f0 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f28f7fff640 1 -- 172.21.15.113:0/761362189 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f290c01c070 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f28f7fff640 1 -- 172.21.15.113:0/761362189 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2908003800 con 0x5616a5f65ea0 2024-04-22T07:17:36.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 10 client.16272 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 20 client.16272 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.158+0000 7f2921a74740 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5f7ebd0 0x5616a5f81010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.159+0000 7f2921a74740 4 client.16272 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.159+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616a5ed7c00 con 0x5616a5f7ebd0 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.159+0000 7f2917fff640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5f7ebd0 0x5616a5f81010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.159+0000 7f2917fff640 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5f7ebd0 0x5616a5f81010 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daaf966610bb1c4a server_cookie=928ca443d4b8a986 in_seq=0 out_seq=0 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.159+0000 7f28f7fff640 10 client.16272 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f28f7fff640 1 -- 172.21.15.113:0/761362189 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616a5ed7c00 con 0x5616a5f7ebd0 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f28f7fff640 10 client.16272 handle_command_reply: tid=0 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 client.16272 shutdown 2024-04-22T07:17:36.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 msgr2=0x5616a5f6c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f6c080 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f290802c210 tx=0x7f29080058b0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 shutdown_connections 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f6c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5f7ebd0 0x5616a5f81010 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f6c080 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 --2- 172.21.15.113:0/761362189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f28f7fff640 0 client.16272 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 >> 172.21.15.113:0/761362189 conn(0x5616a5f5fcb0 msgr2=0x5616a5f61a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 shutdown_connections 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 1 -- 172.21.15.113:0/761362189 wait complete. 2024-04-22T07:17:36.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.160+0000 7f2921a74740 20 client.16272 trim_cache size 0 max 16384 2024-04-22T07:17:36.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.177+0000 7f2921a74740 1 Processor -- start 2024-04-22T07:17:36.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.177+0000 7f2921a74740 1 -- start start 2024-04-22T07:17:36.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2921a74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2921a74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2921a74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2921a74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f29181b0770 con 0x5616a5f65ea0 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2921a74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29181adf80 con 0x5616a5f6ea80 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2921a74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f29181abdd0 con 0x5616a5f65580 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f7a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40726/0 (socket says 172.21.15.113:40726) 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 -- 172.21.15.113:0/1224940915 learned_addr learned my addr 172.21.15.113:0/1224940915 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2917fff640 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2917fff640 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 -- 172.21.15.113:0/1224940915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 msgr2=0x5616a5f6e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 -- 172.21.15.113:0/1224940915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 msgr2=0x5616a5f60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 -- 172.21.15.113:0/1224940915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a5f60ee0 con 0x5616a5f65ea0 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f2917fff640 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f291cc1d640 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f7a760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2908000c00 tx=0x7f290802cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8568b599267907 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f28fe7fc640 1 -- 172.21.15.113:0/1224940915 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908053020 con 0x5616a5f65ea0 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f28fe7fc640 1 -- 172.21.15.113:0/1224940915 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2908009520 con 0x5616a5f65ea0 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.178+0000 7f28fe7fc640 1 -- 172.21.15.113:0/1224940915 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908004920 con 0x5616a5f65ea0 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 -- 172.21.15.113:0/1224940915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 msgr2=0x5616a5f7a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f7a760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2908000c00 tx=0x7f290802cd40 comp rx=0 tx=0).stop 2024-04-22T07:17:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 -- 172.21.15.113:0/1224940915 shutdown_connections 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f6ea80 0x5616a5f60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f65ea0 0x5616a5f7a760 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 --2- 172.21.15.113:0/1224940915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65580 0x5616a5f6e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 -- 172.21.15.113:0/1224940915 >> 172.21.15.113:0/1224940915 conn(0x5616a5f5fcb0 msgr2=0x5616a5f610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 -- 172.21.15.113:0/1224940915 shutdown_connections 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 -- 172.21.15.113:0/1224940915 wait complete. 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 -- start start 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.179+0000 7f2921a74740 1 Processor -- start 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f73360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f29181ac440 con 0x5616a5f6ea80 2024-04-22T07:17:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29181ad8b0 con 0x5616a5f65580 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f29181acb10 con 0x5616a5f65ea0 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 10 client.0 _ll_register_callbacks cb 0x7f2908009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f73360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f73360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40730/0 (socket says 172.21.15.113:40730) 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 -- 172.21.15.113:0/2608810125 learned_addr learned my addr 172.21.15.113:0/2608810125 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f291cc1d640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 -- 172.21.15.113:0/2608810125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 msgr2=0x5616a5f72e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f72e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 -- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 msgr2=0x5616a5f728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 -- 172.21.15.113:0/2608810125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a5fe43e0 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2917fff640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f73360 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f290c010b10 tx=0x7f290802cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baad496aa2abc69b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f28f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f28f7fff640 1 -- 172.21.15.113:0/2608810125 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c035070 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f28f7fff640 1 -- 172.21.15.113:0/2608810125 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f290c02d5f0 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2908009520 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.180+0000 7f28f7fff640 1 -- 172.21.15.113:0/2608810125 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f290c02d7f0 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 10 client.16278 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5616a5fe56c0 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f28f7fff640 1 -- 172.21.15.113:0/2608810125 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2908009520 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f28f7fff640 1 -- 172.21.15.113:0/2608810125 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f290c038020 con 0x5616a5f6ea80 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 10 client.16278 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:36.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 20 client.16278 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5fe5d30 0x5616a5fe8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 4 client.16278 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616a5ed1a70 con 0x5616a5fe5d30 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.181+0000 7f291cc1d640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5fe5d30 0x5616a5fe8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f28f7fff640 10 client.16278 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f291cc1d640 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5fe5d30 0x5616a5fe8150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aadfac81da12f9b server_cookie=e56f90c369575ed8 in_seq=0 out_seq=0 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f28f7fff640 1 -- 172.21.15.113:0/2608810125 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5616a5ed1a70 con 0x5616a5fe5d30 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f28f7fff640 10 client.16278 handle_command_reply: tid=0 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 client.16278 shutdown 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 msgr2=0x5616a5f73360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f73360 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f290c010b10 tx=0x7f290802cdb0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 shutdown_connections 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x5616a5fe5d30 0x5616a5fe8150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5616a5f6ea80 0x5616a5f73360 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5616a5f65ea0 0x5616a5f72e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f28f7fff640 0 client.16278 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 --2- 172.21.15.113:0/2608810125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616a5f65580 0x5616a5f728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 >> 172.21.15.113:0/2608810125 conn(0x5616a5f5fcb0 msgr2=0x5616a5f5e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 shutdown_connections 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 1 -- 172.21.15.113:0/2608810125 wait complete. 2024-04-22T07:17:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.182+0000 7f2921a74740 20 client.16278 trim_cache size 0 max 16384 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: "a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8": { 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8", 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:17:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:17:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f29000a3d20 msgr2=0x7f29000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f29000a3d20 0x7f29000a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29100980d0 tx=0x7f291006e760 comp rx=0 tx=0).stop 2024-04-22T07:17:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 msgr2=0x7f29181987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 0x7f29181987e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f291000b9e0 tx=0x7f291000bde0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 shutdown_connections 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f29000a3d20 0x7f29000a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291810c580 0x7f29181987e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2918103ad0 0x7f291819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 --2- 172.21.15.113:0/1881659415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29181031b0 0x7f291819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 >> 172.21.15.113:0/1881659415 conn(0x7f29180771d0 msgr2=0x7f29180ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 shutdown_connections 2024-04-22T07:17:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.183+0000 7f291ee81640 1 -- 172.21.15.113:0/1881659415 wait complete. 2024-04-22T07:17:36.192 INFO:tasks.cephfs.filesystem:scrub status for tag:a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8 - {'path': '/', 'tag': 'a2b1c5a7-53e5-438a-a6bb-f227c7ebd9c8', 'options': 'recursive,force'} 2024-04-22T07:17:36.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:17:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 -- 172.21.15.113:0/2566530325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 msgr2=0x7fa4c810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/2566530325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c810dc90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009cd0 tx=0x7fa4c002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 -- 172.21.15.113:0/2566530325 shutdown_connections 2024-04-22T07:17:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/2566530325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c810dc90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/2566530325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81011a0 0x7fa4c8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/2566530325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8105e30 0x7fa4c8106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 -- 172.21.15.113:0/2566530325 >> 172.21.15.113:0/2566530325 conn(0x7fa4c8077e30 msgr2=0x7fa4c8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 -- 172.21.15.113:0/2566530325 shutdown_connections 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 -- 172.21.15.113:0/2566530325 wait complete. 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 Processor -- start 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.400+0000 7fa4cf89a640 1 -- start start 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81011a0 0x7fa4c81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8105e30 0x7fa4c81a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8111240 con 0x7fa4c8101b60 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81110c0 con 0x7fa4c8105e30 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81113c0 con 0x7fa4c81011a0 2024-04-22T07:17:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40744/0 (socket says 172.21.15.113:40744) 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 -- 172.21.15.113:0/1741326865 learned_addr learned my addr 172.21.15.113:0/1741326865 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cd636640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81011a0 0x7fa4c81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cde37640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8105e30 0x7fa4c81a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 -- 172.21.15.113:0/1741326865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81011a0 msgr2=0x7fa4c81995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81011a0 0x7fa4c81995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 -- 172.21.15.113:0/1741326865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8105e30 msgr2=0x7fa4c81a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8105e30 0x7fa4c81a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 -- 172.21.15.113:0/1741326865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c81a4ba0 con 0x7fa4c8101b60 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cce35640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c8199b20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800d770 tx=0x7fa4b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8017070 con 0x7fa4c8101b60 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4b80054f0 con 0x7fa4c8101b60 2024-04-22T07:17:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8004de0 con 0x7fa4c8101b60 2024-04-22T07:17:36.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c81a4d70 con 0x7fa4c8101b60 2024-04-22T07:17:36.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.401+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c81a5210 con 0x7fa4c8101b60 2024-04-22T07:17:36.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.402+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c8068de0 con 0x7fa4c8101b60 2024-04-22T07:17:36.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.403+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa4b8004f80 con 0x7fa4c8101b60 2024-04-22T07:17:36.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.404+0000 7fa4be7fc640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d20 0x7fa4b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.404+0000 7fa4cd636640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d20 0x7fa4b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.404+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7fa4b80d02e0 con 0x7fa4c8101b60 2024-04-22T07:17:36.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.404+0000 7fa4cd636640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d20 0x7fa4b40a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4005d00 tx=0x7fa4c400b410 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.408+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4b809c2b0 con 0x7fa4c8101b60 2024-04-22T07:17:36.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:36 smithi113 ceph-mon[27933]: pgmap v88: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 312 KiB/s wr, 37 op/s 2024-04-22T07:17:36.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.623+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4c80696f0 con 0x7fa4c8101b60 2024-04-22T07:17:36.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.624+0000 7fa4be7fc640 1 -- 172.21.15.113:0/1741326865 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11890 (secure 0 0 0) 0x7fa4b8096e20 con 0x7fa4c8101b60 2024-04-22T07:17:36.622 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:17:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:16:39.519389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24445,"mds_2":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":26,"state":"up:active","state_seq":54,"addr":"172.21.15.154:6835/217490889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":217490889},{"type":"v1","addr":"172.21.15.154:6835","nonce":217490889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"c","rank":1,"incarnation":24,"state":"up:active","state_seq":53,"addr":"172.21.15.154:6837/1615298521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1615298521},{"type":"v1","addr":"172.21.15.154:6837","nonce":1615298521}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/982201245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":982201245},{"type":"v1","addr":"172.21.15.133:6839","nonce":982201245}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24433,24445,24506]},"id":1}]} 2024-04-22T07:17:36.622 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 34 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d20 msgr2=0x7fa4b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d20 0x7fa4b40a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4005d00 tx=0x7fa4c400b410 comp rx=0 tx=0).stop 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 msgr2=0x7fa4c8199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c8199b20 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800d770 tx=0x7fa4b800dc30 comp rx=0 tx=0).stop 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 shutdown_connections 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d20 0x7fa4b40a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8105e30 0x7fa4c81a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8101b60 0x7fa4c8199b20 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 --2- 172.21.15.113:0/1741326865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81011a0 0x7fa4c81995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 >> 172.21.15.113:0/1741326865 conn(0x7fa4c8077e30 msgr2=0x7fa4c80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 shutdown_connections 2024-04-22T07:17:36.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.626+0000 7fa4cf89a640 1 -- 172.21.15.113:0/1741326865 wait complete. 2024-04-22T07:17:36.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id f75ef567 --release --await' 2024-04-22T07:17:36.635 DEBUG:teuthology.orchestra.run.smithi113:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24506 quiesce db --set-id f75ef567 --release --await 2024-04-22T07:17:36.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- 172.21.15.113:0/545705191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4106160 msgr2=0x7f22f410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- 172.21.15.113:0/545705191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4106160 0x7f22f410dd30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009ca0 tx=0x7f22ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- 172.21.15.113:0/545705191 shutdown_connections 2024-04-22T07:17:36.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- 172.21.15.113:0/545705191 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4106160 0x7f22f410dd30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- 172.21.15.113:0/545705191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f4101a40 0x7f22f4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- 172.21.15.113:0/545705191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f4101090 0x7f22f4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- 172.21.15.113:0/545705191 >> 172.21.15.113:0/545705191 conn(0x7f22f4077f50 msgr2=0x7f22f4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- 172.21.15.113:0/545705191 shutdown_connections 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- 172.21.15.113:0/545705191 wait complete. 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 Processor -- start 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- start start 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4101090 0x7f22f40fec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f4106160 0x7f22f40ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f40ff6a0 0x7f22f41a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41111e0 con 0x7f22f4101090 2024-04-22T07:17:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4111060 con 0x7f22f4106160 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.742+0000 7f22fc1e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4111360 con 0x7f22f40ff6a0 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4101090 0x7f22f40fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4101090 0x7f22f40fec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40762/0 (socket says 172.21.15.113:40762) 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 -- 172.21.15.113:0/290823437 learned_addr learned my addr 172.21.15.113:0/290823437 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9781640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f4106160 0x7f22f40ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22fa783640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f40ff6a0 0x7f22f41a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 -- 172.21.15.113:0/290823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f40ff6a0 msgr2=0x7f22f41a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22f40ff6a0 0x7f22f41a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 -- 172.21.15.113:0/290823437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f4106160 msgr2=0x7f22f40ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f4106160 0x7f22f40ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 -- 172.21.15.113:0/290823437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f41a6fa0 con 0x7f22f4101090 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22f9f82640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22f4101090 0x7f22f40fec20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c920 tx=0x7f22f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22eaffd640 1 -- 172.21.15.113:0/290823437 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0007c70 con 0x7f22f4101090 2024-04-22T07:17:36.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22eaffd640 1 -- 172.21.15.113:0/290823437 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f22f0007e10 con 0x7f22f4101090 2024-04-22T07:17:36.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22fc1e6640 1 -- 172.21.15.113:0/290823437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f41a7290 con 0x7f22f4101090 2024-04-22T07:17:36.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22eaffd640 1 -- 172.21.15.113:0/290823437 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f00188c0 con 0x7f22f4101090 2024-04-22T07:17:36.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.743+0000 7f22fc1e6640 1 -- 172.21.15.113:0/290823437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f41a7830 con 0x7f22f4101090 2024-04-22T07:17:36.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.745+0000 7f22eaffd640 1 -- 172.21.15.113:0/290823437 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f22f0018a60 con 0x7f22f4101090 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.745+0000 7f22fedd3740 1 Processor -- start 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 -- start start 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d5e6636500 con 0x55d5e66d1c30 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d5e65dc6a0 con 0x55d5e66da810 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fedd3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d5e66324c0 con 0x55d5e66d1310 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22eaffd640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f22e00a3dd0 0x7f22e00a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22f9f82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fa783640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f22e00a3dd0 0x7f22e00a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22f9f82640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66da2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40776/0 (socket says 172.21.15.113:40776) 2024-04-22T07:17:36.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22f9f82640 1 -- 172.21.15.113:0/2552547053 learned_addr learned my addr 172.21.15.113:0/2552547053 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22f9781640 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22eaffd640 1 -- 172.21.15.113:0/290823437 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f22f00d0930 con 0x7f22f4101090 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22fa783640 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22f9f82640 1 -- 172.21.15.113:0/2552547053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 msgr2=0x55d5e66d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.746+0000 7f22f9f82640 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22f9f82640 1 -- 172.21.15.113:0/2552547053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 msgr2=0x55d5e66dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22f9f82640 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66dcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22f9f82640 1 -- 172.21.15.113:0/2552547053 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e65fb5e0 con 0x55d5e66d1c30 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22fa783640 1 --2- 172.21.15.113:0/290823437 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f22e00a3dd0 0x7f22e00a6290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009430 tx=0x7f22ec009880 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22f9f82640 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66da2d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22f0099220 tx=0x7f22f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5968c122f31e1f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22dffff640 1 -- 172.21.15.113:0/2552547053 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f009cf50 con 0x55d5e66d1c30 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22dffff640 1 -- 172.21.15.113:0/2552547053 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f22f0060040 con 0x55d5e66d1c30 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.747+0000 7f22dffff640 1 -- 172.21.15.113:0/2552547053 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0018d50 con 0x55d5e66d1c30 2024-04-22T07:17:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- 172.21.15.113:0/2552547053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 msgr2=0x55d5e66da2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66da2d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f22f0099220 tx=0x7f22f0000bc0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- 172.21.15.113:0/2552547053 shutdown_connections 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66da2d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- 172.21.15.113:0/2552547053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- 172.21.15.113:0/2552547053 >> 172.21.15.113:0/2552547053 conn(0x55d5e66cba40 msgr2=0x55d5e66cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- 172.21.15.113:0/2552547053 shutdown_connections 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- 172.21.15.113:0/2552547053 wait complete. 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- start start 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 Processor -- start 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1c30 0x55d5e66d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 0x55d5e66d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d5e66dd5a0 con 0x55d5e66da810 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d5e65efa60 con 0x55d5e66d1310 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.748+0000 7f22fedd3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d5e66dd720 con 0x55d5e66d1c30 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fedd3740 10 client.0 _ll_register_callbacks cb 0x55d5e6348110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fa783640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 0x55d5e66d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66d7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:53866/0 (socket says 172.21.15.113:53866) 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fa783640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 0x55d5e66d80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40784/0 (socket says 172.21.15.113:40784) 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 -- 172.21.15.113:0/3959947679 learned_addr learned my addr 172.21.15.113:0/3959947679 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9781640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1c30 0x55d5e66d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 -- 172.21.15.113:0/3959947679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1c30 msgr2=0x55d5e66d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1c30 0x55d5e66d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 -- 172.21.15.113:0/3959947679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 msgr2=0x55d5e66d80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 0x55d5e66d80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 -- 172.21.15.113:0/3959947679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e6348920 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fa783640 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 0x55d5e66d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22f9f82640 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66d7670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22f00ce0b0 tx=0x7f22f009cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89cca0e60b7e8d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22dd7fa640 1 -- 172.21.15.113:0/3959947679 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f009c850 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22dd7fa640 1 -- 172.21.15.113:0/3959947679 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f22f0018d50 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f0060040 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22dd7fa640 1 -- 172.21.15.113:0/3959947679 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f22f00870f0 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fedd3740 10 client.25712 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.749+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d5e6414d30 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22dd7fa640 1 -- 172.21.15.113:0/3959947679 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f22f0060040 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22dd7fa640 1 -- 172.21.15.113:0/3959947679 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f22f00b2060 con 0x55d5e66d1310 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22fedd3740 10 client.25712 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22fedd3740 10 client.25712 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:17:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22fedd3740 20 client.25712 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22fedd3740 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e66ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22fedd3740 4 client.25712 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e6609a10 con 0x55d5e66ea7f0 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22f9781640 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e66ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22dd7fa640 10 client.25712 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.750+0000 7f22f9781640 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e66ecbe0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0aa7afc6e39d6f5 server_cookie=cd88d7d9700614b0 in_seq=0 out_seq=0 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.751+0000 7f22dd7fa640 1 -- 172.21.15.113:0/3959947679 <== mds.0 v2:172.21.15.133:6838/982201245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5e6609a10 con 0x55d5e66ea7f0 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.751+0000 7f22dd7fa640 10 client.25712 handle_command_reply: tid=0 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.751+0000 7f22fedd3740 1 client.25712 shutdown 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.751+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 msgr2=0x55d5e66d7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.751+0000 7f22fedd3740 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66d7670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22f00ce0b0 tx=0x7f22f009cf50 comp rx=0 tx=0).stop 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 shutdown_connections 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e66ecbe0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66da810 0x55d5e66d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1c30 0x55d5e66d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22dd7fa640 0 client.25712 ms_handle_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 --2- 172.21.15.113:0/3959947679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66d7670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 >> 172.21.15.113:0/3959947679 conn(0x55d5e66cba40 msgr2=0x55d5e66cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 shutdown_connections 2024-04-22T07:17:36.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 1 -- 172.21.15.113:0/3959947679 wait complete. 2024-04-22T07:17:36.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.752+0000 7f22fedd3740 20 client.25712 trim_cache size 0 max 16384 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.768+0000 7f22fedd3740 1 Processor -- start 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.768+0000 7f22fedd3740 1 -- start start 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22fedd3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41b2870 con 0x55d5e66d1c30 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22fedd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41b0080 con 0x55d5e66da810 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22fedd3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41aded0 con 0x55d5e66d1310 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40788/0 (socket says 172.21.15.113:40788) 2024-04-22T07:17:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 -- 172.21.15.113:0/1287315620 learned_addr learned my addr 172.21.15.113:0/1287315620 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 -- 172.21.15.113:0/1287315620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 msgr2=0x55d5e66d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 -- 172.21.15.113:0/1287315620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 msgr2=0x55d5e66f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 -- 172.21.15.113:0/1287315620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e66cb1b0 con 0x55d5e66d1c30 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22f9781640 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66e6460 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x55d5e66ccbd0 tx=0x7f22e400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48295eab876870b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22df7fe640 1 -- 172.21.15.113:0/1287315620 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4003ac0 con 0x55d5e66d1c30 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22df7fe640 1 -- 172.21.15.113:0/1287315620 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f22e4024020 con 0x55d5e66d1c30 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.769+0000 7f22df7fe640 1 -- 172.21.15.113:0/1287315620 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4011860 con 0x55d5e66d1c30 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 -- 172.21.15.113:0/1287315620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 msgr2=0x55d5e66e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66e6460 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x55d5e66ccbd0 tx=0x7f22e400b390 comp rx=0 tx=0).stop 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 -- 172.21.15.113:0/1287315620 shutdown_connections 2024-04-22T07:17:36.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66da810 0x55d5e66f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66e6460 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 --2- 172.21.15.113:0/1287315620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66d1310 0x55d5e66d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 -- 172.21.15.113:0/1287315620 >> 172.21.15.113:0/1287315620 conn(0x55d5e66cba40 msgr2=0x55d5e66ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 -- 172.21.15.113:0/1287315620 shutdown_connections 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 -- 172.21.15.113:0/1287315620 wait complete. 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 -- start start 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.770+0000 7f22fedd3740 1 Processor -- start 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66def80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66da810 0x55d5e674fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41ae540 con 0x55d5e66d1c30 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41af9b0 con 0x55d5e66d1310 2024-04-22T07:17:36.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41aec10 con 0x55d5e66da810 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 10 client.0 _ll_register_callbacks cb 0x7f22e400b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66def80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40800/0 (socket says 172.21.15.113:40800) 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 -- 172.21.15.113:0/83262998 learned_addr learned my addr 172.21.15.113:0/83262998 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fa783640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66da810 0x55d5e674fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66da810 msgr2=0x55d5e674fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66da810 0x55d5e674fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 msgr2=0x55d5e66dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d5e66d1310 0x55d5e66dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 -- 172.21.15.113:0/83262998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e674ffc0 con 0x55d5e66d1c30 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fa783640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d5e66da810 0x55d5e674fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22f9781640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d5e66d1c30 0x55d5e66def80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f22e4023040 tx=0x7f22e4003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccfa086932ae1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:17:36.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22dd7fa640 1 -- 172.21.15.113:0/83262998 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4005450 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22dd7fa640 1 -- 172.21.15.113:0/83262998 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8641+0+0 (secure 0 0 0) 0x7f22e40055f0 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 -- 172.21.15.113:0/83262998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e4024020 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22dd7fa640 1 -- 172.21.15.113:0/83262998 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f22e4037050 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 10 client.16302 fetch_fsmap learned FSMap version 34 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22fedd3740 1 -- 172.21.15.113:0/83262998 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d5e6751270 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.771+0000 7f22dd7fa640 1 -- 172.21.15.113:0/83262998 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f22e4024020 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22dd7fa640 1 -- 172.21.15.113:0/83262998 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f22e40206b0 con 0x55d5e66d1c30 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22fedd3740 10 client.16302 fetch_fsmap finished waiting for FSMap version 34 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22fedd3740 10 client.16302 resolve_mds: validated gid 24506 aka daemon mds.b 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22fedd3740 20 client.16302 populate_metadata read hostname 'smithi113' 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22fedd3740 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:17:36.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22fedd3740 4 client.16302 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f75ef567", "release": true, "await": true}] 2024-04-22T07:17:36.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22fedd3740 1 -- 172.21.15.113:0/83262998 --> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f75ef567", "release": true, "await": true}) -- 0x55d5e6643f20 con 0x55d5e66ea7f0 2024-04-22T07:17:36.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:36.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22dd7fa640 10 client.16302 ms_handle_connect on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.772+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99afd3f9664fe5c server_cookie=ae4b438ae554ea8 in_seq=0 out_seq=0 2024-04-22T07:17:36.776 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-mon[31198]: pgmap v88: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 312 KiB/s wr, 37 op/s 2024-04-22T07:17:36.776 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-mon[31862]: pgmap v88: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 312 KiB/s wr, 37 op/s 2024-04-22T07:17:36.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:17:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:17:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:17:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-22T07:17:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:6838/982201245 2024-04-22T07:17:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=101 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:17:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:36.813+0000 7f22dd7fa640 1 client.16302 ms_handle_refused on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:37.014+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:6838/982201245 2024-04-22T07:17:37.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:37.014+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=101 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:17:37.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:37.014+0000 7f22dd7fa640 1 client.16302 ms_handle_refused on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5d512b1700 time 2024-04-22T07:17:36.775032+0000 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5d640eb3dd] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.065 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: clone() 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:36.775+0000 7f5d512b1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5d512b1700 time 2024-04-22T07:17:36.775032+0000 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5d640eb3dd] 2024-04-22T07:17:37.066 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f2ae4b68700 time 2024-04-22T07:17:36.774094+0000 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2af81a33dd] 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.067 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: clone() 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:36.774+0000 7f2ae4b68700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f2ae4b68700 time 2024-04-22T07:17:36.774094+0000 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2af81a33dd] 2024-04-22T07:17:37.068 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fec0ca0f700 time 2024-04-22T07:17:36.774136+0000 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec2004a3dd] 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.069 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: clone() 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:36.774+0000 7fec0ca0f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fec0ca0f700 time 2024-04-22T07:17:36.774136+0000 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec2004a3dd] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.070 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: clone() 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: *** Caught signal (Aborted) ** 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: in thread 7fec0ca0f700 thread_name:quiesce.agt 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec1edd2d20] 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: gsignal() 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: abort() 2024-04-22T07:17:37.071 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec2004a437] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 13: clone() 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:36.776+0000 7fec0ca0f700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.072 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: in thread 7fec0ca0f700 thread_name:quiesce.agt 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: clone() 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: *** Caught signal (Aborted) ** 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: in thread 7f2ae4b68700 thread_name:quiesce.agt 2024-04-22T07:17:37.073 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2af6f2bd20] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: gsignal() 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: abort() 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2af81a3437] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.074 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 13: clone() 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:36.776+0000 7f2ae4b68700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: in thread 7f2ae4b68700 thread_name:quiesce.agt 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: clone() 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: *** Caught signal (Aborted) ** 2024-04-22T07:17:37.075 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: in thread 7f5d512b1700 thread_name:quiesce.agt 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5d62e73d20] 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: gsignal() 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: abort() 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5d640eb437] 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.076 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 13: clone() 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:36.776+0000 7f5d512b1700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: in thread 7f5d512b1700 thread_name:quiesce.agt 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5d62e73d20] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: gsignal() 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: abort() 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5d640eb437] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.077 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 13: clone() 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: -1> 2024-04-22T07:17:36.775+0000 7f5d512b1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5d512b1700 time 2024-04-22T07:17:36.775032+0000 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.078 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5d640eb3dd] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: clone() 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 0> 2024-04-22T07:17:36.776+0000 7f5d512b1700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.079 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: in thread 7f5d512b1700 thread_name:quiesce.agt 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5d62e73d20] 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: gsignal() 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: abort() 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5d640eb437] 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.080 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 13: clone() 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: -1> 2024-04-22T07:17:36.775+0000 7f5d512b1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f5d512b1700 time 2024-04-22T07:17:36.775032+0000 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5d640eb3dd] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.081 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: clone() 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 0> 2024-04-22T07:17:36.776+0000 7f5d512b1700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: in thread 7f5d512b1700 thread_name:quiesce.agt 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.082 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5d62e73d20] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2: gsignal() 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 3: abort() 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5d640eb437] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f5d640eb5a3] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x558a2fe30e9c] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x558a2fe30f9f] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x558a2fd5f0d3] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x558a2fd5f03c] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x558a2ffded3f] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x558a30047560] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f5d62e691ca] 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 13: clone() 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[61896]: 2024-04-22T07:17:37.083 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:36 smithi133 podman[65365]: 2024-04-22 07:17:36.848701629 +0000 UTC m=+0.022583999 container died b19aca13ef750ab1f688fcc11759debdcf990ded1d09711f5733bacbcb6786e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec1edd2d20] 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: gsignal() 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: abort() 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec2004a437] 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.084 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 13: clone() 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: -1> 2024-04-22T07:17:36.774+0000 7fec0ca0f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fec0ca0f700 time 2024-04-22T07:17:36.774136+0000 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec2004a3dd] 2024-04-22T07:17:37.085 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2af6f2bd20] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: gsignal() 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: abort() 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2af81a3437] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.086 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 13: clone() 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: -1> 2024-04-22T07:17:36.774+0000 7f2ae4b68700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f2ae4b68700 time 2024-04-22T07:17:36.774094+0000 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2af81a33dd] 2024-04-22T07:17:37.087 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: clone() 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 0> 2024-04-22T07:17:36.776+0000 7fec0ca0f700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: in thread 7fec0ca0f700 thread_name:quiesce.agt 2024-04-22T07:17:37.088 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec1edd2d20] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: gsignal() 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: abort() 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec2004a437] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.089 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 13: clone() 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.090 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: clone() 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 0> 2024-04-22T07:17:36.776+0000 7f2ae4b68700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: in thread 7f2ae4b68700 thread_name:quiesce.agt 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2af6f2bd20] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: gsignal() 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: abort() 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2af81a3437] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.091 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 13: clone() 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: -1> 2024-04-22T07:17:36.774+0000 7fec0ca0f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fec0ca0f700 time 2024-04-22T07:17:36.774136+0000 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec2004a3dd] 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.092 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: clone() 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 0> 2024-04-22T07:17:36.776+0000 7fec0ca0f700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: in thread 7fec0ca0f700 thread_name:quiesce.agt 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec1edd2d20] 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2: gsignal() 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 3: abort() 2024-04-22T07:17:37.093 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec2004a437] 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec2004a5a3] 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: -1> 2024-04-22T07:17:36.774+0000 7f2ae4b68700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f2ae4b68700 time 2024-04-22T07:17:36.774094+0000 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el8/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2af81a33dd] 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.094 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: clone() 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 0> 2024-04-22T07:17:36.776+0000 7f2ae4b68700 -1 *** Caught signal (Aborted) ** 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: in thread 7f2ae4b68700 thread_name:quiesce.agt 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2af6f2bd20] 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2: gsignal() 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 3: abort() 2024-04-22T07:17:37.095 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2af81a3437] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2af81a35a3] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x562eb037be9c] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x562eb037bf9f] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x562eb02aa0d3] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x562eb02aa03c] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x562eb0529d3f] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x562eb0592560] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2af6f211ca] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 13: clone() 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[63283]: 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 6: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xa2c) [0x563e19924e9c] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 7: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x7f) [0x563e19924f9f] 2024-04-22T07:17:37.096 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 8: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x243) [0x563e198530d3] 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1ac) [0x563e1985303c] 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 10: /usr/bin/ceph-mds(+0x60dd3f) [0x563e19ad2d3f] 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 11: (QuiesceAgent::agent_thread_main()+0x1680) [0x563e19b3b560] 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec1edc81ca] 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 13: clone() 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[62958]: 2024-04-22T07:17:37.097 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:36 smithi154 podman[67098]: 2024-04-22 07:17:36.863793288 +0000 UTC m=+0.023600073 container died 1dd73fe23a4974c088fd820e1103b90e679e7de915c1ecab9d436b3bfba461e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-22T07:17:37.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:37.415+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:6838/982201245 2024-04-22T07:17:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:37.415+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=101 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:17:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:37.415+0000 7f22dd7fa640 1 client.16302 ms_handle_refused on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:37.772 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:37 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1741326865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:37.773 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:37 smithi133 podman[65365]: 2024-04-22 07:17:37.532452373 +0000 UTC m=+0.706334787 container remove b19aca13ef750ab1f688fcc11759debdcf990ded1d09711f5733bacbcb6786e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T07:17:37.773 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:37 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service: Main process exited, code=exited, status=134/n/a 2024-04-22T07:17:37.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:37 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1741326865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:37.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:37 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1741326865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:17:38.064 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:37 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service: Failed with result 'exit-code'. 2024-04-22T07:17:38.064 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:37 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service: Consumed 9.475s CPU time. 2024-04-22T07:17:38.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:38.216+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:6838/982201245 2024-04-22T07:17:38.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:38.216+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=101 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:17:38.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:38.216+0000 7f22dd7fa640 1 client.16302 ms_handle_refused on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:38.733 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:38 smithi113 ceph-mon[27933]: pgmap v89: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 303 KiB/s wr, 32 op/s 2024-04-22T07:17:38.800 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:38 smithi154 ceph-mon[31862]: pgmap v89: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 303 KiB/s wr, 32 op/s 2024-04-22T07:17:38.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:38 smithi133 ceph-mon[31198]: pgmap v89: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 303 KiB/s wr, 32 op/s 2024-04-22T07:17:39.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:39.817+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:6838/982201245 2024-04-22T07:17:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:39.817+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=101 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T07:17:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:39.817+0000 7f22dd7fa640 1 client.16302 ms_handle_refused on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:40.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:40 smithi133 ceph-mon[31198]: pgmap v90: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 303 KiB/s wr, 32 op/s 2024-04-22T07:17:40.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:40 smithi154 ceph-mon[31862]: pgmap v90: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 303 KiB/s wr, 32 op/s 2024-04-22T07:17:40.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:40 smithi113 ceph-mon[27933]: pgmap v90: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 303 KiB/s wr, 32 op/s 2024-04-22T07:17:41.240 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:40 smithi154 podman[67099]: 2024-04-22 07:17:40.898803356 +0000 UTC m=+4.057621863 container died 3b72115b888e18c32f9321f0a102017386a5df0ddfbc2aed0220e98ef0c02088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-22T07:17:41.240 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:40 smithi154 podman[67098]: 2024-04-22 07:17:40.830514657 +0000 UTC m=+3.990321442 container remove 1dd73fe23a4974c088fd820e1103b90e679e7de915c1ecab9d436b3bfba461e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T07:17:41.240 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:40 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-04-22T07:17:41.491 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:41 smithi154 podman[67099]: 2024-04-22 07:17:41.483991205 +0000 UTC m=+4.642809710 container remove 3b72115b888e18c32f9321f0a102017386a5df0ddfbc2aed0220e98ef0c02088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-22T07:17:41.492 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:41 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-22T07:17:42.065 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:41 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-22T07:17:42.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:41 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service: Consumed 48.583s CPU time. 2024-04-22T07:17:42.066 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:41 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-22T07:17:42.066 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:41 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service: Consumed 4.003s CPU time. 2024-04-22T07:17:42.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:42 smithi133 ceph-mon[31198]: pgmap v91: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 305 KiB/s wr, 35 op/s 2024-04-22T07:17:42.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:42 smithi154 ceph-mon[31862]: pgmap v91: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 305 KiB/s wr, 35 op/s 2024-04-22T07:17:42.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:42 smithi113 ceph-mon[27933]: pgmap v91: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 305 KiB/s wr, 35 op/s 2024-04-22T07:17:43.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:43.020+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.133:6838/982201245 2024-04-22T07:17:43.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:43.020+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=101 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-22T07:17:43.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:43.020+0000 7f22dd7fa640 1 client.16302 ms_handle_refused on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:44.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:44 smithi133 ceph-mon[31198]: pgmap v92: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 7 op/s 2024-04-22T07:17:44.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:44 smithi154 ceph-mon[31862]: pgmap v92: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 7 op/s 2024-04-22T07:17:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:44 smithi113 ceph-mon[27933]: pgmap v92: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 7 op/s 2024-04-22T07:17:46.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:46 smithi133 ceph-mon[31198]: pgmap v93: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2024-04-22T07:17:46.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:46 smithi154 ceph-mon[31862]: pgmap v93: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2024-04-22T07:17:46.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:46 smithi113 ceph-mon[27933]: pgmap v93: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2024-04-22T07:17:48.135 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:47 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service: Scheduled restart job, restart counter is at 1. 2024-04-22T07:17:48.135 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:47 smithi133 systemd[1]: Stopped Ceph mds.b for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:17:48.135 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:47 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service: Consumed 9.475s CPU time. 2024-04-22T07:17:48.135 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:47 smithi133 systemd[1]: Starting Ceph mds.b for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:17:48.135 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 podman[65461]: 2024-04-22 07:17:48.03253078 +0000 UTC m=+0.093147975 container create a72bad9e888a3595683237956832f1d16e7776e5be7f45ad8fda8cf8759f0b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-22T07:17:48.136 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 podman[65461]: 2024-04-22 07:17:47.957548539 +0000 UTC m=+0.018165701 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:17:48.524 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 podman[65461]: 2024-04-22 07:17:48.277038868 +0000 UTC m=+0.337656018 container init a72bad9e888a3595683237956832f1d16e7776e5be7f45ad8fda8cf8759f0b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:17:48.524 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 podman[65461]: 2024-04-22 07:17:48.280121726 +0000 UTC m=+0.340738875 container start a72bad9e888a3595683237956832f1d16e7776e5be7f45ad8fda8cf8759f0b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-22T07:17:48.525 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mds[65479]: -- 172.21.15.133:0/969940903 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55782a53ba40 con 0x55782a50f000 2024-04-22T07:17:48.525 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[65475]: starting mds.b at 2024-04-22T07:17:48.525 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 bash[65461]: a72bad9e888a3595683237956832f1d16e7776e5be7f45ad8fda8cf8759f0b39 2024-04-22T07:17:48.525 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:17:48 smithi133 systemd[1]: Started Ceph mds.b for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: pgmap v94: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: daemon mds.b restarted 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6838/22002757,v1:172.21.15.133:6839/22002757] up:boot 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: Standby daemon mds.e assigned to filesystem cephfs as rank 0 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: fsmap cephfs:2/3 {1=c=up:active,2=f=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:17:48.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:48 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=e=up:replay,1=c=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:17:48.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: pgmap v94: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-22T07:17:48.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: daemon mds.b restarted 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6838/22002757,v1:172.21.15.133:6839/22002757] up:boot 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: Standby daemon mds.e assigned to filesystem cephfs as rank 0 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: fsmap cephfs:2/3 {1=c=up:active,2=f=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:17:48.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:48 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=e=up:replay,1=c=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:17:48.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: pgmap v94: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: daemon mds.b restarted 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6838/22002757,v1:172.21.15.133:6839/22002757] up:boot 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: Standby daemon mds.e assigned to filesystem cephfs as rank 0 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: fsmap cephfs:2/3 {1=c=up:active,2=f=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-22T07:17:48.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:17:48.836 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:48 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=e=up:replay,1=c=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:17:49.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=101 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:17:49.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_RECONNECTING pgs=101 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-04-22T07:17:49.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_RECONNECTING pgs=101 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-04-22T07:17:49.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22dd7fa640 0 client.16302 ms_handle_remote_reset on v2:172.21.15.133:6838/982201245 2024-04-22T07:17:49.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:17:49.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:17:49.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:17:49.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:17:49.427+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: pgmap v96: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:resolve 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: Replacing daemon mds.f as rank 2 with standby daemon mds.i 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: Replacing daemon mds.c as rank 1 with standby daemon mds.h 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=e=up:resolve,1=c=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:17:50.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T07:17:50.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:50 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=e=up:resolve,1=h=up:replay,2=i=up:replay} 7 up:standby 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: pgmap v96: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:resolve 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: Replacing daemon mds.f as rank 2 with standby daemon mds.i 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: Replacing daemon mds.c as rank 1 with standby daemon mds.h 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=e=up:resolve,1=c=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T07:17:50.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:50 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=e=up:resolve,1=h=up:replay,2=i=up:replay} 7 up:standby 2024-04-22T07:17:50.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: pgmap v96: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-04-22T07:17:50.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:resolve 2024-04-22T07:17:50.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: Replacing daemon mds.f as rank 2 with standby daemon mds.i 2024-04-22T07:17:50.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-04-22T07:17:50.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: Replacing daemon mds.c as rank 1 with standby daemon mds.h 2024-04-22T07:17:50.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-22T07:17:50.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=e=up:resolve,1=c=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T07:17:50.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T07:17:50.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:50 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=e=up:resolve,1=h=up:replay,2=i=up:replay} 7 up:standby 2024-04-22T07:17:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:51 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:resolve 2024-04-22T07:17:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:51 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:resolve 2024-04-22T07:17:51.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:51 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=e=up:resolve,1=h=up:resolve,2=i=up:resolve} 7 up:standby 2024-04-22T07:17:51.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:51 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:resolve 2024-04-22T07:17:51.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:51 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:resolve 2024-04-22T07:17:51.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:51 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=e=up:resolve,1=h=up:resolve,2=i=up:resolve} 7 up:standby 2024-04-22T07:17:51.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:51 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:resolve 2024-04-22T07:17:51.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:51 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:resolve 2024-04-22T07:17:51.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:51 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=e=up:resolve,1=h=up:resolve,2=i=up:resolve} 7 up:standby 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: Stopped Ceph mds.c for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service: Consumed 48.583s CPU time. 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: Starting Ceph mds.c for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: Stopped Ceph mds.f for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service: Consumed 4.003s CPU time. 2024-04-22T07:17:52.075 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:51 smithi154 systemd[1]: Starting Ceph mds.f for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:17:52.422 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67280]: 2024-04-22 07:17:51.983273758 +0000 UTC m=+0.021559371 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:17:52.423 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67280]: 2024-04-22 07:17:52.312618995 +0000 UTC m=+0.350904595 container create 4ff76a22d8d009aa5380be0bfe879ecd7412dbc780d39cc97fdf15ce7253d81c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T07:17:52.423 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67279]: 2024-04-22 07:17:51.981480177 +0000 UTC m=+0.021150788 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:17:52.423 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67279]: 2024-04-22 07:17:52.118382949 +0000 UTC m=+0.158053550 container create 666b931f150d6cf3a83609fd44634b4a7c9ba1187df6c689a1d07527a18dd2b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:17:52.755 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: pgmap v98: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 op/s 2024-04-22T07:17:52.755 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: reconnect by client.15579 172.21.15.113:0/567144824 after 0 2024-04-22T07:17:52.755 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: reconnect by client.15579 172.21.15.113:0/567144824 after 0 2024-04-22T07:17:52.755 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999981 2024-04-22T07:17:52.755 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999974 2024-04-22T07:17:52.755 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999981 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:reconnect 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:reconnect 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:reconnect 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=e=up:reconnect,1=h=up:reconnect,2=i=up:reconnect} 7 up:standby 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:rejoin 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67279]: 2024-04-22 07:17:52.515030563 +0000 UTC m=+0.554701165 container init 666b931f150d6cf3a83609fd44634b4a7c9ba1187df6c689a1d07527a18dd2b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:17:52.756 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67279]: 2024-04-22 07:17:52.517684021 +0000 UTC m=+0.557354624 container start 666b931f150d6cf3a83609fd44634b4a7c9ba1187df6c689a1d07527a18dd2b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-22T07:17:52.757 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mds[67316]: -- 172.21.15.154:0/4059057672 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd405dba40 con 0x55bd405af000 2024-04-22T07:17:52.757 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[67307]: starting mds.f at 2024-04-22T07:17:52.757 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 bash[67279]: 666b931f150d6cf3a83609fd44634b4a7c9ba1187df6c689a1d07527a18dd2b3 2024-04-22T07:17:52.757 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:17:52 smithi154 systemd[1]: Started Ceph mds.f for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: pgmap v98: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 op/s 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: reconnect by client.15579 172.21.15.113:0/567144824 after 0 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: reconnect by client.15579 172.21.15.113:0/567144824 after 0 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999981 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999974 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999981 2024-04-22T07:17:52.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:reconnect 2024-04-22T07:17:52.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:reconnect 2024-04-22T07:17:52.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:reconnect 2024-04-22T07:17:52.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=e=up:reconnect,1=h=up:reconnect,2=i=up:reconnect} 7 up:standby 2024-04-22T07:17:52.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:52 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:rejoin 2024-04-22T07:17:52.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: pgmap v98: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 10 op/s 2024-04-22T07:17:52.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: reconnect by client.15579 172.21.15.113:0/567144824 after 0 2024-04-22T07:17:52.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: reconnect by client.15579 172.21.15.113:0/567144824 after 0 2024-04-22T07:17:52.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999981 2024-04-22T07:17:52.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999974 2024-04-22T07:17:52.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: reconnect by client.24917 172.21.15.113:0/3249358072 after 0.000999981 2024-04-22T07:17:52.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:reconnect 2024-04-22T07:17:52.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:reconnect 2024-04-22T07:17:52.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:reconnect 2024-04-22T07:17:52.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=e=up:reconnect,1=h=up:reconnect,2=i=up:reconnect} 7 up:standby 2024-04-22T07:17:52.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:52 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:rejoin 2024-04-22T07:17:53.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67280]: 2024-04-22 07:17:52.757598624 +0000 UTC m=+0.795884224 container init 4ff76a22d8d009aa5380be0bfe879ecd7412dbc780d39cc97fdf15ce7253d81c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-22T07:17:53.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 podman[67280]: 2024-04-22 07:17:52.760266901 +0000 UTC m=+0.798552504 container start 4ff76a22d8d009aa5380be0bfe879ecd7412dbc780d39cc97fdf15ce7253d81c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:17:53.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-mds[67336]: -- 172.21.15.154:0/3400250207 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2e9a1fa40 con 0x55a2e99f2c00 2024-04-22T07:17:53.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[67312]: starting mds.c at 2024-04-22T07:17:53.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 bash[67280]: 4ff76a22d8d009aa5380be0bfe879ecd7412dbc780d39cc97fdf15ce7253d81c 2024-04-22T07:17:53.066 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:17:52 smithi154 systemd[1]: Started Ceph mds.c for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:rejoin 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:rejoin 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: fsmap cephfs:3/3 {0=e=up:rejoin,1=h=up:rejoin,2=i=up:rejoin} 7 up:standby 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: daemon mds.e is now active in filesystem cephfs as rank 0 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:17:53.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: Cluster is now healthy 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/2460958291,v1:172.21.15.154:6835/2460958291] up:boot 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:active 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:active 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:active 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/2537342418,v1:172.21.15.154:6837/2537342418] up:boot 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:17:53.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:53 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:rejoin 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:rejoin 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: fsmap cephfs:3/3 {0=e=up:rejoin,1=h=up:rejoin,2=i=up:rejoin} 7 up:standby 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: daemon mds.e is now active in filesystem cephfs as rank 0 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: Cluster is now healthy 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/2460958291,v1:172.21.15.154:6835/2460958291] up:boot 2024-04-22T07:17:53.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:active 2024-04-22T07:17:53.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:active 2024-04-22T07:17:53.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:active 2024-04-22T07:17:53.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/2537342418,v1:172.21.15.154:6837/2537342418] up:boot 2024-04-22T07:17:53.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:17:53.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:17:53.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:53 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:rejoin 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:rejoin 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: fsmap cephfs:3/3 {0=e=up:rejoin,1=h=up:rejoin,2=i=up:rejoin} 7 up:standby 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: daemon mds.e is now active in filesystem cephfs as rank 0 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: Cluster is now healthy 2024-04-22T07:17:53.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/2460958291,v1:172.21.15.154:6835/2460958291] up:boot 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] up:active 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] up:active 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] up:active 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/2537342418,v1:172.21.15.154:6837/2537342418] up:boot 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:17:53.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:53 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:17:54.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:54 smithi133 ceph-mon[31198]: pgmap v99: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5 op/s 2024-04-22T07:17:54.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:54 smithi154 ceph-mon[31862]: pgmap v99: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5 op/s 2024-04-22T07:17:54.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:54 smithi113 ceph-mon[27933]: pgmap v99: 97 pgs: 97 active+clean; 187 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5 op/s 2024-04-22T07:17:55.814 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:17:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: 2024-04-22T07:17:55.383+0000 7f50082bc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T07:17:55.814 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:17:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: 2024-04-22T07:17:55.383+0000 7f50082bc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-22T07:17:55.814 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:17:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: 2024-04-22T07:17:55.383+0000 7f50082bc700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-22T07:17:56.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:56 smithi133 ceph-mon[31198]: pgmap v100: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 537 KiB/s wr, 34 op/s 2024-04-22T07:17:56.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:56 smithi154 ceph-mon[31862]: pgmap v100: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 537 KiB/s wr, 34 op/s 2024-04-22T07:17:56.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:56 smithi113 ceph-mon[27933]: pgmap v100: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 537 KiB/s wr, 34 op/s 2024-04-22T07:17:57.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:57 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:17:57.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:57 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:17:57.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:57 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:17:58.006 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-04-22T07:17:58.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:58 smithi133 ceph-mon[31198]: pgmap v101: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 472 KiB/s wr, 30 op/s 2024-04-22T07:17:58.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:17:58.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:17:58.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:17:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:17:58.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:58 smithi154 ceph-mon[31862]: pgmap v101: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 472 KiB/s wr, 30 op/s 2024-04-22T07:17:58.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:17:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:17:58.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:17:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:17:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:58 smithi113 ceph-mon[27933]: pgmap v101: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 472 KiB/s wr, 30 op/s 2024-04-22T07:17:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:17:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:17:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:17:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:17:58.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ tar c linux-4.0.5 2024-04-22T07:17:59.428 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm -rf linux-4.0.5 2024-04-22T07:18:00.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:00 smithi133 ceph-mon[31198]: pgmap v102: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 430 KiB/s wr, 27 op/s 2024-04-22T07:18:00.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:00 smithi154 ceph-mon[31862]: pgmap v102: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 430 KiB/s wr, 27 op/s 2024-04-22T07:18:00.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:00 smithi113 ceph-mon[27933]: pgmap v102: 97 pgs: 97 active+clean; 191 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 430 KiB/s wr, 27 op/s 2024-04-22T07:18:01.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:01 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:02.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:01 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:02.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:01 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:02.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:02.229+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:02.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:02.229+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:18:02.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:02.229+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:18:02.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:02.229+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:18:02.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:02.229+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:18:02.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:02 smithi113 ceph-mon[27933]: pgmap v103: 97 pgs: 97 active+clean; 208 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-22T07:18:03.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:02 smithi133 ceph-mon[31198]: pgmap v103: 97 pgs: 97 active+clean; 208 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-22T07:18:03.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:02 smithi154 ceph-mon[31862]: pgmap v103: 97 pgs: 97 active+clean; 208 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-22T07:18:04.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:04 smithi113 ceph-mon[27933]: pgmap v104: 97 pgs: 97 active+clean; 208 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-22T07:18:05.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:04 smithi133 ceph-mon[31198]: pgmap v104: 97 pgs: 97 active+clean; 208 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-22T07:18:05.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:04 smithi154 ceph-mon[31862]: pgmap v104: 97 pgs: 97 active+clean; 208 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-22T07:18:06.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:18:06.194 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:18:06.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 -- 172.21.15.113:0/3381054355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174101080 msgr2=0x7f6174101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 --2- 172.21.15.113:0/3381054355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174101080 0x7f6174101460 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6164009900 tx=0x7f616402bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 -- 172.21.15.113:0/3381054355 shutdown_connections 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 --2- 172.21.15.113:0/3381054355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6174106150 0x7f617410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 --2- 172.21.15.113:0/3381054355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6174101a30 0x7f6174105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 --2- 172.21.15.113:0/3381054355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174101080 0x7f6174101460 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 -- 172.21.15.113:0/3381054355 >> 172.21.15.113:0/3381054355 conn(0x7f6174078000 msgr2=0x7f6174078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 -- 172.21.15.113:0/3381054355 shutdown_connections 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 -- 172.21.15.113:0/3381054355 wait complete. 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 Processor -- start 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.302+0000 7f617997e640 1 -- start start 2024-04-22T07:18:06.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6174101080 0x7f6174197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 0x7f6174197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6174197ec0 0x7f61741a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61741111d0 con 0x7f6174106150 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6174111050 con 0x7f6174101080 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6174111350 con 0x7f6174197ec0 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 0x7f6174197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 0x7f6174197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44740/0 (socket says 172.21.15.113:44740) 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 -- 172.21.15.113:0/1626685823 learned_addr learned my addr 172.21.15.113:0/1626685823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f6172ffd640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6174101080 0x7f6174197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61737fe640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6174197ec0 0x7f61741a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 -- 172.21.15.113:0/1626685823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6174197ec0 msgr2=0x7f61741a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6174197ec0 0x7f61741a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 -- 172.21.15.113:0/1626685823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6174101080 msgr2=0x7f6174197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6174101080 0x7f6174197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 -- 172.21.15.113:0/1626685823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61741a7080 con 0x7f6174106150 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f61727fc640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 0x7f6174197980 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f616800e9f0 tx=0x7f616800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617897c640 1 -- 172.21.15.113:0/1626685823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616800cd20 con 0x7f6174106150 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61741a7370 con 0x7f6174106150 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617897c640 1 -- 172.21.15.113:0/1626685823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61680054f0 con 0x7f6174106150 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617897c640 1 -- 172.21.15.113:0/1626685823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168010690 con 0x7f6174106150 2024-04-22T07:18:06.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.303+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741a78c0 con 0x7f6174106150 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617897c640 1 -- 172.21.15.113:0/1626685823 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f61680046c0 con 0x7f6174106150 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 Processor -- start 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 -- start start 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e7f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e88a80 0x561778e8ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561778de41e0 con 0x561778e7f580 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561778d81390 con 0x561778e88a80 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617c569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561778de01a0 con 0x561778e7fea0 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e7f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e7f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44754/0 (socket says 172.21.15.113:44754) 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 -- 172.21.15.113:0/3112740000 learned_addr learned my addr 172.21.15.113:0/3112740000 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61727fc640 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e88a80 0x561778e8ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f617897c640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f615c0a3d20 0x7f615c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f6172ffd640 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 -- 172.21.15.113:0/3112740000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 msgr2=0x561778e88540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e88540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 -- 172.21.15.113:0/3112740000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e88a80 msgr2=0x561778e8ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e88a80 0x561778e8ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f61737fe640 1 -- 172.21.15.113:0/3112740000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561778bc07d0 con 0x561778e7f580 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.306+0000 7f6172ffd640 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.307+0000 7f617897c640 1 -- 172.21.15.113:0/1626685823 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f61680d0c90 con 0x7f6174106150 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.307+0000 7f61737fe640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f615c0a3d20 0x7f615c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.307+0000 7f61737fe640 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e7f960 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f616c011590 tx=0x7f616c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf778fc2c75509a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.307+0000 7f6159ffb640 1 -- 172.21.15.113:0/3112740000 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c017050 con 0x561778e7f580 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.307+0000 7f6159ffb640 1 -- 172.21.15.113:0/3112740000 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f616c01bb90 con 0x561778e7f580 2024-04-22T07:18:06.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.307+0000 7f6159ffb640 1 -- 172.21.15.113:0/3112740000 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c00eda0 con 0x561778e7f580 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- 172.21.15.113:0/3112740000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 msgr2=0x561778e7f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e7f960 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f616c011590 tx=0x7f616c011a50 comp rx=0 tx=0).stop 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f61737fe640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f615c0a3d20 0x7f615c0a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f616c022000 tx=0x7f616c023000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- 172.21.15.113:0/3112740000 shutdown_connections 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e88a80 0x561778e8ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e88540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- 172.21.15.113:0/3112740000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e7f960 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- 172.21.15.113:0/3112740000 >> 172.21.15.113:0/3112740000 conn(0x561778e79cb0 msgr2=0x561778e7a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- 172.21.15.113:0/3112740000 shutdown_connections 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- 172.21.15.113:0/3112740000 wait complete. 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- start start 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 Processor -- start 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e85b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7fea0 0x561778e86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561778e8b870 con 0x561778e7f580 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561778d9d2f0 con 0x561778e7fea0 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561778e8b9f0 con 0x561778e88a80 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.308+0000 7f617c569740 10 client.0 _ll_register_callbacks cb 0x561778daa580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6172ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e85b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7fea0 0x561778e86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6172ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e85b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44760/0 (socket says 172.21.15.113:44760) 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6172ffd640 1 -- 172.21.15.113:0/687474959 learned_addr learned my addr 172.21.15.113:0/687474959 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61737fe640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 -- 172.21.15.113:0/687474959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 msgr2=0x561778e865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e865c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 -- 172.21.15.113:0/687474959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 msgr2=0x561778e85b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e85b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 -- 172.21.15.113:0/687474959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561778af63f0 con 0x561778e7fea0 2024-04-22T07:18:06.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61737fe640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6172ffd640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e85b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f61727fc640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7fea0 0x561778e86080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61680ce260 tx=0x7f6168000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709efa576810df8f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6152ffd640 1 -- 172.21.15.113:0/687474959 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616809c5f0 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f616c01bb90 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6152ffd640 1 -- 172.21.15.113:0/687474959 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f6168098800 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f6152ffd640 1 -- 172.21.15.113:0/687474959 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f616809c410 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f617c569740 10 client.25736 fetch_fsmap learned FSMap version 44 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.309+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561778ab60f0 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f6152ffd640 1 -- 172.21.15.113:0/687474959 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f616c01bb90 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f6152ffd640 1 -- 172.21.15.113:0/687474959 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f61680d5040 con 0x561778e7fea0 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f617c569740 10 client.25736 fetch_fsmap finished waiting for FSMap version 44 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f617c569740 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:06.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f617c569740 20 client.25736 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f617c569740 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778e98a70 0x561778e9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f617c569740 4 client.25736 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561778df1c00 con 0x561778e98a70 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f6172ffd640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778e98a70 0x561778e9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f6152ffd640 10 client.25736 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.310+0000 7f6172ffd640 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778e98a70 0x561778e9ae80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca1114a7268e086 server_cookie=7978f124805d26e5 in_seq=0 out_seq=0 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.311+0000 7f6152ffd640 1 -- 172.21.15.113:0/687474959 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561778df1c00 con 0x561778e98a70 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.311+0000 7f6152ffd640 10 client.25736 handle_command_reply: tid=0 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.311+0000 7f617c569740 1 client.25736 shutdown 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.311+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7fea0 msgr2=0x561778e86080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.311+0000 7f617c569740 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7fea0 0x561778e86080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61680ce260 tx=0x7f6168000bc0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 shutdown_connections 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778e98a70 0x561778e9ae80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7fea0 0x561778e86080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f6152ffd640 0 client.25736 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 --2- 172.21.15.113:0/687474959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7f580 0x561778e85b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 >> 172.21.15.113:0/687474959 conn(0x561778e79cb0 msgr2=0x561778e7ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 shutdown_connections 2024-04-22T07:18:06.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 1 -- 172.21.15.113:0/687474959 wait complete. 2024-04-22T07:18:06.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.312+0000 7f617c569740 20 client.25736 trim_cache size 0 max 16384 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 Processor -- start 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 -- start start 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e88270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 0x561778e9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e94730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b2ab0 con 0x561778e7fea0 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b0380 con 0x561778e7f580 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f617c569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61741ae1d0 con 0x561778e88a80 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 0x561778e9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 0x561778e9ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44762/0 (socket says 172.21.15.113:44762) 2024-04-22T07:18:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 -- 172.21.15.113:0/253801745 learned_addr learned my addr 172.21.15.113:0/253801745 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f6172ffd640 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e88270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61737fe640 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e94730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 -- 172.21.15.113:0/253801745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 msgr2=0x561778e94730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e94730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 -- 172.21.15.113:0/253801745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 msgr2=0x561778e88270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e88270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.329+0000 7f61727fc640 1 -- 172.21.15.113:0/253801745 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561778e7a090 con 0x561778e7fea0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f61727fc640 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 0x561778e9ae60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6168099480 tx=0x7f616809afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac290901437adb12 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f61597fa640 1 -- 172.21.15.113:0/253801745 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616809c5f0 con 0x561778e7fea0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f61597fa640 1 -- 172.21.15.113:0/253801745 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61680b6020 con 0x561778e7fea0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f61597fa640 1 -- 172.21.15.113:0/253801745 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61680a0e60 con 0x561778e7fea0 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 -- 172.21.15.113:0/253801745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 msgr2=0x561778e9ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 0x561778e9ae60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6168099480 tx=0x7f616809afc0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 -- 172.21.15.113:0/253801745 shutdown_connections 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e88a80 0x561778e94730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e7fea0 0x561778e9ae60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 --2- 172.21.15.113:0/253801745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e88270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 -- 172.21.15.113:0/253801745 >> 172.21.15.113:0/253801745 conn(0x561778e79cb0 msgr2=0x561778e7b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 -- 172.21.15.113:0/253801745 shutdown_connections 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.330+0000 7f617c569740 1 -- 172.21.15.113:0/253801745 wait complete. 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 -- start start 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 Processor -- start 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e90f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e8c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 0x561778e8c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61741ae840 con 0x561778e88a80 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61741afcb0 con 0x561778e7f580 2024-04-22T07:18:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61741aef10 con 0x561778e7fea0 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f617c569740 10 client.0 _ll_register_callbacks cb 0x7f616800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f61737fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 0x561778e8c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f61737fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 0x561778e8c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44772/0 (socket says 172.21.15.113:44772) 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f61727fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e8c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f6172ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e90f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.331+0000 7f61737fe640 1 -- 172.21.15.113:0/3325982615 learned_addr learned my addr 172.21.15.113:0/3325982615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61737fe640 1 -- 172.21.15.113:0/3325982615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 msgr2=0x561778e8c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61737fe640 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e8c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61737fe640 1 -- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 msgr2=0x561778e90f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61737fe640 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e90f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61737fe640 1 -- 172.21.15.113:0/3325982615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561778e8cf20 con 0x561778e88a80 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61727fc640 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e8c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f61737fe640 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 0x561778e8c7e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f616c011560 tx=0x7f616c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d86f66479511f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f6152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f6152ffd640 1 -- 172.21.15.113:0/3325982615 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0094c0 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f6152ffd640 1 -- 172.21.15.113:0/3325982615 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f616c01b990 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61680b6020 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f6152ffd640 1 -- 172.21.15.113:0/3325982615 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f616c017020 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 10 client.16323 fetch_fsmap learned FSMap version 44 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561778efef00 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f6152ffd640 1 -- 172.21.15.113:0/3325982615 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61680b6020 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f6152ffd640 1 -- 172.21.15.113:0/3325982615 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f616c00e880 con 0x561778e88a80 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 10 client.16323 fetch_fsmap finished waiting for FSMap version 44 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 20 client.16323 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778eff3f0 0x561778f01810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 4 client.16323 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:18:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.332+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561778deba70 con 0x561778eff3f0 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.333+0000 7f6172ffd640 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778eff3f0 0x561778f01810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.333+0000 7f6152ffd640 10 client.16323 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.333+0000 7f6172ffd640 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778eff3f0 0x561778f01810 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612e4b0847c8743d server_cookie=5834a1db6e0b7d06 in_seq=0 out_seq=0 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f6152ffd640 1 -- 172.21.15.113:0/3325982615 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561778deba70 con 0x561778eff3f0 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f6152ffd640 10 client.16323 handle_command_reply: tid=0 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 client.16323 shutdown 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 msgr2=0x561778e8c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 0x561778e8c7e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f616c011560 tx=0x7f616c004ed0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 shutdown_connections 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561778eff3f0 0x561778f01810 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561778e88a80 0x561778e8c7e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561778e7fea0 0x561778e8c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 --2- 172.21.15.113:0/3325982615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561778e7f580 0x561778e90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f6152ffd640 0 client.16323 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 >> 172.21.15.113:0/3325982615 conn(0x561778e79cb0 msgr2=0x561778e78d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 shutdown_connections 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 1 -- 172.21.15.113:0/3325982615 wait complete. 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617c569740 20 client.16323 trim_cache size 0 max 16384 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:18:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:06.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f615c0a3d20 msgr2=0x7f615c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f615c0a3d20 0x7f615c0a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f616c022000 tx=0x7f616c023000 comp rx=0 tx=0).stop 2024-04-22T07:18:06.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 msgr2=0x7f6174197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 0x7f6174197980 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f616800e9f0 tx=0x7f616800eeb0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 shutdown_connections 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f615c0a3d20 0x7f615c0a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6174197ec0 0x7f61741a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6174106150 0x7f6174197980 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 --2- 172.21.15.113:0/1626685823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6174101080 0x7f6174197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.334+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 >> 172.21.15.113:0/1626685823 conn(0x7f6174078000 msgr2=0x7f61740fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.335+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 shutdown_connections 2024-04-22T07:18:06.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.335+0000 7f617997e640 1 -- 172.21.15.113:0/1626685823 wait complete. 2024-04-22T07:18:06.343 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:18:06.343 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:18:06.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.451+0000 7f9c971a4640 1 -- 172.21.15.113:0/125410786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90106f50 msgr2=0x7f9c90107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.451+0000 7f9c971a4640 1 --2- 172.21.15.113:0/125410786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90106f50 0x7f9c90107330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009900 tx=0x7f9c8402bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 -- 172.21.15.113:0/125410786 shutdown_connections 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 --2- 172.21.15.113:0/125410786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c90069aa0 0x7f9c9010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 --2- 172.21.15.113:0/125410786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c900690e0 0x7f9c90069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 --2- 172.21.15.113:0/125410786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90106f50 0x7f9c90107330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 -- 172.21.15.113:0/125410786 >> 172.21.15.113:0/125410786 conn(0x7f9c90077e30 msgr2=0x7f9c90078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 -- 172.21.15.113:0/125410786 shutdown_connections 2024-04-22T07:18:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 -- 172.21.15.113:0/125410786 wait complete. 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 Processor -- start 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.452+0000 7f9c971a4640 1 -- start start 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c971a4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c900690e0 0x7f9c9019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c971a4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 0x7f9c9019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c971a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c90106f50 0x7f9c901985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c971a4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901122d0 con 0x7f9c90069aa0 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c971a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90112150 con 0x7f9c90106f50 2024-04-22T07:18:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c971a4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90112450 con 0x7f9c900690e0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 0x7f9c9019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 0x7f9c9019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44802/0 (socket says 172.21.15.113:44802) 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 -- 172.21.15.113:0/1779470297 learned_addr learned my addr 172.21.15.113:0/1779470297 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c94f40640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c900690e0 0x7f9c9019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c95741640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c90106f50 0x7f9c901985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 -- 172.21.15.113:0/1779470297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c900690e0 msgr2=0x7f9c9019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c900690e0 0x7f9c9019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 -- 172.21.15.113:0/1779470297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c90106f50 msgr2=0x7f9c901985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c90106f50 0x7f9c901985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.453+0000 7f9c8ffff640 1 -- 172.21.15.113:0/1779470297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c90198d30 con 0x7f9c90069aa0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.454+0000 7f9c8ffff640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 0x7f9c9019e940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d770 tx=0x7f9c7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.454+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c017070 con 0x7f9c90069aa0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.454+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9c7c0054f0 con 0x7f9c90069aa0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.454+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c004de0 con 0x7f9c90069aa0 2024-04-22T07:18:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.454+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c90199020 con 0x7f9c90069aa0 2024-04-22T07:18:06.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.454+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c901a52c0 con 0x7f9c90069aa0 2024-04-22T07:18:06.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.455+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c90068de0 con 0x7f9c90069aa0 2024-04-22T07:18:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.457+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9c7c004f80 con 0x7f9c90069aa0 2024-04-22T07:18:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.458+0000 7f9c8dffb640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9c800a3e20 0x7f9c800a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.458+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f9c7c0d03b0 con 0x7f9c90069aa0 2024-04-22T07:18:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.459+0000 7f9c94f40640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9c800a3e20 0x7f9c800a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.459+0000 7f9c94f40640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9c800a3e20 0x7f9c800a62e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c8402efd0 tx=0x7f9c84005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.466+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c7c09c220 con 0x7f9c90069aa0 2024-04-22T07:18:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.722+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c9010a480 con 0x7f9c90069aa0 2024-04-22T07:18:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.723+0000 7f9c8dffb640 1 -- 172.21.15.113:0/1779470297 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11889 (secure 0 0 0) 0x7f9c7c0a0520 con 0x7f9c90069aa0 2024-04-22T07:18:06.720 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:18:06.720 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:18:06.121576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:18:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 45 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9c800a3e20 msgr2=0x7f9c800a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9c800a3e20 0x7f9c800a62e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c8402efd0 tx=0x7f9c84005fd0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 msgr2=0x7f9c9019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 0x7f9c9019e940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d770 tx=0x7f9c7c00dc30 comp rx=0 tx=0).stop 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 shutdown_connections 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9c800a3e20 0x7f9c800a62e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c90106f50 0x7f9c901985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c90069aa0 0x7f9c9019e940 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 --2- 172.21.15.113:0/1779470297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9c900690e0 0x7f9c9019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 >> 172.21.15.113:0/1779470297 conn(0x7f9c90077e30 msgr2=0x7f9c900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 shutdown_connections 2024-04-22T07:18:06.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.725+0000 7f9c971a4640 1 -- 172.21.15.113:0/1779470297 wait complete. 2024-04-22T07:18:06.733 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:18:06.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:06 smithi113 ceph-mon[27933]: pgmap v105: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-04-22T07:18:06.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:06 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:06.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 -- 172.21.15.113:0/3168954071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334106160 msgr2=0x7f933410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 --2- 172.21.15.113:0/3168954071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334106160 0x7f933410dd30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f932c009ca0 tx=0x7f932c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 -- 172.21.15.113:0/3168954071 shutdown_connections 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 --2- 172.21.15.113:0/3168954071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9334106160 0x7f933410dd30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 --2- 172.21.15.113:0/3168954071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9334101a40 0x7f9334105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 --2- 172.21.15.113:0/3168954071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9334101090 0x7f9334101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 -- 172.21.15.113:0/3168954071 >> 172.21.15.113:0/3168954071 conn(0x7f9334077f50 msgr2=0x7f9334078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 -- 172.21.15.113:0/3168954071 shutdown_connections 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 -- 172.21.15.113:0/3168954071 wait complete. 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 Processor -- start 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.839+0000 7f9339385640 1 -- start start 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9334101090 0x7f9334199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9334106160 0x7f9334199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 0x7f93341a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93341111e0 con 0x7f933419a0d0 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9334111060 con 0x7f9334106160 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9334111360 con 0x7f9334101090 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 0x7f93341a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9332ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9334101090 0x7f9334199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f932bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9334106160 0x7f9334199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 0x7f93341a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44816/0 (socket says 172.21.15.113:44816) 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 -- 172.21.15.113:0/3280407248 learned_addr learned my addr 172.21.15.113:0/3280407248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 -- 172.21.15.113:0/3280407248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9334101090 msgr2=0x7f9334199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9334101090 0x7f9334199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 -- 172.21.15.113:0/3280407248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9334106160 msgr2=0x7f9334199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9334106160 0x7f9334199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 -- 172.21.15.113:0/3280407248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93341a4e50 con 0x7f933419a0d0 2024-04-22T07:18:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f93337fe640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 0x7f93341a4680 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f932c02c700 tx=0x7f932c035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9330ff9640 1 -- 172.21.15.113:0/3280407248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c039040 con 0x7f933419a0d0 2024-04-22T07:18:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93341a50e0 con 0x7f933419a0d0 2024-04-22T07:18:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9330ff9640 1 -- 172.21.15.113:0/3280407248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f932c044070 con 0x7f933419a0d0 2024-04-22T07:18:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9330ff9640 1 -- 172.21.15.113:0/3280407248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c004330 con 0x7f933419a0d0 2024-04-22T07:18:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.840+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93341a55d0 con 0x7f933419a0d0 2024-04-22T07:18:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.842+0000 7f9330ff9640 1 -- 172.21.15.113:0/3280407248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f932c0044d0 con 0x7f933419a0d0 2024-04-22T07:18:06.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 Processor -- start 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 -- start start 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f9330ff9640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f93200a3d20 0x7f93200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f932bfff640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f93200a3d20 0x7f93200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f9330ff9640 1 -- 172.21.15.113:0/3280407248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f932c0f2d80 con 0x7f933419a0d0 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9213d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92146430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92148e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562c920a21f0 con 0x562c9213de80 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c9203f4b0 con 0x562c9213d4d0 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f933bf71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562c9209e1b0 con 0x562c92146a00 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44830/0 (socket says 172.21.15.113:44830) 2024-04-22T07:18:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f932bfff640 1 -- 172.21.15.113:0/1613794321 learned_addr learned my addr 172.21.15.113:0/1613794321 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.843+0000 7f9332ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9213d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f93337fe640 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f932bfff640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f93200a3d20 0x7f93200a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f931c013460 tx=0x7f931c013860 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f9332ffd640 1 -- 172.21.15.113:0/1613794321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 msgr2=0x562c92148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f9332ffd640 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f9332ffd640 1 -- 172.21.15.113:0/1613794321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 msgr2=0x562c92146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f9332ffd640 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f9332ffd640 1 -- 172.21.15.113:0/1613794321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c920311b0 con 0x562c9213d4d0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f932bfff640 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f9332ffd640 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9213d8b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f932400bbf0 tx=0x7f93240107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8e92fa006fde04 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f93297fa640 1 -- 172.21.15.113:0/1613794321 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324004030 con 0x562c9213d4d0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f93297fa640 1 -- 172.21.15.113:0/1613794321 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f93240041d0 con 0x562c9213d4d0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.844+0000 7f93297fa640 1 -- 172.21.15.113:0/1613794321 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93240116a0 con 0x562c9213d4d0 2024-04-22T07:18:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 -- 172.21.15.113:0/1613794321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 msgr2=0x562c9213d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9213d8b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f932400bbf0 tx=0x7f93240107a0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 -- 172.21.15.113:0/1613794321 shutdown_connections 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 --2- 172.21.15.113:0/1613794321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9213d8b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 -- 172.21.15.113:0/1613794321 >> 172.21.15.113:0/1613794321 conn(0x562c92137ba0 msgr2=0x562c92137f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 -- 172.21.15.113:0/1613794321 shutdown_connections 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 -- 172.21.15.113:0/1613794321 wait complete. 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 -- start start 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.845+0000 7f933bf71740 1 Processor -- start 2024-04-22T07:18:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c921463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92143780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562c92149970 con 0x562c9213de80 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c9205b500 con 0x562c92146a00 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562c92149af0 con 0x562c9213d4d0 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 10 client.0 _ll_register_callbacks cb 0x562c92029b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44842/0 (socket says 172.21.15.113:44842) 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 -- 172.21.15.113:0/3455094839 learned_addr learned my addr 172.21.15.113:0/3455094839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f93337fe640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f9332ffd640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c921463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 -- 172.21.15.113:0/3455094839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 msgr2=0x562c921463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c921463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 -- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 msgr2=0x562c92143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 -- 172.21.15.113:0/3455094839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c91db43f0 con 0x562c9213de80 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f932bfff640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92143780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f931c003850 tx=0x7f931c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9f8be5f6b5d181 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f931a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f931a7fc640 1 -- 172.21.15.113:0/3455094839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c012270 con 0x562c9213de80 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f931a7fc640 1 -- 172.21.15.113:0/3455094839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f931c00fa70 con 0x562c9213de80 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93240041d0 con 0x562c9213de80 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f931a7fc640 1 -- 172.21.15.113:0/3455094839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f931c00fc70 con 0x562c9213de80 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 10 client.16341 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.846+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562c91e7e7d0 con 0x562c9213de80 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f931a7fc640 1 -- 172.21.15.113:0/3455094839 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f93240041d0 con 0x562c9213de80 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f931a7fc640 1 -- 172.21.15.113:0/3455094839 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f931c01f070 con 0x562c9213de80 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f933bf71740 10 client.16341 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f933bf71740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f933bf71740 20 client.16341 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f933bf71740 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c92156d00 0x562c921590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f933bf71740 4 client.16341 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c92041930 con 0x562c92156d00 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f9332ffd640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c92156d00 0x562c921590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f931a7fc640 10 client.16341 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.847+0000 7f9332ffd640 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c92156d00 0x562c921590f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d0abfb64562bd server_cookie=e95a927ccd27cefc in_seq=0 out_seq=0 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f931a7fc640 1 -- 172.21.15.113:0/3455094839 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c92041930 con 0x562c92156d00 2024-04-22T07:18:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f931a7fc640 10 client.16341 handle_command_reply: tid=0 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 client.16341 shutdown 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 msgr2=0x562c92143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92143780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f931c003850 tx=0x7f931c004730 comp rx=0 tx=0).stop 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 shutdown_connections 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c92156d00 0x562c921590f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92143780 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 --2- 172.21.15.113:0/3455094839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c921463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f931a7fc640 0 client.16341 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 >> 172.21.15.113:0/3455094839 conn(0x562c92137ba0 msgr2=0x562c921398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 shutdown_connections 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 1 -- 172.21.15.113:0/3455094839 wait complete. 2024-04-22T07:18:06.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.848+0000 7f933bf71740 20 client.16341 trim_cache size 0 max 16384 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 Processor -- start 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 -- start start 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c92152830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92142630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93341b0760 con 0x562c9213de80 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f93341ae000 con 0x562c92146a00 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f933bf71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f93341abe70 con 0x562c9213d4d0 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44854/0 (socket says 172.21.15.113:44854) 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 -- 172.21.15.113:0/1659514985 learned_addr learned my addr 172.21.15.113:0/1659514985 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f93337fe640 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f9332ffd640 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c92152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 -- 172.21.15.113:0/1659514985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 msgr2=0x562c92152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c92152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 -- 172.21.15.113:0/1659514985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 msgr2=0x562c92145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 -- 172.21.15.113:0/1659514985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9213c560 con 0x562c9213de80 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.862+0000 7f932bfff640 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92142630 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f931c0044a0 tx=0x7f931c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe87315fc15c3cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f9328ff9640 1 -- 172.21.15.113:0/1659514985 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c012110 con 0x562c9213de80 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f9328ff9640 1 -- 172.21.15.113:0/1659514985 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f931c0047a0 con 0x562c9213de80 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f9328ff9640 1 -- 172.21.15.113:0/1659514985 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0104f0 con 0x562c9213de80 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 -- 172.21.15.113:0/1659514985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 msgr2=0x562c92142630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92142630 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f931c0044a0 tx=0x7f931c005f10 comp rx=0 tx=0).stop 2024-04-22T07:18:06.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 -- 172.21.15.113:0/1659514985 shutdown_connections 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c92146a00 0x562c92145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c92142630 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 --2- 172.21.15.113:0/1659514985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c9213d4d0 0x562c92152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 -- 172.21.15.113:0/1659514985 >> 172.21.15.113:0/1659514985 conn(0x562c92137ba0 msgr2=0x562c92141f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 -- 172.21.15.113:0/1659514985 shutdown_connections 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.863+0000 7f933bf71740 1 -- 172.21.15.113:0/1659514985 wait complete. 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 -- start start 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 Processor -- start 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9214efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c9213fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93341ac4c0 con 0x562c9213de80 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f93341ad930 con 0x562c9213d4d0 2024-04-22T07:18:06.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f93341acb90 con 0x562c92146a00 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f933bf71740 10 client.0 _ll_register_callbacks cb 0x7f931c0044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c9213fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f9332ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9214efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c9213fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44870/0 (socket says 172.21.15.113:44870) 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 -- 172.21.15.113:0/3423799540 learned_addr learned my addr 172.21.15.113:0/3423799540 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f93337fe640 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 -- 172.21.15.113:0/3423799540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 msgr2=0x562c92140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 -- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 msgr2=0x562c9214efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9214efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 -- 172.21.15.113:0/3423799540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c92140bd0 con 0x562c9213de80 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f932bfff640 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c9213fff0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f931c013d60 tx=0x7f931c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cb3a7977acd2a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.864+0000 7f931a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f931a7fc640 1 -- 172.21.15.113:0/3423799540 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c012110 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f931a7fc640 1 -- 172.21.15.113:0/3423799540 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f931c004e00 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931c0047a0 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f931a7fc640 1 -- 172.21.15.113:0/3423799540 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f931c0122b0 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 10 client.16347 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562c921bd100 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f931a7fc640 1 -- 172.21.15.113:0/3423799540 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f931c0047a0 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f931a7fc640 1 -- 172.21.15.113:0/3423799540 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f931c010a60 con 0x562c9213de80 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 10 client.16347 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:06.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 20 client.16347 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c921bd820 0x562c921bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 4 client.16347 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c920afc10 con 0x562c921bd820 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.865+0000 7f9332ffd640 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c921bd820 0x562c921bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f9332ffd640 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c921bd820 0x562c921bfc40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96dacb96916f4349 server_cookie=d37650a92b7d353f in_seq=0 out_seq=0 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f931a7fc640 10 client.16347 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f931a7fc640 1 -- 172.21.15.113:0/3423799540 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c920afc10 con 0x562c921bd820 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f931a7fc640 10 client.16347 handle_command_reply: tid=0 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 client.16347 shutdown 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 msgr2=0x562c9213fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c9213fff0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f931c013d60 tx=0x7f931c009260 comp rx=0 tx=0).stop 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 shutdown_connections 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562c921bd820 0x562c921bfc40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562c92146a00 0x562c92140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f931a7fc640 0 client.16347 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562c9213de80 0x562c9213fff0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 --2- 172.21.15.113:0/3423799540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9213d4d0 0x562c9214efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 >> 172.21.15.113:0/3423799540 conn(0x562c92137ba0 msgr2=0x562c921427b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 shutdown_connections 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 1 -- 172.21.15.113:0/3423799540 wait complete. 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.866+0000 7f933bf71740 20 client.16347 trim_cache size 0 max 16384 2024-04-22T07:18:06.871 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f93200a3d20 msgr2=0x7f93200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f93200a3d20 0x7f93200a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f931c013460 tx=0x7f931c013860 comp rx=0 tx=0).stop 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 msgr2=0x7f93341a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 0x7f93341a4680 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f932c02c700 tx=0x7f932c035d80 comp rx=0 tx=0).stop 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 shutdown_connections 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f93200a3d20 0x7f93200a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f933419a0d0 0x7f93341a4680 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9334106160 0x7f9334199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 --2- 172.21.15.113:0/3280407248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9334101090 0x7f9334199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 >> 172.21.15.113:0/3280407248 conn(0x7f9334077f50 msgr2=0x7f93340fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 shutdown_connections 2024-04-22T07:18:06.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.867+0000 7f9339385640 1 -- 172.21.15.113:0/3280407248 wait complete. 2024-04-22T07:18:06.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 -- 172.21.15.113:0/1261335589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b501072b0 msgr2=0x7f8b5010eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1261335589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b501072b0 0x7f8b5010eec0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009ca0 tx=0x7f8b4802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 -- 172.21.15.113:0/1261335589 shutdown_connections 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1261335589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b501072b0 0x7f8b5010eec0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1261335589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b501014a0 0x7f8b50106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1261335589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 0x7f8b50100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 -- 172.21.15.113:0/1261335589 >> 172.21.15.113:0/1261335589 conn(0x7f8b500760e0 msgr2=0x7f8b500764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 -- 172.21.15.113:0/1261335589 shutdown_connections 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.979+0000 7f8b54f23640 1 -- 172.21.15.113:0/1261335589 wait complete. 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 Processor -- start 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 -- start start 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 0x7f8b5019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b501072b0 0x7f8b5019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 0x7f8b50198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50112390 con 0x7f8b50198690 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50112210 con 0x7f8b50100af0 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b54f23640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50112510 con 0x7f8b501072b0 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 0x7f8b50198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 0x7f8b50198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44888/0 (socket says 172.21.15.113:44888) 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 -- 172.21.15.113:0/1875846136 learned_addr learned my addr 172.21.15.113:0/1875846136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 0x7f8b5019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b501072b0 0x7f8b5019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 -- 172.21.15.113:0/1875846136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b501072b0 msgr2=0x7f8b5019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b501072b0 0x7f8b5019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 -- 172.21.15.113:0/1875846136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 msgr2=0x7f8b5019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 0x7f8b5019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4ed9d640 1 -- 172.21.15.113:0/1875846136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b50199290 con 0x7f8b50198690 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.980+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 0x7f8b5019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.981+0000 7f8b4ed9d640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 0x7f8b50198b10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8b4802c700 tx=0x7f8b48035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.981+0000 7f8b3f7fe640 1 -- 172.21.15.113:0/1875846136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48039040 con 0x7f8b50198690 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.981+0000 7f8b3f7fe640 1 -- 172.21.15.113:0/1875846136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b48044070 con 0x7f8b50198690 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.981+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b50199520 con 0x7f8b50198690 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.981+0000 7f8b3f7fe640 1 -- 172.21.15.113:0/1875846136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48004330 con 0x7f8b50198690 2024-04-22T07:18:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.981+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b501a5660 con 0x7f8b50198690 2024-04-22T07:18:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.983+0000 7f8b3f7fe640 1 -- 172.21.15.113:0/1875846136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8b480044d0 con 0x7f8b50198690 2024-04-22T07:18:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.983+0000 7f8b57beb740 1 Processor -- start 2024-04-22T07:18:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.983+0000 7f8b57beb740 1 -- start start 2024-04-22T07:18:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 0x555872c7f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7fe80 0x555872c88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c8ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b3f7fe640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b380a3d20 0x7f8b380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b57beb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555872be41f0 con 0x555872c7f4d0 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b57beb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555872b814b0 con 0x555872c88a00 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b57beb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555872be01b0 con 0x555872c7fe80 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 0x555872c7f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4ed9d640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b380a3d20 0x7f8b380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 0x555872c7f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44892/0 (socket says 172.21.15.113:44892) 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 -- 172.21.15.113:0/1572529131 learned_addr learned my addr 172.21.15.113:0/1572529131 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c8ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7fe80 0x555872c88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b3f7fe640 1 -- 172.21.15.113:0/1875846136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8b480f2db0 con 0x7f8b50198690 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 -- 172.21.15.113:0/1572529131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7fe80 msgr2=0x555872c88430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7fe80 0x555872c88430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 -- 172.21.15.113:0/1572529131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 msgr2=0x555872c8ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c8ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 -- 172.21.15.113:0/1572529131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555872b731b0 con 0x555872c7f4d0 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4ed9d640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b380a3d20 0x7f8b380a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b480f0980 tx=0x7f8b480099a0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 0x555872c7f8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8b440148a0 tx=0x7f8b4400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da67710890c17679 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.984+0000 7f8b3cff9640 1 -- 172.21.15.113:0/1572529131 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44014270 con 0x555872c7f4d0 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b3cff9640 1 -- 172.21.15.113:0/1572529131 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b440054f0 con 0x555872c7f4d0 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b3cff9640 1 -- 172.21.15.113:0/1572529131 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44004e40 con 0x555872c7f4d0 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 -- 172.21.15.113:0/1572529131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 msgr2=0x555872c7f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 0x555872c7f8b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8b440148a0 tx=0x7f8b4400b420 comp rx=0 tx=0).stop 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 -- 172.21.15.113:0/1572529131 shutdown_connections 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c8ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7fe80 0x555872c88430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1572529131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7f4d0 0x555872c7f8b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 -- 172.21.15.113:0/1572529131 >> 172.21.15.113:0/1572529131 conn(0x555872c79ba0 msgr2=0x555872c79f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 -- 172.21.15.113:0/1572529131 shutdown_connections 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.985+0000 7f8b57beb740 1 -- 172.21.15.113:0/1572529131 wait complete. 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 -- start start 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 Processor -- start 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c88340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555872c8b910 con 0x555872c88a00 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555872b9d500 con 0x555872c7fe80 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555872c8ba90 con 0x555872c7f4d0 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b57beb740 10 client.0 _ll_register_callbacks cb 0x555872b6bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c85ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44906/0 (socket says 172.21.15.113:44906) 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4dd9b640 1 -- 172.21.15.113:0/336507863 learned_addr learned my addr 172.21.15.113:0/336507863 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c88340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4e59c640 1 -- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 msgr2=0x555872c88340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c88340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4e59c640 1 -- 172.21.15.113:0/336507863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 msgr2=0x555872c85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c85ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4e59c640 1 -- 172.21.15.113:0/336507863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558728f63f0 con 0x555872c7fe80 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.986+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c856d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x555872c80280 tx=0x7f8b40012e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16dcc02f925450e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b35ffb640 1 -- 172.21.15.113:0/336507863 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40004050 con 0x555872c7fe80 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b440054f0 con 0x555872c7fe80 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b35ffb640 1 -- 172.21.15.113:0/336507863 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8b400041f0 con 0x555872c7fe80 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b35ffb640 1 -- 172.21.15.113:0/336507863 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8b400043f0 con 0x555872c7fe80 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 10 client.25778 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5558729c07d0 con 0x555872c7fe80 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b35ffb640 1 -- 172.21.15.113:0/336507863 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b440054f0 con 0x555872c7fe80 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b35ffb640 1 -- 172.21.15.113:0/336507863 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f8b40003ce0 con 0x555872c7fe80 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 10 client.25778 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 10 client.25778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 20 client.25778 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872c98ca0 0x555872c9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 4 client.25778 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.987+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555872b83930 con 0x555872c98ca0 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.988+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872c98ca0 0x555872c9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.988+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872c98ca0 0x555872c9b090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54010200cc52740e server_cookie=ca52c56b56f67399 in_seq=0 out_seq=0 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.988+0000 7f8b35ffb640 10 client.25778 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b35ffb640 1 -- 172.21.15.113:0/336507863 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555872b83930 con 0x555872c98ca0 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b35ffb640 10 client.25778 handle_command_reply: tid=0 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 client.25778 shutdown 2024-04-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 msgr2=0x555872c856d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c856d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x555872c80280 tx=0x7f8b40012e70 comp rx=0 tx=0).stop 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 shutdown_connections 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872c98ca0 0x555872c9b090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c856d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 --2- 172.21.15.113:0/336507863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c88340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b35ffb640 0 client.25778 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 >> 172.21.15.113:0/336507863 conn(0x555872c79ba0 msgr2=0x555872c7b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 shutdown_connections 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 1 -- 172.21.15.113:0/336507863 wait complete. 2024-04-22T07:18:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:06.989+0000 7f8b57beb740 20 client.25778 trim_cache size 0 max 16384 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.002+0000 7f8b57beb740 1 Processor -- start 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.002+0000 7f8b57beb740 1 -- start start 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 0x555872c87e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c88340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b57beb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501b07d0 con 0x555872c7fe80 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b57beb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501ae070 con 0x555872c88a00 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b57beb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501abee0 con 0x555872c7f4d0 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 0x555872c87e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 0x555872c87e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44922/0 (socket says 172.21.15.113:44922) 2024-04-22T07:18:07.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 -- 172.21.15.113:0/1475802047 learned_addr learned my addr 172.21.15.113:0/1475802047 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c88340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 -- 172.21.15.113:0/1475802047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 msgr2=0x555872c94800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c94800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 -- 172.21.15.113:0/1475802047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 msgr2=0x555872c88340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c88340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 -- 172.21.15.113:0/1475802047 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555872c86050 con 0x555872c7fe80 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 0x555872c87e00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8b40000c00 tx=0x7f8b40010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd2ef4ebb6f5a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b37fff640 1 -- 172.21.15.113:0/1475802047 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40010af0 con 0x555872c7fe80 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b37fff640 1 -- 172.21.15.113:0/1475802047 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b4002f070 con 0x555872c7fe80 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.003+0000 7f8b37fff640 1 -- 172.21.15.113:0/1475802047 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40004590 con 0x555872c7fe80 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- 172.21.15.113:0/1475802047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 msgr2=0x555872c87e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 0x555872c87e00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8b40000c00 tx=0x7f8b40010a00 comp rx=0 tx=0).stop 2024-04-22T07:18:07.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- 172.21.15.113:0/1475802047 shutdown_connections 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c88a00 0x555872c88340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c7fe80 0x555872c87e00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- 172.21.15.113:0/1475802047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c94800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- 172.21.15.113:0/1475802047 >> 172.21.15.113:0/1475802047 conn(0x555872c79ba0 msgr2=0x555872c7b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- 172.21.15.113:0/1475802047 shutdown_connections 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- 172.21.15.113:0/1475802047 wait complete. 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- start start 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 Processor -- start 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c8c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c8c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501ac530 con 0x555872c88a00 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501ad9a0 con 0x555872c7fe80 2024-04-22T07:18:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b501acc00 con 0x555872c7f4d0 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.004+0000 7f8b57beb740 10 client.0 _ll_register_callbacks cb 0x7f8b40012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c8c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c8c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4dd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c8c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:53096/0 (socket says 172.21.15.113:53096) 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 -- 172.21.15.113:0/3620426871 learned_addr learned my addr 172.21.15.113:0/3620426871 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 -- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 msgr2=0x555872c90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 -- 172.21.15.113:0/3620426871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 msgr2=0x555872c8c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c8c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 -- 172.21.15.113:0/3620426871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555872c8cd30 con 0x555872c7fe80 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:07.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b4e59c640 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c8c2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b40010500 tx=0x7f8b40011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=458e6efafdf7d380 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b35ffb640 1 -- 172.21.15.113:0/3620426871 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40037020 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4002f070 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b35ffb640 1 -- 172.21.15.113:0/3620426871 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8b40010c80 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b35ffb640 1 -- 172.21.15.113:0/3620426871 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8b40010720 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b57beb740 10 client.25784 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.005+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555872cfee90 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b35ffb640 1 -- 172.21.15.113:0/3620426871 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b4002f070 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b35ffb640 1 -- 172.21.15.113:0/3620426871 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f8b40004750 con 0x555872c7fe80 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 10 client.25784 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 10 client.25784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 20 client.25784 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872cff580 0x555872d019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 4 client.25784 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555872bf1c10 con 0x555872cff580 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872cff580 0x555872d019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b35ffb640 10 client.25784 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b4dd9b640 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872cff580 0x555872d019a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dc7d0dc7ba3e77f server_cookie=c91cd9f7e5e44878 in_seq=0 out_seq=0 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b35ffb640 1 -- 172.21.15.113:0/3620426871 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555872bf1c10 con 0x555872cff580 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b35ffb640 10 client.25784 handle_command_reply: tid=0 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 1 client.25784 shutdown 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 msgr2=0x555872c8c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.006+0000 7f8b57beb740 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c8c2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b40010500 tx=0x7f8b40011d30 comp rx=0 tx=0).stop 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 shutdown_connections 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x555872cff580 0x555872d019a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555872c88a00 0x555872c8c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555872c7fe80 0x555872c8c2b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b35ffb640 0 client.25784 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:07.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 --2- 172.21.15.113:0/3620426871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555872c7f4d0 0x555872c90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 >> 172.21.15.113:0/3620426871 conn(0x555872c79ba0 msgr2=0x555872c80310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 shutdown_connections 2024-04-22T07:18:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 1 -- 172.21.15.113:0/3620426871 wait complete. 2024-04-22T07:18:07.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b57beb740 20 client.25784 trim_cache size 0 max 16384 2024-04-22T07:18:07.012 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b380a3d20 msgr2=0x7f8b380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b380a3d20 0x7f8b380a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b480f0980 tx=0x7f8b480099a0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 msgr2=0x7f8b50198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 0x7f8b50198b10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8b4802c700 tx=0x7f8b48035d80 comp rx=0 tx=0).stop 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 shutdown_connections 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b380a3d20 0x7f8b380a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b50198690 0x7f8b50198b10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b501072b0 0x7f8b5019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 --2- 172.21.15.113:0/1875846136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b50100af0 0x7f8b5019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 >> 172.21.15.113:0/1875846136 conn(0x7f8b500760e0 msgr2=0x7f8b500fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 shutdown_connections 2024-04-22T07:18:07.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.007+0000 7f8b54f23640 1 -- 172.21.15.113:0/1875846136 wait complete. 2024-04-22T07:18:07.015 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:18:07.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:06 smithi133 ceph-mon[31198]: pgmap v105: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-04-22T07:18:07.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:06 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:07.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:06 smithi154 ceph-mon[31862]: pgmap v105: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-04-22T07:18:07.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:06 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- 172.21.15.113:0/696672063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 msgr2=0x7f3cd0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/696672063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd0101470 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009900 tx=0x7f3cc002bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- 172.21.15.113:0/696672063 shutdown_connections 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/696672063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0106160 0x7f3cd010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/696672063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd0101a40 0x7f3cd0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/696672063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd0101470 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- 172.21.15.113:0/696672063 >> 172.21.15.113:0/696672063 conn(0x7f3cd0077f50 msgr2=0x7f3cd0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- 172.21.15.113:0/696672063 shutdown_connections 2024-04-22T07:18:07.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- 172.21.15.113:0/696672063 wait complete. 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 Processor -- start 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- start start 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd0106160 0x7f3cd019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0198650 0x7f3cd0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0111200 con 0x7f3cd0101090 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0111080 con 0x7f3cd0106160 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd6c31640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0111380 con 0x7f3cd0198650 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.119+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44936/0 (socket says 172.21.15.113:44936) 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3923335775 learned_addr learned my addr 172.21.15.113:0/3923335775 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3ccffff640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd0106160 0x7f3cd019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0198650 0x7f3cd0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3923335775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0198650 msgr2=0x7f3cd0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0198650 0x7f3cd0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3923335775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd0106160 msgr2=0x7f3cd019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd0106160 0x7f3cd019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3923335775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0199250 con 0x7f3cd0101090 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd019e590 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0006010 tx=0x7f3cc002cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3ccdffb640 1 -- 172.21.15.113:0/3923335775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0041070 con 0x7f3cd0101090 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd01994e0 con 0x7f3cd0101090 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3ccdffb640 1 -- 172.21.15.113:0/3923335775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cc0004450 con 0x7f3cd0101090 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3ccdffb640 1 -- 172.21.15.113:0/3923335775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0003ce0 con 0x7f3cd0101090 2024-04-22T07:18:07.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.120+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd01a5680 con 0x7f3cd0101090 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.122+0000 7f3ccdffb640 1 -- 172.21.15.113:0/3923335775 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f3cc0044070 con 0x7f3cd0101090 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 Processor -- start 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 -- start start 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3ccdffb640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3cb80a3d20 0x7f3cb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cc88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cd3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557bd0c2d1f0 con 0x557bd0cc84d0 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557bd0bca4b0 con 0x557bd0cd1a00 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd981e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557bd0c291b0 con 0x557bd0cc8e80 2024-04-22T07:18:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cc88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3cb80a3d20 0x7f3cb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cc88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44944/0 (socket says 172.21.15.113:44944) 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 -- 172.21.15.113:0/362490390 learned_addr learned my addr 172.21.15.113:0/362490390 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cd3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3ccdffb640 1 -- 172.21.15.113:0/3923335775 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f3cc00f2200 con 0x7f3cd0101090 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3ccffff640 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 -- 172.21.15.113:0/362490390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 msgr2=0x557bd0cd1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 -- 172.21.15.113:0/362490390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 msgr2=0x557bd0cd3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cd3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3cd49cd640 1 -- 172.21.15.113:0/362490390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd0bbc1b0 con 0x557bd0cc84d0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.123+0000 7f3ccffff640 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3cb80a3d20 0x7f3cb80a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009fd0 tx=0x7f3cc8009de0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cc88b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00be170 tx=0x7f3cc0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca04ae0b4a6d4fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cb6ffd640 1 -- 172.21.15.113:0/362490390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc004b070 con 0x557bd0cc84d0 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cb6ffd640 1 -- 172.21.15.113:0/362490390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cc00a7720 con 0x557bd0cc84d0 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cb6ffd640 1 -- 172.21.15.113:0/362490390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00bf690 con 0x557bd0cc84d0 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd981e740 1 -- 172.21.15.113:0/362490390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 msgr2=0x557bd0cc88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd981e740 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cc88b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00be170 tx=0x7f3cc0005be0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd981e740 1 -- 172.21.15.113:0/362490390 shutdown_connections 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd981e740 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cd3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.124+0000 7f3cd981e740 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 --2- 172.21.15.113:0/362490390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cc88b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- 172.21.15.113:0/362490390 >> 172.21.15.113:0/362490390 conn(0x557bd0cc2ba0 msgr2=0x557bd0cc2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- 172.21.15.113:0/362490390 shutdown_connections 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- 172.21.15.113:0/362490390 wait complete. 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- start start 2024-04-22T07:18:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 Processor -- start 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cd1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557bd0cd4730 con 0x557bd0cc84d0 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557bd0be6500 con 0x557bd0cd1a00 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557bd0cd48b0 con 0x557bd0cc8e80 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd981e740 10 client.0 _ll_register_callbacks cb 0x557bd0bb4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cd1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cd1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44954/0 (socket says 172.21.15.113:44954) 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3580734407 learned_addr learned my addr 172.21.15.113:0/3580734407 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.125+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3ccffff640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3580734407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 msgr2=0x557bd0cce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 msgr2=0x557bd0cceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd49cd640 1 -- 172.21.15.113:0/3580734407 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd093f3f0 con 0x557bd0cc84d0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3ccffff640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cd1210 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3cc002e040 tx=0x7f3cc00f2190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07d7410f6aba643 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3caffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3caffff640 1 -- 172.21.15.113:0/3580734407 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc004b040 con 0x557bd0cc84d0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3caffff640 1 -- 172.21.15.113:0/3580734407 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f3cc00c20d0 con 0x557bd0cc84d0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc00a7720 con 0x557bd0cc84d0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3caffff640 1 -- 172.21.15.113:0/3580734407 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3cc00c3670 con 0x557bd0cc84d0 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd981e740 10 client.16377 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557bd0a097d0 con 0x557bd0cc84d0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3caffff640 1 -- 172.21.15.113:0/3580734407 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cc00a7720 con 0x557bd0cc84d0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3caffff640 1 -- 172.21.15.113:0/3580734407 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f3cc00de040 con 0x557bd0cc84d0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd981e740 10 client.16377 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd981e740 10 client.16377 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.126+0000 7f3cd981e740 20 client.16377 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.127+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0ce16b0 0x557bd0ce3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.127+0000 7f3cd981e740 4 client.16377 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.127+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bd0bcc930 con 0x557bd0ce16b0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.127+0000 7f3ccffff640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0ce16b0 0x557bd0ce3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.127+0000 7f3ccffff640 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0ce16b0 0x557bd0ce3aa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9f35ce228e44170 server_cookie=69ae18d223085759 in_seq=0 out_seq=0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.127+0000 7f3caffff640 10 client.16377 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.128+0000 7f3caffff640 1 -- 172.21.15.113:0/3580734407 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bd0bcc930 con 0x557bd0ce16b0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.128+0000 7f3caffff640 10 client.16377 handle_command_reply: tid=0 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.128+0000 7f3cd981e740 1 client.16377 shutdown 2024-04-22T07:18:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.128+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 msgr2=0x557bd0cd1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.128+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cd1210 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3cc002e040 tx=0x7f3cc00f2190 comp rx=0 tx=0).stop 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 shutdown_connections 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0ce16b0 0x557bd0ce3aa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0cceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3caffff640 0 client.16377 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3580734407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0cd1210 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 >> 172.21.15.113:0/3580734407 conn(0x557bd0cc2ba0 msgr2=0x557bd0cc48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 shutdown_connections 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 1 -- 172.21.15.113:0/3580734407 wait complete. 2024-04-22T07:18:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.129+0000 7f3cd981e740 20 client.16377 trim_cache size 0 max 16384 2024-04-22T07:18:07.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.142+0000 7f3cd981e740 1 Processor -- start 2024-04-22T07:18:07.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.142+0000 7f3cd981e740 1 -- start start 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.142+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0ccdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0ce9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd981e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b07c0 con 0x557bd0cc84d0 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd981e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01ae060 con 0x557bd0cd1a00 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd981e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01abed0 con 0x557bd0cc8e80 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0ccdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0ccdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44960/0 (socket says 172.21.15.113:44960) 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 -- 172.21.15.113:0/491370834 learned_addr learned my addr 172.21.15.113:0/491370834 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0ce9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 -- 172.21.15.113:0/491370834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 msgr2=0x557bd0cd0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 -- 172.21.15.113:0/491370834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 msgr2=0x557bd0ce9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0ce9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 -- 172.21.15.113:0/491370834 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bd0cc8be0 con 0x557bd0cc84d0 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0ce9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0ccdef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00bc490 tx=0x7f3cc00bc6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec54e89bc3a67a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cb67fc640 1 -- 172.21.15.113:0/491370834 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc004b050 con 0x557bd0cc84d0 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cb67fc640 1 -- 172.21.15.113:0/491370834 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cc00be1e0 con 0x557bd0cc84d0 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.143+0000 7f3cb67fc640 1 -- 172.21.15.113:0/491370834 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00c6da0 con 0x557bd0cc84d0 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 -- 172.21.15.113:0/491370834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 msgr2=0x557bd0ccdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0ccdef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00bc490 tx=0x7f3cc00bc6f0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 -- 172.21.15.113:0/491370834 shutdown_connections 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cd1a00 0x557bd0ce9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 --2- 172.21.15.113:0/491370834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cc84d0 0x557bd0ccdef0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 -- 172.21.15.113:0/491370834 >> 172.21.15.113:0/491370834 conn(0x557bd0cc2ba0 msgr2=0x557bd0cc1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 -- 172.21.15.113:0/491370834 shutdown_connections 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 -- 172.21.15.113:0/491370834 wait complete. 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 -- start start 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.144+0000 7f3cd981e740 1 Processor -- start 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cc84d0 0x557bd0cd4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 0x557bd0d45b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01ac520 con 0x557bd0cd1a00 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01ad990 con 0x557bd0cc84d0 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01acbf0 con 0x557bd0cc8e80 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 10 client.0 _ll_register_callbacks cb 0x7f3cc0003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 0x557bd0d45b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd49cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cc84d0 0x557bd0cd4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 0x557bd0d45b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44964/0 (socket says 172.21.15.113:44964) 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 -- 172.21.15.113:0/3406353320 learned_addr learned my addr 172.21.15.113:0/3406353320 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3ccffff640 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 -- 172.21.15.113:0/3406353320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 msgr2=0x557bd0cd50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 -- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cc84d0 msgr2=0x557bd0cd4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cc84d0 0x557bd0cd4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 -- 172.21.15.113:0/3406353320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bd0d460a0 con 0x557bd0cd1a00 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd51ce640 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 0x557bd0d45b60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80130b0 tx=0x7f3cc801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93278d4046b938a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3caffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3caffff640 1 -- 172.21.15.113:0/3406353320 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8031060 con 0x557bd0cd1a00 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3caffff640 1 -- 172.21.15.113:0/3406353320 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f3cc80043d0 con 0x557bd0cd1a00 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc00be1e0 con 0x557bd0cd1a00 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3caffff640 1 -- 172.21.15.113:0/3406353320 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3cc8038050 con 0x557bd0cd1a00 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 10 client.16383 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:07.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.145+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557bd0d47340 con 0x557bd0cd1a00 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3caffff640 1 -- 172.21.15.113:0/3406353320 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cc00be1e0 con 0x557bd0cd1a00 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3caffff640 1 -- 172.21.15.113:0/3406353320 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f3cc802b070 con 0x557bd0cd1a00 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd981e740 10 client.16383 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd981e740 10 client.16383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd981e740 20 client.16383 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0d47ac0 0x557bd0d49ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd981e740 4 client.16383 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bd0c3ac10 con 0x557bd0d47ac0 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0d47ac0 0x557bd0d49ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3cd49cd640 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0d47ac0 0x557bd0d49ee0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=398c890c0cc17fcc server_cookie=d038a5c92d3a33d in_seq=0 out_seq=0 2024-04-22T07:18:07.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.146+0000 7f3caffff640 10 client.16383 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:07.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3caffff640 1 -- 172.21.15.113:0/3406353320 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bd0c3ac10 con 0x557bd0d47ac0 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3caffff640 10 client.16383 handle_command_reply: tid=0 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 client.16383 shutdown 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 msgr2=0x557bd0d45b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 0x557bd0d45b60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80130b0 tx=0x7f3cc801d000 comp rx=0 tx=0).stop 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 shutdown_connections 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557bd0d47ac0 0x557bd0d49ee0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557bd0cd1a00 0x557bd0d45b60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557bd0cc8e80 0x557bd0cd50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 --2- 172.21.15.113:0/3406353320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bd0cc84d0 0x557bd0cd4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3caffff640 0 client.16383 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.148+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 >> 172.21.15.113:0/3406353320 conn(0x557bd0cc2ba0 msgr2=0x557bd0cc3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 shutdown_connections 2024-04-22T07:18:07.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd981e740 1 -- 172.21.15.113:0/3406353320 wait complete. 2024-04-22T07:18:07.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd981e740 20 client.16383 trim_cache size 0 max 16384 2024-04-22T07:18:07.152 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3cb80a3d20 msgr2=0x7f3cb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3cb80a3d20 0x7f3cb80a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009fd0 tx=0x7f3cc8009de0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 msgr2=0x7f3cd019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd019e590 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0006010 tx=0x7f3cc002cc40 comp rx=0 tx=0).stop 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 shutdown_connections 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3cb80a3d20 0x7f3cb80a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cd0198650 0x7f3cd0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd0106160 0x7f3cd019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 --2- 172.21.15.113:0/3923335775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cd0101090 0x7f3cd019e590 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 >> 172.21.15.113:0/3923335775 conn(0x7f3cd0077f50 msgr2=0x7f3cd0069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:07.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 shutdown_connections 2024-04-22T07:18:07.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:07.149+0000 7f3cd6c31640 1 -- 172.21.15.113:0/3923335775 wait complete. 2024-04-22T07:18:08.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:07 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:08.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:07 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1779470297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:08.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:07 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:08.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:07 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1779470297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:07 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:08.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:07 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1779470297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:08.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:18:08.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:18:08.159 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:18:08.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 -- 172.21.15.113:0/651376125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec101a40 msgr2=0x7f97ec105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/651376125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec101a40 0x7f97ec105a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e002bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:08.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 -- 172.21.15.113:0/651376125 shutdown_connections 2024-04-22T07:18:08.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/651376125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec106160 0x7f97ec10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/651376125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec101a40 0x7f97ec105a80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/651376125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec101090 0x7f97ec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 -- 172.21.15.113:0/651376125 >> 172.21.15.113:0/651376125 conn(0x7f97ec077f50 msgr2=0x7f97ec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 -- 172.21.15.113:0/651376125 shutdown_connections 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.267+0000 7f97f1f3d640 1 -- 172.21.15.113:0/651376125 wait complete. 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 Processor -- start 2024-04-22T07:18:08.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 -- start start 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec101090 0x7f97ec19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec106160 0x7f97ec19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 0x7f97ec198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1111e0 con 0x7f97ec198640 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec111060 con 0x7f97ec101090 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97f1f3d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec111360 con 0x7f97ec106160 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 0x7f97ec198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 0x7f97ec198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44976/0 (socket says 172.21.15.113:44976) 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 -- 172.21.15.113:0/2769323177 learned_addr learned my addr 172.21.15.113:0/2769323177 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97eaffd640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec106160 0x7f97ec19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec101090 0x7f97ec19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 -- 172.21.15.113:0/2769323177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec106160 msgr2=0x7f97ec19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec106160 0x7f97ec19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 -- 172.21.15.113:0/2769323177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec101090 msgr2=0x7f97ec19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec101090 0x7f97ec19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.268+0000 7f97ebfff640 1 -- 172.21.15.113:0/2769323177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec199240 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.269+0000 7f97ebfff640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 0x7f97ec198ac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f97e400eb10 tx=0x7f97e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.269+0000 7f97e8ff9640 1 -- 172.21.15.113:0/2769323177 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4004b20 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.269+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec1a5320 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.269+0000 7f97e8ff9640 1 -- 172.21.15.113:0/2769323177 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f97e4004cc0 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.269+0000 7f97e8ff9640 1 -- 172.21.15.113:0/2769323177 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4010720 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.269+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec1a56d0 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.271+0000 7f97e8ff9640 1 -- 172.21.15.113:0/2769323177 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f97e4005020 con 0x7f97ec198640 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.271+0000 7f97f4c07740 1 Processor -- start 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.271+0000 7f97f4c07740 1 -- start start 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.271+0000 7f97e8ff9640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f97d40a3d20 0x7f97d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca17bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97e8ff9640 1 -- 172.21.15.113:0/2769323177 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f97e40d0e70 con 0x7f97ec198640 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eaffd640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f97d40a3d20 0x7f97d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca184840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca187260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97f4c07740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560dca0dff00 con 0x560dca17b8e0 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97f4c07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560dca07d590 con 0x560dca17c290 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97f4c07740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560dca0dbec0 con 0x560dca184e10 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca17bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca17bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44992/0 (socket says 172.21.15.113:44992) 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca184840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 -- 172.21.15.113:0/219871004 learned_addr learned my addr 172.21.15.113:0/219871004 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97ebfff640 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca187260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 -- 172.21.15.113:0/219871004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 msgr2=0x560dca187260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca187260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 -- 172.21.15.113:0/219871004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 msgr2=0x560dca184840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca184840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 -- 172.21.15.113:0/219871004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc9eba7e0 con 0x560dca17b8e0 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eaffd640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f97d40a3d20 0x7f97d40a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005ab0 tx=0x7f97e0009680 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca17bcc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d20 tx=0x7f97dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae517662135b890 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97d1ffb640 1 -- 172.21.15.113:0/219871004 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc01e070 con 0x560dca17b8e0 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97d1ffb640 1 -- 172.21.15.113:0/219871004 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f97dc009e10 con 0x560dca17b8e0 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.272+0000 7f97d1ffb640 1 -- 172.21.15.113:0/219871004 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00e990 con 0x560dca17b8e0 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 -- 172.21.15.113:0/219871004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 msgr2=0x560dca17bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca17bcc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d20 tx=0x7f97dc011810 comp rx=0 tx=0).stop 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 -- 172.21.15.113:0/219871004 shutdown_connections 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca187260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca184840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 --2- 172.21.15.113:0/219871004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca17bcc0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 -- 172.21.15.113:0/219871004 >> 172.21.15.113:0/219871004 conn(0x560dca175fb0 msgr2=0x560dca176390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 -- 172.21.15.113:0/219871004 shutdown_connections 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 -- 172.21.15.113:0/219871004 wait complete. 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.273+0000 7f97f4c07740 1 -- start start 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 Processor -- start 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca17b8e0 0x560dca181df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca182330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca184e10 0x560dca182870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560dca187d80 con 0x560dca17c290 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560dca099580 con 0x560dca184e10 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560dca187f00 con 0x560dca17b8e0 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97f4c07740 10 client.0 _ll_register_callbacks cb 0x560dca09f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca182330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca182330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45006/0 (socket says 172.21.15.113:45006) 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 -- 172.21.15.113:0/135635325 learned_addr learned my addr 172.21.15.113:0/135635325 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97ebfff640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca184e10 0x560dca182870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca17b8e0 0x560dca181df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 -- 172.21.15.113:0/135635325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca17b8e0 msgr2=0x560dca181df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca17b8e0 0x560dca181df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 -- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca184e10 msgr2=0x560dca182870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca184e10 0x560dca182870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 -- 172.21.15.113:0/135635325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc9f1af70 con 0x560dca17c290 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97ebfff640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca184e10 0x560dca182870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97eaffd640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca182330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005c40 tx=0x7f97e002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc778eec290421 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.274+0000 7f97caffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97caffd640 1 -- 172.21.15.113:0/135635325 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0052060 con 0x560dca17c290 2024-04-22T07:18:08.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97caffd640 1 -- 172.21.15.113:0/135635325 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f97e00046c0 con 0x560dca17c290 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97dc009e10 con 0x560dca17c290 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97caffd640 1 -- 172.21.15.113:0/135635325 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f97e0059050 con 0x560dca17c290 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 10 client.16395 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560dc9db2170 con 0x560dca17c290 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97caffd640 1 -- 172.21.15.113:0/135635325 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f97dc009e10 con 0x560dca17c290 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97caffd640 1 -- 172.21.15.113:0/135635325 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f97e0003b10 con 0x560dca17c290 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 10 client.16395 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 20 client.16395 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca197810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 4 client.16395 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dca0ed920 con 0x560dca195400 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.275+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97caffd640 10 client.16395 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca197810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbfc017ceba1dab server_cookie=49a978b403c25230 in_seq=0 out_seq=0 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97caffd640 1 -- 172.21.15.113:0/135635325 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dca0ed920 con 0x560dca195400 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97caffd640 10 client.16395 handle_command_reply: tid=0 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 client.16395 shutdown 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 msgr2=0x560dca182330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca182330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005c40 tx=0x7f97e002bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 shutdown_connections 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca197810 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca184e10 0x560dca182870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca182330 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97caffd640 0 client.16395 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.276+0000 7f97f4c07740 1 --2- 172.21.15.113:0/135635325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca17b8e0 0x560dca181df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.277+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 >> 172.21.15.113:0/135635325 conn(0x560dca175fb0 msgr2=0x560dca177d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.277+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 shutdown_connections 2024-04-22T07:18:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.277+0000 7f97f4c07740 1 -- 172.21.15.113:0/135635325 wait complete. 2024-04-22T07:18:08.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.277+0000 7f97f4c07740 20 client.16395 trim_cache size 0 max 16384 2024-04-22T07:18:08.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.295+0000 7f97f4c07740 1 Processor -- start 2024-04-22T07:18:08.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.295+0000 7f97f4c07740 1 -- start start 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca184520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca180d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca190c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97f4c07740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1b0770 con 0x560dca17b8e0 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97f4c07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1ae010 con 0x560dca17c290 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97f4c07740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1abe60 con 0x560dca184e10 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca184520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca184520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45022/0 (socket says 172.21.15.113:45022) 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 -- 172.21.15.113:0/1962845398 learned_addr learned my addr 172.21.15.113:0/1962845398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eaffd640 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca180d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97ebfff640 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca190c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 -- 172.21.15.113:0/1962845398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 msgr2=0x560dca190c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca190c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 -- 172.21.15.113:0/1962845398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 msgr2=0x560dca180d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca180d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 -- 172.21.15.113:0/1962845398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dca17ac30 con 0x560dca17b8e0 2024-04-22T07:18:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eaffd640 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca180d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.296+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca184520 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009cf0 tx=0x7f97dc009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c090eda1d5290d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97d17fa640 1 -- 172.21.15.113:0/1962845398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc02d070 con 0x560dca17b8e0 2024-04-22T07:18:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97d17fa640 1 -- 172.21.15.113:0/1962845398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f97dc005040 con 0x560dca17b8e0 2024-04-22T07:18:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97d17fa640 1 -- 172.21.15.113:0/1962845398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc029e60 con 0x560dca17b8e0 2024-04-22T07:18:08.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 -- 172.21.15.113:0/1962845398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 msgr2=0x560dca184520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca184520 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009cf0 tx=0x7f97dc009a60 comp rx=0 tx=0).stop 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 -- 172.21.15.113:0/1962845398 shutdown_connections 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca190c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17c290 0x560dca180d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 --2- 172.21.15.113:0/1962845398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17b8e0 0x560dca184520 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 -- 172.21.15.113:0/1962845398 >> 172.21.15.113:0/1962845398 conn(0x560dca175fb0 msgr2=0x560dca182640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.297+0000 7f97f4c07740 1 -- 172.21.15.113:0/1962845398 shutdown_connections 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 -- 172.21.15.113:0/1962845398 wait complete. 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 -- start start 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 Processor -- start 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17b8e0 0x560dca1891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca189710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca1fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1ac4d0 con 0x560dca17c290 2024-04-22T07:18:08.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1ad940 con 0x560dca17b8e0 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1acba0 con 0x560dca184e10 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97f4c07740 10 client.0 _ll_register_callbacks cb 0x7f97e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca189710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca189710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45032/0 (socket says 172.21.15.113:45032) 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17b8e0 0x560dca1891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97eaffd640 1 -- 172.21.15.113:0/2331288864 learned_addr learned my addr 172.21.15.113:0/2331288864 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.298+0000 7f97ebfff640 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca1fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97eaffd640 1 -- 172.21.15.113:0/2331288864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 msgr2=0x560dca1fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97eaffd640 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca1fa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97eaffd640 1 -- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17b8e0 msgr2=0x560dca1891d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97eaffd640 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17b8e0 0x560dca1891d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97eaffd640 1 -- 172.21.15.113:0/2331288864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dca1faed0 con 0x560dca17c290 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97eaffd640 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca189710 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f97e00057c0 tx=0x7f97e0042440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7504e57cd4db0d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97caffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97caffd640 1 -- 172.21.15.113:0/2331288864 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0062060 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97dc005040 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97caffd640 1 -- 172.21.15.113:0/2331288864 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f97e0042880 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97caffd640 1 -- 172.21.15.113:0/2331288864 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f97e0069050 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97f4c07740 10 client.16401 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560dca1fc140 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97caffd640 1 -- 172.21.15.113:0/2331288864 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f97dc005040 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97caffd640 1 -- 172.21.15.113:0/2331288864 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f97e00046c0 con 0x560dca17c290 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.299+0000 7f97f4c07740 10 client.16401 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97f4c07740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97f4c07740 20 client.16401 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97f4c07740 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca1fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97f4c07740 4 client.16401 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560dca0e7790 con 0x560dca195400 2024-04-22T07:18:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca1fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97eb7fe640 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca1fe8d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76360cadd45ab93 server_cookie=4f84688fbdc90024 in_seq=0 out_seq=0 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.300+0000 7f97caffd640 10 client.16401 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97caffd640 1 -- 172.21.15.113:0/2331288864 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560dca0e7790 con 0x560dca195400 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97caffd640 10 client.16401 handle_command_reply: tid=0 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 client.16401 shutdown 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 msgr2=0x560dca189710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca189710 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f97e00057c0 tx=0x7f97e0042440 comp rx=0 tx=0).stop 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 shutdown_connections 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x560dca195400 0x560dca1fe8d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560dca184e10 0x560dca1fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560dca17c290 0x560dca189710 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 --2- 172.21.15.113:0/2331288864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dca17b8e0 0x560dca1891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 >> 172.21.15.113:0/2331288864 conn(0x560dca175fb0 msgr2=0x560dca177330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97caffd640 0 client.16401 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 shutdown_connections 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 1 -- 172.21.15.113:0/2331288864 wait complete. 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f4c07740 20 client.16401 trim_cache size 0 max 16384 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "05879fab-789b-4a92-9b0f-595c39fdf236", 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:18:08.305 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f97d40a3d20 msgr2=0x7f97d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f97d40a3d20 0x7f97d40a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005ab0 tx=0x7f97e0009680 comp rx=0 tx=0).stop 2024-04-22T07:18:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 msgr2=0x7f97ec198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.302+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 0x7f97ec198ac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f97e400eb10 tx=0x7f97e400c2d0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 shutdown_connections 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f97d40a3d20 0x7f97d40a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ec198640 0x7f97ec198ac0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec106160 0x7f97ec19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 --2- 172.21.15.113:0/2769323177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec101090 0x7f97ec19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 >> 172.21.15.113:0/2769323177 conn(0x7f97ec077f50 msgr2=0x7f97ec0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 shutdown_connections 2024-04-22T07:18:08.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.303+0000 7f97f1f3d640 1 -- 172.21.15.113:0/2769323177 wait complete. 2024-04-22T07:18:08.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:18:08.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 -- 172.21.15.113:0/1590577281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf4106150 msgr2=0x7fddf410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 --2- 172.21.15.113:0/1590577281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf4106150 0x7fddf410dd20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009900 tx=0x7fdde402bc80 comp rx=0 tx=0).stop 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 -- 172.21.15.113:0/1590577281 shutdown_connections 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 --2- 172.21.15.113:0/1590577281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf4106150 0x7fddf410dd20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 --2- 172.21.15.113:0/1590577281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101a30 0x7fddf4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 --2- 172.21.15.113:0/1590577281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4101080 0x7fddf4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 -- 172.21.15.113:0/1590577281 >> 172.21.15.113:0/1590577281 conn(0x7fddf4078000 msgr2=0x7fddf4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 -- 172.21.15.113:0/1590577281 shutdown_connections 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 -- 172.21.15.113:0/1590577281 wait complete. 2024-04-22T07:18:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.439+0000 7fddf9306640 1 Processor -- start 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.440+0000 7fddf9306640 1 -- start start 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf9306640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101080 0x7fddf410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf9306640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4106150 0x7fddf410aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf9306640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 0x7fddf410b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf9306640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4110230 con 0x7fddf410afa0 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf9306640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41100b0 con 0x7fddf4101080 2024-04-22T07:18:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf9306640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41103b0 con 0x7fddf4106150 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 0x7fddf410b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101080 0x7fddf410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101080 0x7fddf410d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:53226/0 (socket says 172.21.15.113:53226) 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 0x7fddf410b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45056/0 (socket says 172.21.15.113:45056) 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf2ffd640 1 -- 172.21.15.113:0/2904413590 learned_addr learned my addr 172.21.15.113:0/2904413590 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf37fe640 1 -- 172.21.15.113:0/2904413590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4106150 msgr2=0x7fddf410aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.441+0000 7fddf27fc640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4106150 0x7fddf410aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.442+0000 7fddf37fe640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4106150 0x7fddf410aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.442+0000 7fddf37fe640 1 -- 172.21.15.113:0/2904413590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101080 msgr2=0x7fddf410d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.442+0000 7fddf37fe640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101080 0x7fddf410d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.442+0000 7fddf37fe640 1 -- 172.21.15.113:0/2904413590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf41ab590 con 0x7fddf410afa0 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.442+0000 7fddf37fe640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 0x7fddf410b450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdde4036040 tx=0x7fdde402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.442+0000 7fddf27fc640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4106150 0x7fddf410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.443+0000 7fdddbfff640 1 -- 172.21.15.113:0/2904413590 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4041070 con 0x7fddf410afa0 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.443+0000 7fdddbfff640 1 -- 172.21.15.113:0/2904413590 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdde40052f0 con 0x7fddf410afa0 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.443+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf41ab820 con 0x7fddf410afa0 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.443+0000 7fdddbfff640 1 -- 172.21.15.113:0/2904413590 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4041070 con 0x7fddf410afa0 2024-04-22T07:18:08.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.443+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf41abd40 con 0x7fddf410afa0 2024-04-22T07:18:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.446+0000 7fdddbfff640 1 -- 172.21.15.113:0/2904413590 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fdde4044070 con 0x7fddf410afa0 2024-04-22T07:18:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.446+0000 7fddfbfcc740 1 Processor -- start 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.447+0000 7fdddbfff640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdddc0a3f20 0x7fdddc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.447+0000 7fdddbfff640 1 -- 172.21.15.113:0/2904413590 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fdde40bbdf0 con 0x7fddf410afa0 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.447+0000 7fddf27fc640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdddc0a3f20 0x7fdddc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 -- start start 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59d580 0x55973c59d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55973c5021e0 con 0x55973c59d580 2024-04-22T07:18:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55973c49f390 con 0x55973c59dea0 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55973c4fe1a0 con 0x55973c5b7a90 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5b9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:43982/0 (socket says 172.21.15.113:43982) 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddf37fe640 1 -- 172.21.15.113:0/1323936370 learned_addr learned my addr 172.21.15.113:0/1323936370 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.448+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59d580 0x55973c59d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf37fe640 1 -- 172.21.15.113:0/1323936370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 msgr2=0x55973c5a6540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf37fe640 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5a6540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf37fe640 1 -- 172.21.15.113:0/1323936370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59d580 msgr2=0x55973c59d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf37fe640 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59d580 0x55973c59d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf37fe640 1 -- 172.21.15.113:0/1323936370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973c2de7d0 con 0x55973c5b7a90 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf27fc640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdddc0a3f20 0x7fdddc0a63e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fddf410c1d0 tx=0x7fdde8008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59d580 0x55973c59d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddf37fe640 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5b9e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdde40bafd0 tx=0x7fdde40c3220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f2ba20a369bc37 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddd97fa640 1 -- 172.21.15.113:0/1323936370 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40ba780 con 0x55973c5b7a90 2024-04-22T07:18:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddd97fa640 1 -- 172.21.15.113:0/1323936370 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdde4004c70 con 0x55973c5b7a90 2024-04-22T07:18:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.449+0000 7fddd97fa640 1 -- 172.21.15.113:0/1323936370 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40ba920 con 0x55973c5b7a90 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.450+0000 7fddfbfcc740 1 -- 172.21.15.113:0/1323936370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 msgr2=0x55973c5b9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.450+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5b9e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdde40bafd0 tx=0x7fdde40c3220 comp rx=0 tx=0).stop 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 -- 172.21.15.113:0/1323936370 shutdown_connections 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5b9e80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/1323936370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59d580 0x55973c59d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 -- 172.21.15.113:0/1323936370 >> 172.21.15.113:0/1323936370 conn(0x55973c597cb0 msgr2=0x55973c598090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 -- 172.21.15.113:0/1323936370 shutdown_connections 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 -- 172.21.15.113:0/1323936370 wait complete. 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.451+0000 7fddfbfcc740 1 -- start start 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 Processor -- start 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59dea0 0x55973c5a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55973c5ba640 con 0x55973c59dea0 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55973c4bb2f0 con 0x55973c59d580 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55973c5ba7c0 con 0x55973c5b7a90 2024-04-22T07:18:08.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.452+0000 7fddfbfcc740 10 client.0 _ll_register_callbacks cb 0x55973c4c8580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:08.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5a44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:43986/0 (socket says 172.21.15.113:43986) 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 -- 172.21.15.113:0/183594588 learned_addr learned my addr 172.21.15.113:0/183594588 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf27fc640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59dea0 0x55973c5a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 -- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 msgr2=0x55973c5a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 -- 172.21.15.113:0/183594588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59dea0 msgr2=0x55973c5a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59dea0 0x55973c5a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 -- 172.21.15.113:0/183594588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973c2143f0 con 0x55973c5b7a90 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddf37fe640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5a44f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdde402e040 tx=0x7fdde40be590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380ad96bc065da01 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.453+0000 7fddd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddd6ff5640 1 -- 172.21.15.113:0/183594588 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde404b040 con 0x55973c5b7a90 2024-04-22T07:18:08.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde4004c70 con 0x55973c5b7a90 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddd6ff5640 1 -- 172.21.15.113:0/183594588 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fdde40ef670 con 0x55973c5b7a90 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddd6ff5640 1 -- 172.21.15.113:0/183594588 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdde40baac0 con 0x55973c5b7a90 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddfbfcc740 10 client.25672 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55973c1d40f0 con 0x55973c5b7a90 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.454+0000 7fddd6ff5640 1 -- 172.21.15.113:0/183594588 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdde4004c70 con 0x55973c5b7a90 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddd6ff5640 1 -- 172.21.15.113:0/183594588 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fdde40b9e20 con 0x55973c5b7a90 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddfbfcc740 10 client.25672 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddfbfcc740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddfbfcc740 20 client.25672 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c5d8600 0x55973c5da9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddfbfcc740 4 client.25672 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.455+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55973c50fc00 con 0x55973c5d8600 2024-04-22T07:18:08.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.456+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c5d8600 0x55973c5da9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.457+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c5d8600 0x55973c5da9f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b589e8540bc2ebaf server_cookie=e9964e69ef96aa61 in_seq=0 out_seq=0 2024-04-22T07:18:08.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.457+0000 7fddd6ff5640 10 client.25672 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddd6ff5640 1 -- 172.21.15.113:0/183594588 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55973c50fc00 con 0x55973c5d8600 2024-04-22T07:18:08.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddd6ff5640 10 client.25672 handle_command_reply: tid=0 2024-04-22T07:18:08.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 client.25672 shutdown 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 msgr2=0x55973c5a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5a44f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdde402e040 tx=0x7fdde40be590 comp rx=0 tx=0).stop 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 shutdown_connections 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c5d8600 0x55973c5da9f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c5b7a90 0x55973c5a44f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c59dea0 0x55973c5a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/183594588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 >> 172.21.15.113:0/183594588 conn(0x55973c597cb0 msgr2=0x55973c599a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddd6ff5640 0 client.25672 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 shutdown_connections 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 1 -- 172.21.15.113:0/183594588 wait complete. 2024-04-22T07:18:08.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.458+0000 7fddfbfcc740 20 client.25672 trim_cache size 0 max 16384 2024-04-22T07:18:08.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.480+0000 7fddfbfcc740 1 Processor -- start 2024-04-22T07:18:08.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.480+0000 7fddfbfcc740 1 -- start start 2024-04-22T07:18:08.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.480+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.480+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59dea0 0x55973c5c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.480+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c59ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.480+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00111f0 con 0x55973c5b7a90 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000ea90 con 0x55973c59d580 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000c8e0 con 0x55973c59dea0 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c59ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c59ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45076/0 (socket says 172.21.15.113:45076) 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 -- 172.21.15.113:0/3642762815 learned_addr learned my addr 172.21.15.113:0/3642762815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf27fc640 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59dea0 0x55973c5c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 -- 172.21.15.113:0/3642762815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59dea0 msgr2=0x55973c5c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59dea0 0x55973c5c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 -- 172.21.15.113:0/3642762815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 msgr2=0x55973c5a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 -- 172.21.15.113:0/3642762815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973c5a2a20 con 0x55973c5b7a90 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddf37fe640 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c59ef90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdde40bb1a0 tx=0x7fdde4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6740efd613b7a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddd8ff9640 1 -- 172.21.15.113:0/3642762815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde404b070 con 0x55973c5b7a90 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddd8ff9640 1 -- 172.21.15.113:0/3642762815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdde40de330 con 0x55973c5b7a90 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.481+0000 7fddd8ff9640 1 -- 172.21.15.113:0/3642762815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40bac90 con 0x55973c5b7a90 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3642762815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 msgr2=0x55973c59ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c59ef90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdde40bb1a0 tx=0x7fdde4005b50 comp rx=0 tx=0).stop 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3642762815 shutdown_connections 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c59ef90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59dea0 0x55973c5c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3642762815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59d580 0x55973c5a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3642762815 >> 172.21.15.113:0/3642762815 conn(0x55973c597cb0 msgr2=0x55973c599100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3642762815 shutdown_connections 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3642762815 wait complete. 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.482+0000 7fddfbfcc740 1 -- start start 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 Processor -- start 2024-04-22T07:18:08.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 0x55973c5bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c5bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000cf50 con 0x55973c5b7a90 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000e3c0 con 0x55973c59dea0 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000d620 con 0x55973c59d580 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddfbfcc740 10 client.0 _ll_register_callbacks cb 0x7fdde40c0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 0x55973c5bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 0x55973c5bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:43994/0 (socket says 172.21.15.113:43994) 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5bb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:53276/0 (socket says 172.21.15.113:53276) 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 -- 172.21.15.113:0/3142452866 learned_addr learned my addr 172.21.15.113:0/3142452866 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf37fe640 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c5bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 -- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 msgr2=0x55973c5bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5bb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 -- 172.21.15.113:0/3142452866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 msgr2=0x55973c5bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c5bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.483+0000 7fddf2ffd640 1 -- 172.21.15.113:0/3142452866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973c5bbed0 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddf2ffd640 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 0x55973c5bfdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddec01dba0 tx=0x7fddec007bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f51e4eef5e92b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddd6ff5640 1 -- 172.21.15.113:0/3142452866 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec01dc90 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddd6ff5640 1 -- 172.21.15.113:0/3142452866 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fddec0178f0 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde40de330 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddd6ff5640 1 -- 172.21.15.113:0/3142452866 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fddec01de30 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 10 client.25678 fetch_fsmap learned FSMap version 45 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55973c63eb70 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddd6ff5640 1 -- 172.21.15.113:0/3142452866 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdde40de330 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddd6ff5640 1 -- 172.21.15.113:0/3142452866 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fddec026470 con 0x55973c59d580 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 10 client.25678 fetch_fsmap finished waiting for FSMap version 45 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:08.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 20 client.25678 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c63f2d0 0x55973c6416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 4 client.25678 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.484+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55973c59f670 con 0x55973c63f2d0 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.485+0000 7fddf27fc640 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c63f2d0 0x55973c6416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.485+0000 7fddd6ff5640 10 client.25678 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.485+0000 7fddf27fc640 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c63f2d0 0x55973c6416f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881437502e284440 server_cookie=8f4fd1a5c55cea88 in_seq=0 out_seq=0 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.485+0000 7fddd6ff5640 1 -- 172.21.15.113:0/3142452866 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55973c59f670 con 0x55973c63f2d0 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.485+0000 7fddd6ff5640 10 client.25678 handle_command_reply: tid=0 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.485+0000 7fddfbfcc740 1 client.25678 shutdown 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 msgr2=0x55973c5bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 0x55973c5bfdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddec01dba0 tx=0x7fddec007bb0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 shutdown_connections 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55973c63f2d0 0x55973c6416f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddd6ff5640 0 client.25678 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:08.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55973c5b7a90 0x55973c5bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55973c59dea0 0x55973c5bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 --2- 172.21.15.113:0/3142452866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55973c59d580 0x55973c5bfdb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 >> 172.21.15.113:0/3142452866 conn(0x55973c597cb0 msgr2=0x55973c5a42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 shutdown_connections 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 1 -- 172.21.15.113:0/3142452866 wait complete. 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddfbfcc740 20 client.25678 trim_cache size 0 max 16384 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:18:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "05879fab-789b-4a92-9b0f-595c39fdf236": { 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "05879fab-789b-4a92-9b0f-595c39fdf236", 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:18:08.491 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdddc0a3f20 msgr2=0x7fdddc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddf9306640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdddc0a3f20 0x7fdddc0a63e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fddf410c1d0 tx=0x7fdde8008040 comp rx=0 tx=0).stop 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 msgr2=0x7fddf410b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.486+0000 7fddf9306640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 0x7fddf410b450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdde4036040 tx=0x7fdde402c8a0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 shutdown_connections 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdddc0a3f20 0x7fdddc0a63e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddf410afa0 0x7fddf410b450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddf4106150 0x7fddf410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 --2- 172.21.15.113:0/2904413590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddf4101080 0x7fddf410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 >> 172.21.15.113:0/2904413590 conn(0x7fddf4078000 msgr2=0x7fddf40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 shutdown_connections 2024-04-22T07:18:08.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:08.487+0000 7fddf9306640 1 -- 172.21.15.113:0/2904413590 wait complete. 2024-04-22T07:18:08.495 INFO:tasks.cephfs.filesystem:scrub status for tag:05879fab-789b-4a92-9b0f-595c39fdf236 - {'path': '/', 'tag': '05879fab-789b-4a92-9b0f-595c39fdf236', 'options': 'recursive,force'} 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: pgmap v106: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:18:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:08.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:18:08.835 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:08 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: pgmap v106: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:18:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:08 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: pgmap v106: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:18:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:08 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.967 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:09 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:18:09.967 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:09 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:18:09.967 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:09 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:09.967 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:09 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:10.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:09 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:18:10.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:09 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:18:10.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:09 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:10.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:09 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:10.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:09 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:18:10.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:09 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:18:10.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:09 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:10.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:09 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:10.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:10 smithi133 ceph-mon[31198]: pgmap v107: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-04-22T07:18:11.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:10 smithi154 ceph-mon[31862]: pgmap v107: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-04-22T07:18:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:10 smithi113 ceph-mon[27933]: pgmap v107: 97 pgs: 97 active+clean; 218 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-04-22T07:18:12.882 INFO:tasks.workunit.client.0.smithi113.stderr:+ tar xf doc.tar 2024-04-22T07:18:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:12 smithi133 ceph-mon[31198]: pgmap v108: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 475 op/s 2024-04-22T07:18:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:12 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:13.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:12 smithi154 ceph-mon[31862]: pgmap v108: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 475 op/s 2024-04-22T07:18:13.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:12 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:13.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:12 smithi113 ceph-mon[27933]: pgmap v108: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 475 op/s 2024-04-22T07:18:13.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:12 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:15.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:14 smithi133 ceph-mon[31198]: pgmap v109: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 294 op/s 2024-04-22T07:18:15.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:14 smithi154 ceph-mon[31862]: pgmap v109: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 294 op/s 2024-04-22T07:18:15.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:14 smithi113 ceph-mon[27933]: pgmap v109: 97 pgs: 97 active+clean; 228 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 294 op/s 2024-04-22T07:18:17.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:16 smithi133 ceph-mon[31198]: pgmap v110: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-04-22T07:18:17.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:16 smithi154 ceph-mon[31862]: pgmap v110: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-04-22T07:18:17.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:16 smithi113 ceph-mon[27933]: pgmap v110: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-04-22T07:18:17.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:17.237+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:17.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:17.237+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:18:17.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:17.237+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:18:17.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:17.237+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:18:17.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:17.237+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:18:19.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:18 smithi133 ceph-mon[31198]: pgmap v111: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-04-22T07:18:19.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:18 smithi154 ceph-mon[31862]: pgmap v111: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-04-22T07:18:19.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:18 smithi113 ceph-mon[27933]: pgmap v111: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-04-22T07:18:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:20 smithi133 ceph-mon[31198]: pgmap v112: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-04-22T07:18:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:20 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:18:21.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:20 smithi154 ceph-mon[31862]: pgmap v112: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-04-22T07:18:21.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:20 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:18:21.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:20 smithi113 ceph-mon[27933]: pgmap v112: 97 pgs: 97 active+clean; 234 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-04-22T07:18:21.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:20 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:18:22.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:21 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:18:22.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:21 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:18:22.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:21 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:18:22.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:21 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:18:22.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:21 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:18:22.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:21 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:18:23.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:22 smithi133 ceph-mon[31198]: pgmap v113: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-04-22T07:18:23.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:22 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:23.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:22 smithi154 ceph-mon[31862]: pgmap v113: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-04-22T07:18:23.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:22 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:23.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:22 smithi113 ceph-mon[27933]: pgmap v113: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-04-22T07:18:23.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:22 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:18:24.603 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-04-22T07:18:25.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:24 smithi133 ceph-mon[31198]: pgmap v114: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-04-22T07:18:25.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:24 smithi154 ceph-mon[31862]: pgmap v114: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-04-22T07:18:25.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:24 smithi113 ceph-mon[27933]: pgmap v114: 97 pgs: 97 active+clean; 240 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-04-22T07:18:25.427 INFO:tasks.workunit.client.0.smithi113.stderr:+ tar c linux-4.0.5 2024-04-22T07:18:25.829 INFO:tasks.workunit.client.0.smithi113.stdout:Ok 2024-04-22T07:18:25.830 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo Ok 2024-04-22T07:18:25.831 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T07:18:25.831 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T07:18:27.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:26 smithi133 ceph-mon[31198]: pgmap v115: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 459 op/s 2024-04-22T07:18:27.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:26 smithi154 ceph-mon[31862]: pgmap v115: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 459 op/s 2024-04-22T07:18:27.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:26 smithi113 ceph-mon[27933]: pgmap v115: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 459 op/s 2024-04-22T07:18:29.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:28 smithi133 ceph-mon[31198]: pgmap v116: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-04-22T07:18:29.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:28 smithi154 ceph-mon[31862]: pgmap v116: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-04-22T07:18:29.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:28 smithi113 ceph-mon[27933]: pgmap v116: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-04-22T07:18:31.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:30 smithi133 ceph-mon[31198]: pgmap v117: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-04-22T07:18:31.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:30 smithi154 ceph-mon[31862]: pgmap v117: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-04-22T07:18:31.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:30 smithi113 ceph-mon[27933]: pgmap v117: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 343 op/s 2024-04-22T07:18:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:32.239+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:32.240+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:18:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:32.240+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:18:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:32.240+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:18:32.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:32.240+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:18:33.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:32 smithi133 ceph-mon[31198]: pgmap v118: 97 pgs: 97 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-04-22T07:18:33.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:32 smithi154 ceph-mon[31862]: pgmap v118: 97 pgs: 97 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-04-22T07:18:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:32 smithi113 ceph-mon[27933]: pgmap v118: 97 pgs: 97 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-04-22T07:18:35.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:34 smithi133 ceph-mon[31198]: pgmap v119: 97 pgs: 97 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 459 op/s 2024-04-22T07:18:35.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:34 smithi154 ceph-mon[31862]: pgmap v119: 97 pgs: 97 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 459 op/s 2024-04-22T07:18:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:34 smithi113 ceph-mon[27933]: pgmap v119: 97 pgs: 97 active+clean; 277 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 459 op/s 2024-04-22T07:18:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:36 smithi133 ceph-mon[31198]: pgmap v120: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 613 op/s 2024-04-22T07:18:37.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:36 smithi154 ceph-mon[31862]: pgmap v120: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 613 op/s 2024-04-22T07:18:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:36 smithi113 ceph-mon[27933]: pgmap v120: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 613 op/s 2024-04-22T07:18:38.010 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-04-22T07:18:38.011 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T07:18:38.279 DEBUG:teuthology.parallel:result is None 2024-04-22T07:18:38.280 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T07:18:38.316 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T07:18:38.316 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-22T07:18:38.329 INFO:tasks.quiescer:joining Quiescers 2024-04-22T07:18:38.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:18:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 -- 172.21.15.113:0/4037258515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 msgr2=0x7f9f3010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 --2- 172.21.15.113:0/4037258515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3010dd50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009ca0 tx=0x7f9f2802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 -- 172.21.15.113:0/4037258515 shutdown_connections 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 --2- 172.21.15.113:0/4037258515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3010dd50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 --2- 172.21.15.113:0/4037258515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f30101a60 0x7f9f30105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 --2- 172.21.15.113:0/4037258515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f301010b0 0x7f9f30101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 -- 172.21.15.113:0/4037258515 >> 172.21.15.113:0/4037258515 conn(0x7f9f300771d0 msgr2=0x7f9f300775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 -- 172.21.15.113:0/4037258515 shutdown_connections 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.457+0000 7f9f3866d640 1 -- 172.21.15.113:0/4037258515 wait complete. 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 Processor -- start 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 -- start start 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f301010b0 0x7f9f3019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f301985c0 0x7f9f30198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9f30111220 con 0x7f9f30106180 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9f301110a0 con 0x7f9f301985c0 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f3866d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f301113a0 con 0x7f9f301010b0 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58142/0 (socket says 172.21.15.113:58142) 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 -- 172.21.15.113:0/3746071384 learned_addr learned my addr 172.21.15.113:0/3746071384 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f36c0a640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f301985c0 0x7f9f30198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f36409640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f301010b0 0x7f9f3019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 -- 172.21.15.113:0/3746071384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f301010b0 msgr2=0x7f9f3019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f301010b0 0x7f9f3019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 -- 172.21.15.113:0/3746071384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f301985c0 msgr2=0x7f9f30198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.458+0000 7f9f35c08640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f301985c0 0x7f9f30198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f35c08640 1 -- 172.21.15.113:0/3746071384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f301990e0 con 0x7f9f30106180 2024-04-22T07:18:38.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f35c08640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3019e950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000d770 tx=0x7f9f2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20017070 con 0x7f9f30106180 2024-04-22T07:18:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f20009c30 con 0x7f9f30106180 2024-04-22T07:18:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f301993d0 con 0x7f9f30106180 2024-04-22T07:18:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000f620 con 0x7f9f30106180 2024-04-22T07:18:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.459+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f301a5620 con 0x7f9f30106180 2024-04-22T07:18:38.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.460+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f30069230 con 0x7f9f30106180 2024-04-22T07:18:38.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.461+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9f2001a070 con 0x7f9f30106180 2024-04-22T07:18:38.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.461+0000 7f9f277fe640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9f1c0a3d20 0x7f9f1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.462+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f9f200d0280 con 0x7f9f30106180 2024-04-22T07:18:38.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.462+0000 7f9f36409640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9f1c0a3d20 0x7f9f1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.462+0000 7f9f36409640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9f1c0a3d20 0x7f9f1c0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bd0 tx=0x7f9f2c005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.465+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f20009de0 con 0x7f9f30106180 2024-04-22T07:18:38.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:18:38.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.620+0000 7fe115fe0640 1 -- 172.21.15.113:0/1353401457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe110100fb0 msgr2=0x7fe110101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.620+0000 7fe115fe0640 1 --2- 172.21.15.113:0/1353401457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe110100fb0 0x7fe110101390 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe100009890 tx=0x7fe10002bc00 comp rx=0 tx=0).stop 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 -- 172.21.15.113:0/1353401457 shutdown_connections 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 --2- 172.21.15.113:0/1353401457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1101063b0 0x7fe110106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 --2- 172.21.15.113:0/1353401457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101018d0 0x7fe110105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 --2- 172.21.15.113:0/1353401457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe110100fb0 0x7fe110101390 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 -- 172.21.15.113:0/1353401457 >> 172.21.15.113:0/1353401457 conn(0x7fe110078000 msgr2=0x7fe110078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 -- 172.21.15.113:0/1353401457 shutdown_connections 2024-04-22T07:18:38.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 -- 172.21.15.113:0/1353401457 wait complete. 2024-04-22T07:18:38.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.621+0000 7fe115fe0640 1 Processor -- start 2024-04-22T07:18:38.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 -- start start 2024-04-22T07:18:38.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe110100fb0 0x7fe11019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101018d0 0x7fe11019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101063b0 0x7fe1101a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101188e0 con 0x7fe1101018d0 2024-04-22T07:18:38.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe110118760 con 0x7fe110100fb0 2024-04-22T07:18:38.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe115fe0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe110118a60 con 0x7fe1101063b0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe110100fb0 0x7fe11019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe110100fb0 0x7fe11019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57228/0 (socket says 172.21.15.113:57228) 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe10f7fe640 1 -- 172.21.15.113:0/3144993595 learned_addr learned my addr 172.21.15.113:0/3144993595 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101063b0 0x7fe1101a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.622+0000 7fe10effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101018d0 0x7fe11019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.623+0000 7fe10ffff640 1 -- 172.21.15.113:0/3144993595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe110100fb0 msgr2=0x7fe11019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.623+0000 7fe10ffff640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe110100fb0 0x7fe11019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.623+0000 7fe10ffff640 1 -- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101018d0 msgr2=0x7fe11019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.623+0000 7fe10ffff640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101018d0 0x7fe11019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.623+0000 7fe10ffff640 1 -- 172.21.15.113:0/3144993595 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1101a3e10 con 0x7fe1101063b0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.624+0000 7fe10ffff640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101063b0 0x7fe1101a36e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe10800ee70 tx=0x7fe10800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.624+0000 7fe10cff9640 1 -- 172.21.15.113:0/3144993595 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080099a0 con 0x7fe1101063b0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.624+0000 7fe10cff9640 1 -- 172.21.15.113:0/3144993595 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe108003970 con 0x7fe1101063b0 2024-04-22T07:18:38.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.624+0000 7fe10cff9640 1 -- 172.21.15.113:0/3144993595 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10800b570 con 0x7fe1101063b0 2024-04-22T07:18:38.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.624+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1101a4100 con 0x7fe1101063b0 2024-04-22T07:18:38.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.624+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1101ab990 con 0x7fe1101063b0 2024-04-22T07:18:38.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.627+0000 7fe10cff9640 1 -- 172.21.15.113:0/3144993595 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe10801d020 con 0x7fe1101063b0 2024-04-22T07:18:38.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.627+0000 7fe118bcd740 1 Processor -- start 2024-04-22T07:18:38.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.627+0000 7fe118bcd740 1 -- start start 2024-04-22T07:18:38.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 0x55e2f874a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f874aea0 0x55e2f8753540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f8755e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe118bcd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e2f86af1e0 con 0x55e2f8753a80 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe118bcd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e2f864c390 con 0x55e2f874a580 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe118bcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e2f86ab1a0 con 0x55e2f874aea0 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe10cff9640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe0f80a3da0 0x7fe0f80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe10effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 0x55e2f874a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe10ffff640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe0f80a3da0 0x7fe0f80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe10effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 0x55e2f874a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57230/0 (socket says 172.21.15.113:57230) 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe10effd640 1 -- 172.21.15.113:0/471536798 learned_addr learned my addr 172.21.15.113:0/471536798 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.628+0000 7fe10effd640 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f8755e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10effd640 1 -- 172.21.15.113:0/471536798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f874aea0 msgr2=0x55e2f8753540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:18:38.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10effd640 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f874aea0 0x55e2f8753540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10effd640 1 -- 172.21.15.113:0/471536798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 msgr2=0x55e2f8755e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10effd640 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f8755e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10effd640 1 -- 172.21.15.113:0/471536798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f848b7d0 con 0x55e2f874a580 2024-04-22T07:18:38.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10cff9640 1 -- 172.21.15.113:0/3144993595 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fe1080d0690 con 0x7fe1101063b0 2024-04-22T07:18:38.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10effd640 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 0x55e2f874a960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe104011490 tx=0x7fe104005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c28ce5a25656b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe10ffff640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe0f80a3da0 0x7fe0f80a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cdd10 tx=0x7fe108076f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe0f5ffb640 1 -- 172.21.15.113:0/471536798 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104004030 con 0x55e2f874a580 2024-04-22T07:18:38.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe0f5ffb640 1 -- 172.21.15.113:0/471536798 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe1040054f0 con 0x55e2f874a580 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.629+0000 7fe0f5ffb640 1 -- 172.21.15.113:0/471536798 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104004d60 con 0x55e2f874a580 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 -- 172.21.15.113:0/471536798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 msgr2=0x55e2f874a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 0x55e2f874a960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe104011490 tx=0x7fe104005850 comp rx=0 tx=0).stop 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 -- 172.21.15.113:0/471536798 shutdown_connections 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f8755e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f874aea0 0x55e2f8753540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 --2- 172.21.15.113:0/471536798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874a580 0x55e2f874a960 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe104011490 tx=0x7fe104005850 comp rx=0 tx=0).stop 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.630+0000 7fe118bcd740 1 -- 172.21.15.113:0/471536798 >> 172.21.15.113:0/471536798 conn(0x55e2f8744cb0 msgr2=0x55e2f8745090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.631+0000 7fe118bcd740 1 -- 172.21.15.113:0/471536798 shutdown_connections 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.631+0000 7fe118bcd740 1 -- 172.21.15.113:0/471536798 wait complete. 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.631+0000 7fe118bcd740 1 -- start start 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 Processor -- start 2024-04-22T07:18:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f8750ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f87511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f875fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e2f8756870 con 0x55e2f8753a80 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e2f86682f0 con 0x55e2f874aea0 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e2f87569f0 con 0x55e2f8751770 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe118bcd740 10 client.0 _ll_register_callbacks cb 0x55e2f8675580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe10effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f8750ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe10effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f875fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe10effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f8750ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57236/0 (socket says 172.21.15.113:57236) 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe10effd640 1 -- 172.21.15.113:0/1435364454 learned_addr learned my addr 172.21.15.113:0/1435364454 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.632+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f87511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe10effd640 1 -- 172.21.15.113:0/1435364454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 msgr2=0x55e2f875fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe10effd640 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f875fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe10effd640 1 -- 172.21.15.113:0/1435364454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 msgr2=0x55e2f87511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe10effd640 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f87511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe10effd640 1 -- 172.21.15.113:0/1435364454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f83c13f0 con 0x55e2f874aea0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe10effd640 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f8750ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe104004a40 tx=0x7fe104005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf29dfcd0fc7c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.633+0000 7fe0eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1435364454 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104025030 con 0x55e2f874aea0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1435364454 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fe104004230 con 0x55e2f874aea0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1040054f0 con 0x55e2f874aea0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1435364454 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe104034050 con 0x55e2f874aea0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe118bcd740 10 client.25850 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e2f83810f0 con 0x55e2f874aea0 2024-04-22T07:18:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.634+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1435364454 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe104004430 con 0x55e2f874aea0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1435364454 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7fe10401e420 con 0x55e2f874aea0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe118bcd740 10 client.25850 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe118bcd740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe118bcd740 20 client.25850 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f8763ff0 0x55e2f87663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe118bcd740 4 client.25850 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1040054f0 con 0x55e2f8763ff0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.635+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f8763ff0 0x55e2f87663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.636+0000 7fe0eeffd640 10 client.25850 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.636+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f8763ff0 0x55e2f87663e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd20820ccdf6148a server_cookie=f7fa9d89ed12f864 in_seq=0 out_seq=0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.636+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1435364454 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fe1040054f0 con 0x55e2f8763ff0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.636+0000 7fe0eeffd640 10 client.25850 handle_command_reply: tid=0 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.636+0000 7fe118bcd740 1 client.25850 shutdown 2024-04-22T07:18:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 msgr2=0x55e2f8750ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f8750ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe104004a40 tx=0x7fe104005f90 comp rx=0 tx=0).stop 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 shutdown_connections 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f8763ff0 0x55e2f87663e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f875fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8753a80 0x55e2f87511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1435364454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f8750ca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 >> 172.21.15.113:0/1435364454 conn(0x55e2f8744cb0 msgr2=0x55e2f8746a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe0eeffd640 0 client.25850 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 shutdown_connections 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 1 -- 172.21.15.113:0/1435364454 wait complete. 2024-04-22T07:18:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.637+0000 7fe118bcd740 20 client.25850 trim_cache size 0 max 16384 2024-04-22T07:18:38.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.664+0000 7fe118bcd740 1 Processor -- start 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.664+0000 7fe118bcd740 1 -- start start 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.664+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f874aea0 0x55e2f87466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.664+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f8746c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f8753a80 0x55e2f875f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe118bcd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101b7020 con 0x55e2f874aea0 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe118bcd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101b4890 con 0x55e2f8753a80 2024-04-22T07:18:38.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe118bcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101b26e0 con 0x55e2f8751770 2024-04-22T07:18:38.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f8746c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f8746c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:45386/0 (socket says 172.21.15.113:45386) 2024-04-22T07:18:38.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.665+0000 7fe10f7fe640 1 -- 172.21.15.113:0/2525257650 learned_addr learned my addr 172.21.15.113:0/2525257650 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10effd640 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f8753a80 0x55e2f875f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10effd640 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f874aea0 0x55e2f87466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10f7fe640 1 -- 172.21.15.113:0/2525257650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f8753a80 msgr2=0x55e2f875f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f8753a80 0x55e2f875f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10f7fe640 1 -- 172.21.15.113:0/2525257650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f874aea0 msgr2=0x55e2f87466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f874aea0 0x55e2f87466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10f7fe640 1 -- 172.21.15.113:0/2525257650 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f874f230 con 0x55e2f8751770 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10effd640 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f8753a80 0x55e2f875f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.666+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f8746c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe10002c210 tx=0x7fe1000056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba050a0fd99de4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.667+0000 7fe0f57fa640 1 -- 172.21.15.113:0/2525257650 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100045e60 con 0x55e2f8751770 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.667+0000 7fe0f57fa640 1 -- 172.21.15.113:0/2525257650 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe10004a070 con 0x55e2f8751770 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.667+0000 7fe0f57fa640 1 -- 172.21.15.113:0/2525257650 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100039040 con 0x55e2f8751770 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 -- 172.21.15.113:0/2525257650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 msgr2=0x55e2f8746c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f8746c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe10002c210 tx=0x7fe1000056d0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 -- 172.21.15.113:0/2525257650 shutdown_connections 2024-04-22T07:18:38.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f8753a80 0x55e2f875f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8751770 0x55e2f8746c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 --2- 172.21.15.113:0/2525257650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f874aea0 0x55e2f87466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 -- 172.21.15.113:0/2525257650 >> 172.21.15.113:0/2525257650 conn(0x55e2f8744cb0 msgr2=0x55e2f8745fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 -- 172.21.15.113:0/2525257650 shutdown_connections 2024-04-22T07:18:38.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.668+0000 7fe118bcd740 1 -- 172.21.15.113:0/2525257650 wait complete. 2024-04-22T07:18:38.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.669+0000 7fe118bcd740 1 -- start start 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.669+0000 7fe118bcd740 1 Processor -- start 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.669+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f875bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8751770 0x55e2f875c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe118bcd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8753a80 0x55e2f875a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe118bcd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101b2d50 con 0x55e2f8751770 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe118bcd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101b41c0 con 0x55e2f874aea0 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe118bcd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101b3420 con 0x55e2f8753a80 2024-04-22T07:18:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe118bcd740 10 client.0 _ll_register_callbacks cb 0x7fe104009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f875bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8751770 0x55e2f875c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f875bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57244/0 (socket says 172.21.15.113:57244) 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10effd640 1 -- 172.21.15.113:0/1887449270 learned_addr learned my addr 172.21.15.113:0/1887449270 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10effd640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8753a80 0x55e2f875a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10f7fe640 1 -- 172.21.15.113:0/1887449270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8753a80 msgr2=0x55e2f875a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8753a80 0x55e2f875a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10f7fe640 1 -- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 msgr2=0x55e2f875bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f875bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.670+0000 7fe10f7fe640 1 -- 172.21.15.113:0/1887449270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f875b1b0 con 0x55e2f8751770 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.671+0000 7fe10effd640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8753a80 0x55e2f875a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.671+0000 7fe10f7fe640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8751770 0x55e2f875c190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe1000095f0 tx=0x7fe100005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b7f7fd1717f243 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.671+0000 7fe0eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.671+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1887449270 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100039040 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.671+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1887449270 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fe10002ca80 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.671+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10004a070 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.672+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1887449270 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe100045e60 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.672+0000 7fe118bcd740 10 client.16443 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.672+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e2f87c9fb0 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1887449270 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe10004a070 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1887449270 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7fe100058030 con 0x55e2f8751770 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe118bcd740 10 client.16443 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe118bcd740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe118bcd740 20 client.16443 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f87ca710 0x55e2f87ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe118bcd740 4 client.16443 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2f86bcc00 con 0x55e2f87ca710 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe10effd640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f87ca710 0x55e2f87ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.673+0000 7fe0eeffd640 10 client.16443 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe10effd640 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f87ca710 0x55e2f87ccb30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08876069bdad7c5 server_cookie=24045fa8a63addf in_seq=0 out_seq=0 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe0eeffd640 1 -- 172.21.15.113:0/1887449270 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e2f86bcc00 con 0x55e2f87ca710 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe0eeffd640 10 client.16443 handle_command_reply: tid=0 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 client.16443 shutdown 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8751770 msgr2=0x55e2f875c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8751770 0x55e2f875c190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe1000095f0 tx=0x7fe100005b20 comp rx=0 tx=0).stop 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 shutdown_connections 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e2f87ca710 0x55e2f87ccb30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e2f8753a80 0x55e2f875a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e2f8751770 0x55e2f875c190 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe0eeffd640 0 client.16443 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 --2- 172.21.15.113:0/1887449270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2f874aea0 0x55e2f875bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 >> 172.21.15.113:0/1887449270 conn(0x55e2f8744cb0 msgr2=0x55e2f8746840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 shutdown_connections 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 1 -- 172.21.15.113:0/1887449270 wait complete. 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.674+0000 7fe118bcd740 20 client.16443 trim_cache size 0 max 16384 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:18:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe0f80a3da0 msgr2=0x7fe0f80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe0f80a3da0 0x7fe0f80a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cdd10 tx=0x7fe108076f90 comp rx=0 tx=0).stop 2024-04-22T07:18:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101063b0 msgr2=0x7fe1101a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101063b0 0x7fe1101a36e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe10800ee70 tx=0x7fe10800c740 comp rx=0 tx=0).stop 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 shutdown_connections 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe0f80a3da0 0x7fe0f80a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1101063b0 0x7fe1101a36e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101018d0 0x7fe11019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 --2- 172.21.15.113:0/3144993595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe110100fb0 0x7fe11019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 >> 172.21.15.113:0/3144993595 conn(0x7fe110078000 msgr2=0x7fe110103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 shutdown_connections 2024-04-22T07:18:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.675+0000 7fe115fe0640 1 -- 172.21.15.113:0/3144993595 wait complete. 2024-04-22T07:18:38.684 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:18:38.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:18:38.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.734+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f30199af0 con 0x7f9f30106180 2024-04-22T07:18:38.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.735+0000 7f9f277fe640 1 -- 172.21.15.113:0/3746071384 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11894 (secure 0 0 0) 0x7f9f30199af0 con 0x7f9f30106180 2024-04-22T07:18:38.733 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:18:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:18:21.107438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:18:38.733 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9f1c0a3d20 msgr2=0x7f9f1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9f1c0a3d20 0x7f9f1c0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bd0 tx=0x7f9f2c005f90 comp rx=0 tx=0).stop 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 msgr2=0x7f9f3019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3019e950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000d770 tx=0x7f9f2000dc30 comp rx=0 tx=0).stop 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 shutdown_connections 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9f1c0a3d20 0x7f9f1c0a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f301985c0 0x7f9f30198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f30106180 0x7f9f3019e950 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 --2- 172.21.15.113:0/3746071384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f301010b0 0x7f9f3019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 >> 172.21.15.113:0/3746071384 conn(0x7f9f300771d0 msgr2=0x7f9f3006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.737+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 shutdown_connections 2024-04-22T07:18:38.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.738+0000 7f9f3866d640 1 -- 172.21.15.113:0/3746071384 wait complete. 2024-04-22T07:18:38.747 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24446 quiesce db --cancel --all' 2024-04-22T07:18:38.747 DEBUG:teuthology.orchestra.run.smithi113:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24446 quiesce db --cancel --all 2024-04-22T07:18:38.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 -- 172.21.15.113:0/2148066658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118105e30 msgr2=0x7f0118106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 --2- 172.21.15.113:0/2148066658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118105e30 0x7f0118106210 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0108009930 tx=0x7f010802bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 -- 172.21.15.113:0/2148066658 shutdown_connections 2024-04-22T07:18:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 --2- 172.21.15.113:0/2148066658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0118101b60 0x7f011810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 --2- 172.21.15.113:0/2148066658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01181011a0 0x7f0118101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 --2- 172.21.15.113:0/2148066658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118105e30 0x7f0118106210 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 -- 172.21.15.113:0/2148066658 >> 172.21.15.113:0/2148066658 conn(0x7f0118077e30 msgr2=0x7f0118078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 -- 172.21.15.113:0/2148066658 shutdown_connections 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.787+0000 7f011c8ba640 1 -- 172.21.15.113:0/2148066658 wait complete. 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 Processor -- start 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 -- start start 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 0x7f0118197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0118101b60 0x7f0118197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0118105e30 0x7f011819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0118111240 con 0x7f01181011a0 2024-04-22T07:18:38.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01181110c0 con 0x7f0118101b60 2024-04-22T07:18:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011c8ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f01181113c0 con 0x7f0118105e30 2024-04-22T07:18:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 0x7f0118197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 0x7f0118197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58224/0 (socket says 172.21.15.113:58224) 2024-04-22T07:18:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 -- 172.21.15.113:0/1505621373 learned_addr learned my addr 172.21.15.113:0/1505621373 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f0116d9d640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0118105e30 0x7f011819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f010ffff640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0118101b60 0x7f0118197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 -- 172.21.15.113:0/1505621373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0118105e30 msgr2=0x7f011819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0118105e30 0x7f011819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 -- 172.21.15.113:0/1505621373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0118101b60 msgr2=0x7f0118197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0118101b60 0x7f0118197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 -- 172.21.15.113:0/1505621373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011819c700 con 0x7f01181011a0 2024-04-22T07:18:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.788+0000 7f011659c640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 0x7f0118197760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0108009230 tx=0x7f010802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.789+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108043020 con 0x7f01181011a0 2024-04-22T07:18:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.789+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011819c990 con 0x7f01181011a0 2024-04-22T07:18:38.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.789+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0108004350 con 0x7f01181011a0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.789+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108003ba0 con 0x7f01181011a0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.789+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01181a4280 con 0x7f01181011a0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.790+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0118068de0 con 0x7f01181011a0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.791+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0108046020 con 0x7f01181011a0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.791+0000 7f010f7fe640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f01040a3d20 0x7f01040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.791+0000 7f010ffff640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f01040a3d20 0x7f01040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.791+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f01080f22c0 con 0x7f01181011a0 2024-04-22T07:18:38.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.792+0000 7f010ffff640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f01040a3d20 0x7f01040a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0118100d60 tx=0x7f010000b410 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.796+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f010804d070 con 0x7f01181011a0 2024-04-22T07:18:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 -- 172.21.15.113:0/1448318327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90106160 msgr2=0x7f2d9010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1448318327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90106160 0x7f2d9010dd30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009be0 tx=0x7f2d8802bef0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 -- 172.21.15.113:0/1448318327 shutdown_connections 2024-04-22T07:18:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1448318327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90106160 0x7f2d9010dd30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1448318327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d90101a40 0x7f2d90105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1448318327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90101090 0x7f2d90101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 -- 172.21.15.113:0/1448318327 >> 172.21.15.113:0/1448318327 conn(0x7f2d90077f50 msgr2=0x7f2d90078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 -- 172.21.15.113:0/1448318327 shutdown_connections 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.855+0000 7f2d94bef640 1 -- 172.21.15.113:0/1448318327 wait complete. 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 Processor -- start 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 -- start start 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90101090 0x7f2d901996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 0x7f2d90199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 0x7f2d901a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901111e0 con 0x7f2d90106160 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90111060 con 0x7f2d90101090 2024-04-22T07:18:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.856+0000 7f2d94bef640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90111360 con 0x7f2d9019a120 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 0x7f2d901a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 0x7f2d90199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 0x7f2d901a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:45422/0 (socket says 172.21.15.113:45422) 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 -- 172.21.15.113:0/1149616676 learned_addr learned my addr 172.21.15.113:0/1149616676 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 0x7f2d90199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58228/0 (socket says 172.21.15.113:58228) 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90101090 0x7f2d901996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 -- 172.21.15.113:0/1149616676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90101090 msgr2=0x7f2d901996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90101090 0x7f2d901996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 -- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 msgr2=0x7f2d90199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 0x7f2d90199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 -- 172.21.15.113:0/1149616676 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901a4ea0 con 0x7f2d9019a120 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 0x7f2d90199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d8ed9d640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 0x7f2d901a4720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d880096f0 tx=0x7f2d8802cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d7f7fe640 1 -- 172.21.15.113:0/1149616676 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88043020 con 0x7f2d9019a120 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901a5130 con 0x7f2d9019a120 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d7f7fe640 1 -- 172.21.15.113:0/1149616676 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2d88003e40 con 0x7f2d9019a120 2024-04-22T07:18:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d7f7fe640 1 -- 172.21.15.113:0/1149616676 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8803e6b0 con 0x7f2d9019a120 2024-04-22T07:18:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.857+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d901a56d0 con 0x7f2d9019a120 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.859+0000 7f2d7f7fe640 1 -- 172.21.15.113:0/1149616676 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2d88046020 con 0x7f2d9019a120 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 Processor -- start 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 -- start start 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d3d10 0x559e5f0d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0dcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e5f039000 con 0x559e5f0dd240 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559e5efd5b20 con 0x559e5f0d46c0 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d977d6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559e5f034fc0 con 0x559e5f0d3d10 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0dcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0dcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57282/0 (socket says 172.21.15.113:57282) 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 -- 172.21.15.113:0/3395647948 learned_addr learned my addr 172.21.15.113:0/3395647948 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d3d10 0x559e5f0d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d7f7fe640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2d780a3da0 0x7f2d780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8ed9d640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2d780a3da0 0x7f2d780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 -- 172.21.15.113:0/3395647948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d3d10 msgr2=0x559e5f0d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d3d10 0x559e5f0d40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 -- 172.21.15.113:0/3395647948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 msgr2=0x559e5f0df690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.860+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0df690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d8e59c640 1 -- 172.21.15.113:0/3395647948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5efefea0 con 0x559e5f0d46c0 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d7f7fe640 1 -- 172.21.15.113:0/1149616676 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2d880f3060 con 0x7f2d9019a120 2024-04-22T07:18:38.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0dcc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009c40 tx=0x7f2d840117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e646699318647bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d8ed9d640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2d780a3da0 0x7f2d780a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d880beec0 tx=0x7f2d880b43c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d7cff9640 1 -- 172.21.15.113:0/3395647948 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84012450 con 0x559e5f0d46c0 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d7cff9640 1 -- 172.21.15.113:0/3395647948 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2d84012ac0 con 0x559e5f0d46c0 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.861+0000 7f2d7cff9640 1 -- 172.21.15.113:0/3395647948 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400fb60 con 0x559e5f0d46c0 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- 172.21.15.113:0/3395647948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 msgr2=0x559e5f0dcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0dcc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009c40 tx=0x7f2d840117b0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- 172.21.15.113:0/3395647948 shutdown_connections 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0dcc70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- 172.21.15.113:0/3395647948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d3d10 0x559e5f0d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- 172.21.15.113:0/3395647948 >> 172.21.15.113:0/3395647948 conn(0x559e5f0ce3e0 msgr2=0x559e5f0ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- 172.21.15.113:0/3395647948 shutdown_connections 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- 172.21.15.113:0/3395647948 wait complete. 2024-04-22T07:18:38.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- start start 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 Processor -- start 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e5f0e00c0 con 0x559e5f0d3d10 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559e5efd3510 con 0x559e5f0d46c0 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.862+0000 7f2d977d6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559e5f0e0240 con 0x559e5f0dd240 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d977d6740 10 client.0 _ll_register_callbacks cb 0x559e5efd2f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0da1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58242/0 (socket says 172.21.15.113:58242) 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/592300690 learned_addr learned my addr 172.21.15.113:0/592300690 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/592300690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 msgr2=0x559e5f0dac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0dac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 msgr2=0x559e5f0da710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0da710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/592300690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5ee17a20 con 0x559e5f0d3d10 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0da1d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000f440 tx=0x7f2d80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cee4282e91c3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d75ffb640 1 -- 172.21.15.113:0/592300690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000fe80 con 0x559e5f0d3d10 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d75ffb640 1 -- 172.21.15.113:0/592300690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2d80004020 con 0x559e5f0d3d10 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d84012ac0 con 0x559e5f0d3d10 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d75ffb640 1 -- 172.21.15.113:0/592300690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2d80004220 con 0x559e5f0d3d10 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d977d6740 10 client.16461 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559e5ee781b0 con 0x559e5f0d3d10 2024-04-22T07:18:38.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.863+0000 7f2d75ffb640 1 -- 172.21.15.113:0/592300690 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2d84012ac0 con 0x559e5f0d3d10 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d75ffb640 1 -- 172.21.15.113:0/592300690 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f2d80004b40 con 0x559e5f0d3d10 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d977d6740 10 client.16461 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d977d6740 10 client.16461 resolve_mds: validated gid 24446 aka daemon mds.e 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d977d6740 20 client.16461 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d977d6740 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f0ed700 0x559e5f0efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d977d6740 4 client.16461 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e5f046a20 con 0x559e5f0ed700 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f0ed700 0x559e5f0efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d75ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.864+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f0ed700 0x559e5f0efb40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa4b7117d4e5052 server_cookie=bfa06527f8238750 in_seq=0 out_seq=0 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d75ffb640 1 -- 172.21.15.113:0/592300690 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e5f046a20 con 0x559e5f0ed700 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d75ffb640 10 client.16461 handle_command_reply: tid=0 2024-04-22T07:18:38.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 client.16461 shutdown 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 msgr2=0x559e5f0da1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0da1d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000f440 tx=0x7f2d80004730 comp rx=0 tx=0).stop 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 shutdown_connections 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f0ed700 0x559e5f0efb40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0da710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d75ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 --2- 172.21.15.113:0/592300690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0da1d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 >> 172.21.15.113:0/592300690 conn(0x559e5f0ce3e0 msgr2=0x559e5f0cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 shutdown_connections 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 1 -- 172.21.15.113:0/592300690 wait complete. 2024-04-22T07:18:38.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.865+0000 7f2d977d6740 20 client.16461 trim_cache size 0 max 16384 2024-04-22T07:18:38.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 Processor -- start 2024-04-22T07:18:38.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 -- start start 2024-04-22T07:18:38.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d3d10 0x559e5f0e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d46c0 0x559e5f0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901b0800 con 0x559e5f0dd240 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901ae070 con 0x559e5f0d3d10 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.882+0000 7f2d977d6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901abee0 con 0x559e5f0d46c0 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58258/0 (socket says 172.21.15.113:58258) 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/938395668 learned_addr learned my addr 172.21.15.113:0/938395668 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d3d10 0x559e5f0e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d46c0 0x559e5f0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/938395668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d46c0 msgr2=0x559e5f0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d46c0 0x559e5f0cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/938395668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d3d10 msgr2=0x559e5f0e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d3d10 0x559e5f0e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/938395668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5f0d4460 con 0x559e5f0dd240 2024-04-22T07:18:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0d8aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2d80012bb0 tx=0x7f2d80012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa645b91bc0315e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d77fff640 1 -- 172.21.15.113:0/938395668 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000fe80 con 0x559e5f0dd240 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d77fff640 1 -- 172.21.15.113:0/938395668 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2d80031020 con 0x559e5f0dd240 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.883+0000 7f2d77fff640 1 -- 172.21.15.113:0/938395668 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80003b30 con 0x559e5f0dd240 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 -- 172.21.15.113:0/938395668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 msgr2=0x559e5f0d8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0d8aa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2d80012bb0 tx=0x7f2d80012500 comp rx=0 tx=0).stop 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 -- 172.21.15.113:0/938395668 shutdown_connections 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0dd240 0x559e5f0d8aa0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0d46c0 0x559e5f0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 --2- 172.21.15.113:0/938395668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d3d10 0x559e5f0e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 -- 172.21.15.113:0/938395668 >> 172.21.15.113:0/938395668 conn(0x559e5f0ce3e0 msgr2=0x559e5f0daa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 -- 172.21.15.113:0/938395668 shutdown_connections 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 -- 172.21.15.113:0/938395668 wait complete. 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 -- start start 2024-04-22T07:18:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.884+0000 7f2d977d6740 1 Processor -- start 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901ac530 con 0x559e5f0d3d10 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901ad9a0 con 0x559e5f0d46c0 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901acc00 con 0x559e5f0dd240 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 10 client.0 _ll_register_callbacks cb 0x7f2d80012c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0e5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58272/0 (socket says 172.21.15.113:58272) 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/1046015497 learned_addr learned my addr 172.21.15.113:0/1046015497 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/1046015497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 msgr2=0x559e5f0e2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0e2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 msgr2=0x559e5f0e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 -- 172.21.15.113:0/1046015497 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5ed0b310 con 0x559e5f0d3d10 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d8dd9b640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0e5550 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2d80000c00 tx=0x7f2d8000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312810d7363398de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d75ffb640 1 -- 172.21.15.113:0/1046015497 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80003cc0 con 0x559e5f0d3d10 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d75ffb640 1 -- 172.21.15.113:0/1046015497 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2d80032040 con 0x559e5f0d3d10 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.885+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d80031020 con 0x559e5f0d3d10 2024-04-22T07:18:38.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d75ffb640 1 -- 172.21.15.113:0/1046015497 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2d80003e60 con 0x559e5f0d3d10 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 10 client.16467 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559e5f1545d0 con 0x559e5f0d3d10 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d75ffb640 1 -- 172.21.15.113:0/1046015497 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2d80031020 con 0x559e5f0d3d10 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d75ffb640 1 -- 172.21.15.113:0/1046015497 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f2d80005120 con 0x559e5f0d3d10 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 10 client.16467 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 10 client.16467 resolve_mds: validated gid 24446 aka daemon mds.e 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 20 client.16467 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f154d00 0x559e5f157120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 4 client.16467 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x559e5f040890 con 0x559e5f154d00 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f154d00 0x559e5f157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d75ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.886+0000 7f2d8e59c640 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f154d00 0x559e5f157120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2aeb85b6eda8f5d server_cookie=d62e983c4162632d in_seq=0 out_seq=0 2024-04-22T07:18:38.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d75ffb640 1 -- 172.21.15.113:0/1046015497 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x559e5f040890 con 0x559e5f154d00 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d75ffb640 10 client.16467 handle_command_reply: tid=0 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 client.16467 shutdown 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 msgr2=0x559e5f0e5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0e5550 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2d80000c00 tx=0x7f2d8000bfb0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 shutdown_connections 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559e5f154d00 0x559e5f157120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559e5f0dd240 0x559e5f0e2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d75ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e5f0d46c0 0x559e5f0e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 --2- 172.21.15.113:0/1046015497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e5f0d3d10 0x559e5f0e5550 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 >> 172.21.15.113:0/1046015497 conn(0x559e5f0ce3e0 msgr2=0x559e5f0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 shutdown_connections 2024-04-22T07:18:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 1 -- 172.21.15.113:0/1046015497 wait complete. 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d977d6740 20 client.16467 trim_cache size 0 max 16384 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stdout: "epoch": 47, 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stdout: "leader": 24446, 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stdout: "set_version": 0, 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stdout: "sets": {} 2024-04-22T07:18:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2d780a3da0 msgr2=0x7f2d780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2d780a3da0 0x7f2d780a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d880beec0 tx=0x7f2d880b43c0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 msgr2=0x7f2d901a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.887+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 0x7f2d901a4720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d880096f0 tx=0x7f2d8802cbc0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 shutdown_connections 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2d780a3da0 0x7f2d780a6260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d9019a120 0x7f2d901a4720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d90106160 0x7f2d90199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 --2- 172.21.15.113:0/1149616676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d90101090 0x7f2d901996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 >> 172.21.15.113:0/1149616676 conn(0x7f2d90077f50 msgr2=0x7f2d900feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 shutdown_connections 2024-04-22T07:18:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:38.888+0000 7f2d94bef640 1 -- 172.21.15.113:0/1149616676 wait complete. 2024-04-22T07:18:39.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.019+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0118109330 con 0x7f01181011a0 2024-04-22T07:18:39.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.020+0000 7f010f7fe640 1 -- 172.21.15.113:0/1505621373 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11894 (secure 0 0 0) 0x7f01080be8f0 con 0x7f01181011a0 2024-04-22T07:18:39.018 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:18:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:18:21.107438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:18:39.018 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f01040a3d20 msgr2=0x7f01040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f01040a3d20 0x7f01040a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0118100d60 tx=0x7f010000b410 comp rx=0 tx=0).stop 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 msgr2=0x7f0118197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 0x7f0118197760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0108009230 tx=0x7f010802c6e0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 shutdown_connections 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f01040a3d20 0x7f01040a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0118105e30 0x7f011819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0118101b60 0x7f0118197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 --2- 172.21.15.113:0/1505621373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01181011a0 0x7f0118197760 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 >> 172.21.15.113:0/1505621373 conn(0x7f0118077e30 msgr2=0x7f01180feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 shutdown_connections 2024-04-22T07:18:39.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.022+0000 7f011c8ba640 1 -- 172.21.15.113:0/1505621373 wait complete. 2024-04-22T07:18:39.030 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:38 smithi113 ceph-mon[27933]: pgmap v121: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-04-22T07:18:39.032 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:18:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:38 smithi133 ceph-mon[31198]: pgmap v121: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-04-22T07:18:39.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:38 smithi154 ceph-mon[31862]: pgmap v121: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-04-22T07:18:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 -- 172.21.15.113:0/2877041888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 msgr2=0x7f208010dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 --2- 172.21.15.113:0/2877041888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f208010dc80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2078009cd0 tx=0x7f207802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 -- 172.21.15.113:0/2877041888 shutdown_connections 2024-04-22T07:18:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 --2- 172.21.15.113:0/2877041888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f208010dc80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 --2- 172.21.15.113:0/2877041888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20801011a0 0x7f2080101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 --2- 172.21.15.113:0/2877041888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2080107030 0x7f2080107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 -- 172.21.15.113:0/2877041888 >> 172.21.15.113:0/2877041888 conn(0x7f2080076c50 msgr2=0x7f2080077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 -- 172.21.15.113:0/2877041888 shutdown_connections 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.135+0000 7f2088670640 1 -- 172.21.15.113:0/2877041888 wait complete. 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 Processor -- start 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 -- start start 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20801011a0 0x7f20801995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f2080199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080107030 0x7f20801a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2080111260 con 0x7f2080101b40 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f20801110e0 con 0x7f20801011a0 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2088670640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f20801113e0 con 0x7f2080107030 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2085c0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f2080199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2085c0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f2080199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58282/0 (socket says 172.21.15.113:58282) 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2085c0b640 1 -- 172.21.15.113:0/3042483695 learned_addr learned my addr 172.21.15.113:0/3042483695 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f2086c0d640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080107030 0x7f20801a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.136+0000 7f208640c640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20801011a0 0x7f20801995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2085c0b640 1 -- 172.21.15.113:0/3042483695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080107030 msgr2=0x7f20801a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2085c0b640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080107030 0x7f20801a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2085c0b640 1 -- 172.21.15.113:0/3042483695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20801011a0 msgr2=0x7f20801995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2085c0b640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20801011a0 0x7f20801995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2085c0b640 1 -- 172.21.15.113:0/3042483695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20801a4b60 con 0x7f2080101b40 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2086c0d640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080107030 0x7f20801a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2085c0b640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f2080199ae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f207c00d950 tx=0x7f207c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f20777fe640 1 -- 172.21.15.113:0/3042483695 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c017070 con 0x7f2080101b40 2024-04-22T07:18:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a4d30 con 0x7f2080101b40 2024-04-22T07:18:39.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f20777fe640 1 -- 172.21.15.113:0/3042483695 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f207c00de80 con 0x7f2080101b40 2024-04-22T07:18:39.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f20777fe640 1 -- 172.21.15.113:0/3042483695 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00f620 con 0x7f2080101b40 2024-04-22T07:18:39.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.137+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a5220 con 0x7f2080101b40 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.139+0000 7f20777fe640 1 -- 172.21.15.113:0/3042483695 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f207c009c30 con 0x7f2080101b40 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.139+0000 7f208b25b740 1 Processor -- start 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 -- start start 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a334d0 0x559225a338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a3c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5592259981f0 con 0x559225a3ca00 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f20777fe640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f206c0a3d20 0x7f206c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592259354b0 con 0x559225a334d0 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208b25b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5592259941b0 con 0x559225a33e80 2024-04-22T07:18:39.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2086c0d640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f206c0a3d20 0x7f206c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2086c0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a334d0 0x559225a338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58286/0 (socket says 172.21.15.113:58286) 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2086c0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a334d0 0x559225a338b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.113:57336/0 (socket says 172.21.15.113:57336) 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 -- 172.21.15.113:0/3985174329 learned_addr learned my addr 172.21.15.113:0/3985174329 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f20777fe640 1 -- 172.21.15.113:0/3042483695 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f207c0d0510 con 0x7f2080101b40 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f208640c640 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a3c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 -- 172.21.15.113:0/3985174329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 msgr2=0x559225a3c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a3c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 -- 172.21.15.113:0/3985174329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a334d0 msgr2=0x559225a338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a334d0 0x559225a338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2085c0b640 1 -- 172.21.15.113:0/3985174329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592259271b0 con 0x559225a3ca00 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.140+0000 7f2086c0d640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f206c0a3d20 0x7f206c0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2078005cc0 tx=0x7f20780323c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f2085c0b640 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3ee50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f207c09c030 tx=0x7f207c06bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ff1f0fb0a59df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f2074bfe640 1 -- 172.21.15.113:0/3985174329 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c01f170 con 0x559225a3ca00 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f2074bfe640 1 -- 172.21.15.113:0/3985174329 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f207c09ca30 con 0x559225a3ca00 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f2074bfe640 1 -- 172.21.15.113:0/3985174329 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0cdd60 con 0x559225a3ca00 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 -- 172.21.15.113:0/3985174329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 msgr2=0x559225a3ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3ee50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f207c09c030 tx=0x7f207c06bc10 comp rx=0 tx=0).stop 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 -- 172.21.15.113:0/3985174329 shutdown_connections 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3ee50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a3c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 --2- 172.21.15.113:0/3985174329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a334d0 0x559225a338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.141+0000 7f208b25b740 1 -- 172.21.15.113:0/3985174329 >> 172.21.15.113:0/3985174329 conn(0x559225a2dba0 msgr2=0x559225a2df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 -- 172.21.15.113:0/3985174329 shutdown_connections 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 -- 172.21.15.113:0/3985174329 wait complete. 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 -- start start 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 Processor -- start 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 0x559225a3c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a3ca00 0x559225a39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559225a3f910 con 0x559225a334d0 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559225951500 con 0x559225a3ca00 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559225a3fa90 con 0x559225a33e80 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208b25b740 10 client.0 _ll_register_callbacks cb 0x55922591fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208640c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 0x559225a3c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208640c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 0x559225a3c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58300/0 (socket says 172.21.15.113:58300) 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208640c640 1 -- 172.21.15.113:0/1193489400 learned_addr learned my addr 172.21.15.113:0/1193489400 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f2086c0d640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a3ca00 0x559225a39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f2085c0b640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208640c640 1 -- 172.21.15.113:0/1193489400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 msgr2=0x559225a396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.142+0000 7f208640c640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208640c640 1 -- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a3ca00 msgr2=0x559225a39ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208640c640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a3ca00 0x559225a39ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208640c640 1 -- 172.21.15.113:0/1193489400 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592256aa3f0 con 0x559225a334d0 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2085c0b640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2086c0d640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a3ca00 0x559225a39ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208640c640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 0x559225a3c340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f20700126d0 tx=0x7f2070012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bd427c8bbeac4e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2069ffb640 1 -- 172.21.15.113:0/1193489400 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207001f060 con 0x559225a334d0 2024-04-22T07:18:39.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2069ffb640 1 -- 172.21.15.113:0/1193489400 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2070010dc0 con 0x559225a334d0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207c09ca30 con 0x559225a334d0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2069ffb640 1 -- 172.21.15.113:0/1193489400 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2070026050 con 0x559225a334d0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 10 client.16479 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5592257747d0 con 0x559225a334d0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2069ffb640 1 -- 172.21.15.113:0/1193489400 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f207c09ca30 con 0x559225a334d0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f2069ffb640 1 -- 172.21.15.113:0/1193489400 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f207000d500 con 0x559225a334d0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 10 client.16479 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 20 client.16479 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225a4ccd0 0x559225a4f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 4 client.16479 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.143+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559225937930 con 0x559225a4ccd0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.144+0000 7f2085c0b640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225a4ccd0 0x559225a4f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.144+0000 7f2085c0b640 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225a4ccd0 0x559225a4f140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ae59431c14c6e9 server_cookie=e473db8852d7c9c8 in_seq=0 out_seq=0 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.144+0000 7f2069ffb640 10 client.16479 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.144+0000 7f2069ffb640 1 -- 172.21.15.113:0/1193489400 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559225937930 con 0x559225a4ccd0 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f2069ffb640 10 client.16479 handle_command_reply: tid=0 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 client.16479 shutdown 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 msgr2=0x559225a3c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 0x559225a3c340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f20700126d0 tx=0x7f2070012b90 comp rx=0 tx=0).stop 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 shutdown_connections 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225a4ccd0 0x559225a4f140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a3ca00 0x559225a39ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a33e80 0x559225a396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 --2- 172.21.15.113:0/1193489400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a334d0 0x559225a3c340 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f2069ffb640 0 client.16479 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 >> 172.21.15.113:0/1193489400 conn(0x559225a2dba0 msgr2=0x559225a2f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 shutdown_connections 2024-04-22T07:18:39.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 1 -- 172.21.15.113:0/1193489400 wait complete. 2024-04-22T07:18:39.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.145+0000 7f208b25b740 20 client.16479 trim_cache size 0 max 16384 2024-04-22T07:18:39.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 Processor -- start 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 -- start start 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20801b0880 con 0x559225a3ca00 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ae0f0 con 0x559225a33e80 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f208b25b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f20801abf40 con 0x559225a334d0 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58304/0 (socket says 172.21.15.113:58304) 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 -- 172.21.15.113:0/128776164 learned_addr learned my addr 172.21.15.113:0/128776164 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2085c0b640 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 -- 172.21.15.113:0/128776164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 msgr2=0x559225a487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 -- 172.21.15.113:0/128776164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 msgr2=0x559225a3be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a3be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 -- 172.21.15.113:0/128776164 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559225a3a2e0 con 0x559225a3ca00 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2085c0b640 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a3be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.158+0000 7f2086c0d640 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3c340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2078036000 tx=0x7f2078047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6684a014804274d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f206bfff640 1 -- 172.21.15.113:0/128776164 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078039040 con 0x559225a3ca00 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f206bfff640 1 -- 172.21.15.113:0/128776164 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2078054070 con 0x559225a3ca00 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f206bfff640 1 -- 172.21.15.113:0/128776164 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207804ed60 con 0x559225a3ca00 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 -- 172.21.15.113:0/128776164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 msgr2=0x559225a3c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3c340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2078036000 tx=0x7f2078047000 comp rx=0 tx=0).stop 2024-04-22T07:18:39.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 -- 172.21.15.113:0/128776164 shutdown_connections 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a3c340 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a3be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 --2- 172.21.15.113:0/128776164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 -- 172.21.15.113:0/128776164 >> 172.21.15.113:0/128776164 conn(0x559225a2dba0 msgr2=0x559225a34af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 -- 172.21.15.113:0/128776164 shutdown_connections 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.159+0000 7f208b25b740 1 -- 172.21.15.113:0/128776164 wait complete. 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 -- start start 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 Processor -- start 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a44ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a40280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ac5b0 con 0x559225a3ca00 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ada20 con 0x559225a33e80 2024-04-22T07:18:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f20801acc80 con 0x559225a334d0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208b25b740 10 client.0 _ll_register_callbacks cb 0x7f207000ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2085c0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a40280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58310/0 (socket says 172.21.15.113:58310) 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 -- 172.21.15.113:0/241020316 learned_addr learned my addr 172.21.15.113:0/241020316 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f208640c640 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a44ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 -- 172.21.15.113:0/241020316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 msgr2=0x559225a44ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a44ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 -- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 msgr2=0x559225a40280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a40280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 -- 172.21.15.113:0/241020316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559225a40e00 con 0x559225a3ca00 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2086c0d640 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a407c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f207802c630 tx=0x7f207802c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ed286bf43e5f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.160+0000 7f2069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f2069ffb640 1 -- 172.21.15.113:0/241020316 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078039040 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f2069ffb640 1 -- 172.21.15.113:0/241020316 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2078034dd0 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2078054070 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f2069ffb640 1 -- 172.21.15.113:0/241020316 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2078068050 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 10 client.16485 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559225ab3090 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f2069ffb640 1 -- 172.21.15.113:0/241020316 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2078054070 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f2069ffb640 1 -- 172.21.15.113:0/241020316 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f207803f040 con 0x559225a3ca00 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 10 client.16485 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 20 client.16485 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225ab3800 0x559225ab5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 4 client.16485 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:18:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592259a5c10 con 0x559225ab3800 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.161+0000 7f208640c640 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225ab3800 0x559225ab5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2069ffb640 10 client.16485 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208640c640 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225ab3800 0x559225ab5bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f863c437da43311c server_cookie=6c7d63bc864a0184 in_seq=0 out_seq=0 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2069ffb640 1 -- 172.21.15.113:0/241020316 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592259a5c10 con 0x559225ab3800 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2069ffb640 10 client.16485 handle_command_reply: tid=0 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 client.16485 shutdown 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 msgr2=0x559225a407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a407c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f207802c630 tx=0x7f207802c770 comp rx=0 tx=0).stop 2024-04-22T07:18:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 shutdown_connections 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559225ab3800 0x559225ab5bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559225a3ca00 0x559225a407c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559225a33e80 0x559225a40280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 --2- 172.21.15.113:0/241020316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559225a334d0 0x559225a44ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2069ffb640 0 client.16485 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 >> 172.21.15.113:0/241020316 conn(0x559225a2dba0 msgr2=0x559225a2c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 shutdown_connections 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 1 -- 172.21.15.113:0/241020316 wait complete. 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f208b25b740 20 client.16485 trim_cache size 0 max 16384 2024-04-22T07:18:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:18:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f206c0a3d20 msgr2=0x7f206c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2088670640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f206c0a3d20 0x7f206c0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2078005cc0 tx=0x7f20780323c0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 msgr2=0x7f2080199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2088670640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f2080199ae0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f207c00d950 tx=0x7f207c00dd50 comp rx=0 tx=0).stop 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 shutdown_connections 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.162+0000 7f2088670640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f206c0a3d20 0x7f206c0a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.163+0000 7f2088670640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2080107030 0x7f20801a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.163+0000 7f2088670640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2080101b40 0x7f2080199ae0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.163+0000 7f2088670640 1 --2- 172.21.15.113:0/3042483695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20801011a0 0x7f20801995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.163+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 >> 172.21.15.113:0/3042483695 conn(0x7f2080076c50 msgr2=0x7f20800ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.163+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 shutdown_connections 2024-04-22T07:18:39.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.163+0000 7f2088670640 1 -- 172.21.15.113:0/3042483695 wait complete. 2024-04-22T07:18:39.171 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:18:39.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/1014918998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68105ea0 msgr2=0x7f7c68106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/1014918998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68105ea0 0x7f7c68106280 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009900 tx=0x7f7c5802bc80 comp rx=0 tx=0).stop 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/1014918998 shutdown_connections 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/1014918998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c68101ad0 0x7f7c6810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/1014918998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c68101130 0x7f7c68101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/1014918998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68105ea0 0x7f7c68106280 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/1014918998 >> 172.21.15.113:0/1014918998 conn(0x7f7c68077e30 msgr2=0x7f7c68078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/1014918998 shutdown_connections 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/1014918998 wait complete. 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 Processor -- start 2024-04-22T07:18:39.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.274+0000 7f7c6d8e4640 1 -- start start 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c6d8e4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 0x7f7c68199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c6d8e4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c68101ad0 0x7f7c68199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c6d8e4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c68105ea0 0x7f7c681a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c6d8e4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111350 con 0x7f7c68101130 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c6d8e4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681111d0 con 0x7f7c68101ad0 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c6d8e4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681114d0 con 0x7f7c68105ea0 2024-04-22T07:18:39.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 0x7f7c68199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 0x7f7c68199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58320/0 (socket says 172.21.15.113:58320) 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 -- 172.21.15.113:0/2617426778 learned_addr learned my addr 172.21.15.113:0/2617426778 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c68101ad0 0x7f7c68199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c677fe640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c68105ea0 0x7f7c681a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 -- 172.21.15.113:0/2617426778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c68105ea0 msgr2=0x7f7c681a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c68105ea0 0x7f7c681a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 -- 172.21.15.113:0/2617426778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c68101ad0 msgr2=0x7f7c68199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c68101ad0 0x7f7c68199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 -- 172.21.15.113:0/2617426778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681a4d30 con 0x7f7c68101130 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.275+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 0x7f7c68199770 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7c58006010 tx=0x7f7c5802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.276+0000 7f7c6c8e2640 1 -- 172.21.15.113:0/2617426778 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58043020 con 0x7f7c68101130 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.276+0000 7f7c6c8e2640 1 -- 172.21.15.113:0/2617426778 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7c580043d0 con 0x7f7c68101130 2024-04-22T07:18:39.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.276+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c681a4f00 con 0x7f7c68101130 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.276+0000 7f7c6c8e2640 1 -- 172.21.15.113:0/2617426778 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003b20 con 0x7f7c68101130 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.276+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681a5360 con 0x7f7c68101130 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.278+0000 7f7c6c8e2640 1 -- 172.21.15.113:0/2617426778 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7c58046020 con 0x7f7c68101130 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.278+0000 7f7c705b2740 1 Processor -- start 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.278+0000 7f7c705b2740 1 -- start start 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x5578398198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x557839822430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839824e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c705b2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55783977e1f0 con 0x557839819e80 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c705b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55783971b4b0 con 0x5578398194d0 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c705b2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55783977a1b0 con 0x557839822a00 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x557839822430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x557839822430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58336/0 (socket says 172.21.15.113:58336) 2024-04-22T07:18:39.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3638702758 learned_addr learned my addr 172.21.15.113:0/3638702758 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839824e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c677fe640 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x5578398198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c6c8e2640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7c540a3d20 0x7f7c540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3638702758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 msgr2=0x557839824e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839824e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3638702758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 msgr2=0x5578398198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x5578398198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3638702758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55783970d1b0 con 0x557839819e80 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c677fe640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7c540a3d20 0x7f7c540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c6c8e2640 1 -- 172.21.15.113:0/2617426778 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f7c580f24b0 con 0x7f7c68101130 2024-04-22T07:18:39.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x557839822430 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7c580efb00 tx=0x7f7c580889e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ada2c2927daf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.279+0000 7f7c5dffb640 1 -- 172.21.15.113:0/3638702758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580bdf20 con 0x557839819e80 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c5dffb640 1 -- 172.21.15.113:0/3638702758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7c58088ef0 con 0x557839819e80 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c5dffb640 1 -- 172.21.15.113:0/3638702758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580bdd40 con 0x557839819e80 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 -- 172.21.15.113:0/3638702758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 msgr2=0x557839822430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x557839822430 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7c580efb00 tx=0x7f7c580889e0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c677fe640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7c540a3d20 0x7f7c540a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005fd0 tx=0x7f7c60004a40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 -- 172.21.15.113:0/3638702758 shutdown_connections 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839824e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x557839822430 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3638702758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x5578398198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 -- 172.21.15.113:0/3638702758 >> 172.21.15.113:0/3638702758 conn(0x557839813ba0 msgr2=0x557839813f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 -- 172.21.15.113:0/3638702758 shutdown_connections 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.280+0000 7f7c705b2740 1 -- 172.21.15.113:0/3638702758 wait complete. 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 -- start start 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 Processor -- start 2024-04-22T07:18:39.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x557839822280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x55783981f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x55783981fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55783971b4b0 con 0x5578398194d0 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557839737500 con 0x557839822a00 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55783977e1f0 con 0x557839819e80 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c705b2740 10 client.0 _ll_register_callbacks cb 0x557839705b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x557839822280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x557839822280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58338/0 (socket says 172.21.15.113:58338) 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 -- 172.21.15.113:0/1639135952 learned_addr learned my addr 172.21.15.113:0/1639135952 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x55783981f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c677fe640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x55783981fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 -- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 msgr2=0x55783981f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x55783981f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 -- 172.21.15.113:0/1639135952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 msgr2=0x55783981fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x55783981fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.281+0000 7f7c66ffd640 1 -- 172.21.15.113:0/1639135952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578394903f0 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x557839822280 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7c580c6b20 tx=0x7f7c580035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8e45c44ed479b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c46ffd640 1 -- 172.21.15.113:0/1639135952 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580bdd40 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c46ffd640 1 -- 172.21.15.113:0/1639135952 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f7c580d2b40 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c58088ef0 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c46ffd640 1 -- 172.21.15.113:0/1639135952 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7c580be1b0 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 10 client.16497 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55783955a7d0 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c46ffd640 1 -- 172.21.15.113:0/1639135952 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7c58088ef0 con 0x5578398194d0 2024-04-22T07:18:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c46ffd640 1 -- 172.21.15.113:0/1639135952 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f7c580baa50 con 0x5578398194d0 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 10 client.16497 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 10 client.16497 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 20 client.16497 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5578398326b0 0x557839834af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 4 client.16497 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55783971d930 con 0x5578398326b0 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.282+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5578398326b0 0x557839834af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.283+0000 7f7c46ffd640 10 client.16497 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.283+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5578398326b0 0x557839834af0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e50688828bddac5 server_cookie=27bbfdc62f34594a in_seq=0 out_seq=0 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.283+0000 7f7c46ffd640 1 -- 172.21.15.113:0/1639135952 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55783971d930 con 0x5578398326b0 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.283+0000 7f7c46ffd640 10 client.16497 handle_command_reply: tid=0 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.283+0000 7f7c705b2740 1 client.16497 shutdown 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 msgr2=0x557839822280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x557839822280 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7c580c6b20 tx=0x7f7c580035f0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 shutdown_connections 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5578398326b0 0x557839834af0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x55783981fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x55783981f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 --2- 172.21.15.113:0/1639135952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x557839822280 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c46ffd640 0 client.16497 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 >> 172.21.15.113:0/1639135952 conn(0x557839813ba0 msgr2=0x5578398158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 shutdown_connections 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 1 -- 172.21.15.113:0/1639135952 wait complete. 2024-04-22T07:18:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.284+0000 7f7c705b2740 20 client.16497 trim_cache size 0 max 16384 2024-04-22T07:18:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 Processor -- start 2024-04-22T07:18:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 -- start start 2024-04-22T07:18:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x55783982e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x557839821d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x557839822280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b07b0 con 0x5578398194d0 2024-04-22T07:18:39.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681ae020 con 0x557839822a00 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c705b2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681abe70 con 0x557839819e80 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x55783982e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x55783982e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58354/0 (socket says 172.21.15.113:58354) 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3337077947 learned_addr learned my addr 172.21.15.113:0/3337077947 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c677fe640 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x557839822280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3337077947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 msgr2=0x557839821d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x557839821d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x557839821d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3337077947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 msgr2=0x557839822280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x557839822280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 -- 172.21.15.113:0/3337077947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557839835170 con 0x5578398194d0 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x557839821d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.297+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x55783982e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7c580be0e0 tx=0x7f7c580bc8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80740eb705163534 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c5d7fa640 1 -- 172.21.15.113:0/3337077947 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580bdf20 con 0x5578398194d0 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c5d7fa640 1 -- 172.21.15.113:0/3337077947 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7c5808f0e0 con 0x5578398194d0 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c5d7fa640 1 -- 172.21.15.113:0/3337077947 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580d2ac0 con 0x5578398194d0 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 -- 172.21.15.113:0/3337077947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 msgr2=0x55783982e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x55783982e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7c580be0e0 tx=0x7f7c580bc8f0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 -- 172.21.15.113:0/3337077947 shutdown_connections 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557839822a00 0x557839822280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839819e80 0x557839821d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 --2- 172.21.15.113:0/3337077947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578398194d0 0x55783982e2f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 -- 172.21.15.113:0/3337077947 >> 172.21.15.113:0/3337077947 conn(0x557839813ba0 msgr2=0x557839812370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 -- 172.21.15.113:0/3337077947 shutdown_connections 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.298+0000 7f7c705b2740 1 -- 172.21.15.113:0/3337077947 wait complete. 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 -- start start 2024-04-22T07:18:39.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 Processor -- start 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x55783981c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x55783981cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839897c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681ac4e0 con 0x557839819e80 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681ad950 con 0x5578398194d0 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681acbb0 con 0x557839822a00 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c705b2740 10 client.0 _ll_register_callbacks cb 0x7f7c50004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x55783981cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x55783981cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58368/0 (socket says 172.21.15.113:58368) 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/2717879252 learned_addr learned my addr 172.21.15.113:0/2717879252 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x55783981c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 msgr2=0x557839897c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839897c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/2717879252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 msgr2=0x55783981c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x55783981c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 -- 172.21.15.113:0/2717879252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578398981b0 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c5e7fc640 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x55783981cc40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x55783981ee70 tx=0x7f7c500035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac2651e55f18c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.299+0000 7f7c46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c46ffd640 1 -- 172.21.15.113:0/2717879252 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50020360 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c46ffd640 1 -- 172.21.15.113:0/2717879252 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f7c50020500 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55783978bc10 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c46ffd640 1 -- 172.21.15.113:0/2717879252 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7c50030050 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 10 client.16503 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557839899350 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c46ffd640 1 -- 172.21.15.113:0/2717879252 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x55783978bc10 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c46ffd640 1 -- 172.21.15.113:0/2717879252 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f7c5001d3f0 con 0x557839819e80 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 10 client.16503 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 10 client.16503 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:18:39.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 20 client.16503 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x557839899b60 0x55783989bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 4 client.16503 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557839785a80 con 0x557839899b60 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.300+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x557839899b60 0x55783989bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c66ffd640 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x557839899b60 0x55783989bf80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49b596e9bab86e33 server_cookie=6422d46e44c760c1 in_seq=0 out_seq=0 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c46ffd640 10 client.16503 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c46ffd640 1 -- 172.21.15.113:0/2717879252 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557839785a80 con 0x557839899b60 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c46ffd640 10 client.16503 handle_command_reply: tid=0 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 client.16503 shutdown 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 msgr2=0x55783981cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x55783981cc40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x55783981ee70 tx=0x7f7c500035f0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 shutdown_connections 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x557839899b60 0x55783989bf80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557839822a00 0x557839897c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557839819e80 0x55783981cc40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c46ffd640 0 client.16503 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:18:39.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 --2- 172.21.15.113:0/2717879252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578398194d0 0x55783981c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 >> 172.21.15.113:0/2717879252 conn(0x557839813ba0 msgr2=0x55783981aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 shutdown_connections 2024-04-22T07:18:39.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 1 -- 172.21.15.113:0/2717879252 wait complete. 2024-04-22T07:18:39.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.301+0000 7f7c705b2740 20 client.16503 trim_cache size 0 max 16384 2024-04-22T07:18:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:18:39.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7c540a3d20 msgr2=0x7f7c540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7c540a3d20 0x7f7c540a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005fd0 tx=0x7f7c60004a40 comp rx=0 tx=0).stop 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 msgr2=0x7f7c68199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 0x7f7c68199770 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7c58006010 tx=0x7f7c5802c720 comp rx=0 tx=0).stop 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 shutdown_connections 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7c540a3d20 0x7f7c540a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c68105ea0 0x7f7c681a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c68101ad0 0x7f7c68199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 --2- 172.21.15.113:0/2617426778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c68101130 0x7f7c68199770 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 >> 172.21.15.113:0/2617426778 conn(0x7f7c68077e30 msgr2=0x7f7c680fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 shutdown_connections 2024-04-22T07:18:39.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.302+0000 7f7c6d8e4640 1 -- 172.21.15.113:0/2617426778 wait complete. 2024-04-22T07:18:39.313 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:18:39.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 -- 172.21.15.113:0/68365735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101a40 msgr2=0x7f0c80105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 --2- 172.21.15.113:0/68365735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101a40 0x7f0c80105a80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009900 tx=0x7f0c7002bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:39.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 -- 172.21.15.113:0/68365735 shutdown_connections 2024-04-22T07:18:39.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 --2- 172.21.15.113:0/68365735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c80106160 0x7f0c8010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 --2- 172.21.15.113:0/68365735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101a40 0x7f0c80105a80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 --2- 172.21.15.113:0/68365735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c80101090 0x7f0c80101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 -- 172.21.15.113:0/68365735 >> 172.21.15.113:0/68365735 conn(0x7f0c80077f50 msgr2=0x7f0c80078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 -- 172.21.15.113:0/68365735 shutdown_connections 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.415+0000 7f0c87d32640 1 -- 172.21.15.113:0/68365735 wait complete. 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 Processor -- start 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 -- start start 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 0x7f0c8019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c80106160 0x7f0c8019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c80198660 0x7f0c80198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801122c0 con 0x7f0c80101090 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80112140 con 0x7f0c80106160 2024-04-22T07:18:39.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c87d32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80112440 con 0x7f0c80198660 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 0x7f0c8019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 0x7f0c8019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58398/0 (socket says 172.21.15.113:58398) 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 -- 172.21.15.113:0/1477776544 learned_addr learned my addr 172.21.15.113:0/1477776544 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c852cd640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c80106160 0x7f0c8019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 -- 172.21.15.113:0/1477776544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c80198660 msgr2=0x7f0c80198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c80198660 0x7f0c80198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 -- 172.21.15.113:0/1477776544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c80106160 msgr2=0x7f0c8019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c80106160 0x7f0c8019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.416+0000 7f0c85ace640 1 -- 172.21.15.113:0/1477776544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c80199260 con 0x7f0c80101090 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.417+0000 7f0c85ace640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 0x7f0c8019e5a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00d950 tx=0x7f0c7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.417+0000 7f0c76ffd640 1 -- 172.21.15.113:0/1477776544 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c017030 con 0x7f0c80101090 2024-04-22T07:18:39.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.417+0000 7f0c76ffd640 1 -- 172.21.15.113:0/1477776544 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c7c004020 con 0x7f0c80101090 2024-04-22T07:18:39.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.417+0000 7f0c76ffd640 1 -- 172.21.15.113:0/1477776544 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0047a0 con 0x7f0c80101090 2024-04-22T07:18:39.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.417+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c801a5340 con 0x7f0c80101090 2024-04-22T07:18:39.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.417+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801a5740 con 0x7f0c80101090 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.419+0000 7f0c76ffd640 1 -- 172.21.15.113:0/1477776544 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0c7c0041d0 con 0x7f0c80101090 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 Processor -- start 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 -- start start 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c76ffd640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c6c0a3d20 0x7f0c6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca82cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca8b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56195c9e31f0 con 0x56195ca8be10 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56195c9804b0 con 0x56195ca83290 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56195c9df1b0 con 0x56195ca828e0 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c862cf640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c6c0a3d20 0x7f0c6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c76ffd640 1 -- 172.21.15.113:0/1477776544 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f0c7c0d0420 con 0x7f0c80101090 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c862cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c862cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca8e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58404/0 (socket says 172.21.15.113:58404) 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c852cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca8b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c862cf640 1 -- 172.21.15.113:0/2185285502 learned_addr learned my addr 172.21.15.113:0/2185285502 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.420+0000 7f0c85ace640 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca82cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 -- 172.21.15.113:0/2185285502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 msgr2=0x56195ca82cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca82cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 -- 172.21.15.113:0/2185285502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 msgr2=0x56195ca8b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca8b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 -- 172.21.15.113:0/2185285502 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195c9721b0 con 0x56195ca8be10 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c852cd640 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c6c0a3d20 0x7f0c6c0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009fd0 tx=0x7f0c78009c60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c862cf640 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca8e260 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800f730 tx=0x7f0c78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b080b4f261cf27d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c6bfff640 1 -- 172.21.15.113:0/2185285502 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78022070 con 0x56195ca8be10 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c6bfff640 1 -- 172.21.15.113:0/2185285502 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c7800fe60 con 0x56195ca8be10 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.421+0000 7f0c6bfff640 1 -- 172.21.15.113:0/2185285502 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78005580 con 0x56195ca8be10 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2185285502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 msgr2=0x56195ca8e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca8e260 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800f730 tx=0x7f0c78019000 comp rx=0 tx=0).stop 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2185285502 shutdown_connections 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca8e260 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2185285502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca82cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2185285502 >> 172.21.15.113:0/2185285502 conn(0x56195ca7cfb0 msgr2=0x56195ca7d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2185285502 shutdown_connections 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2185285502 wait complete. 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 -- start start 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.422+0000 7f0c8aa02740 1 Processor -- start 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56195ca8ed20 con 0x56195ca8be10 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56195c99c500 con 0x56195ca83290 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56195ca8eea0 con 0x56195ca828e0 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 10 client.0 _ll_register_callbacks cb 0x56195c96ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca890e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58418/0 (socket says 172.21.15.113:58418) 2024-04-22T07:18:39.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 -- 172.21.15.113:0/2777153765 learned_addr learned my addr 172.21.15.113:0/2777153765 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c85ace640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c852cd640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 -- 172.21.15.113:0/2777153765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 msgr2=0x56195ca8b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca8b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 -- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 msgr2=0x56195ca88ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca88ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 -- 172.21.15.113:0/2777153765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56195c6f53f0 con 0x56195ca8be10 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c852cd640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca88ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c862cf640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca890e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0c78010010 tx=0x7f0c7800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9913e4dceddfa772 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c697fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c697fa640 1 -- 172.21.15.113:0/2777153765 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78034060 con 0x56195ca8be10 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7800fe60 con 0x56195ca8be10 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c697fa640 1 -- 172.21.15.113:0/2777153765 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f0c78004d30 con 0x56195ca8be10 2024-04-22T07:18:39.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c697fa640 1 -- 172.21.15.113:0/2777153765 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0c7803b050 con 0x56195ca8be10 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 10 client.16515 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.423+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56195c7bf7d0 con 0x56195ca8be10 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c697fa640 1 -- 172.21.15.113:0/2777153765 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c7800fe60 con 0x56195ca8be10 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c697fa640 1 -- 172.21.15.113:0/2777153765 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f0c7802e030 con 0x56195ca8be10 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c8aa02740 10 client.16515 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c8aa02740 10 client.16515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c8aa02740 20 client.16515 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195ca9bec0 0x56195ca9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c8aa02740 4 client.16515 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56195c982930 con 0x56195ca9bec0 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c85ace640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195ca9bec0 0x56195ca9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c697fa640 10 client.16515 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.424+0000 7f0c85ace640 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195ca9bec0 0x56195ca9e2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d67b47b9155ffa9 server_cookie=3b47193c0477bf72 in_seq=0 out_seq=0 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c697fa640 1 -- 172.21.15.113:0/2777153765 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56195c982930 con 0x56195ca9bec0 2024-04-22T07:18:39.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c697fa640 10 client.16515 handle_command_reply: tid=0 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 client.16515 shutdown 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 msgr2=0x56195ca890e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca890e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0c78010010 tx=0x7f0c7800de60 comp rx=0 tx=0).stop 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 shutdown_connections 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195ca9bec0 0x56195ca9e2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca890e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca83290 0x56195ca88ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c697fa640 0 client.16515 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/2777153765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca828e0 0x56195ca8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 >> 172.21.15.113:0/2777153765 conn(0x56195ca7cfb0 msgr2=0x56195ca7ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 shutdown_connections 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.425+0000 7f0c8aa02740 1 -- 172.21.15.113:0/2777153765 wait complete. 2024-04-22T07:18:39.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.426+0000 7f0c8aa02740 20 client.16515 trim_cache size 0 max 16384 2024-04-22T07:18:39.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 Processor -- start 2024-04-22T07:18:39.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 -- start start 2024-04-22T07:18:39.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca97c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 0x56195ca7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca8be10 0x56195ca8b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ae720 con 0x56195ca83290 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801abf90 con 0x56195ca828e0 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801a9de0 con 0x56195ca8be10 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 0x56195ca7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 0x56195ca7eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58430/0 (socket says 172.21.15.113:58430) 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 -- 172.21.15.113:0/212036819 learned_addr learned my addr 172.21.15.113:0/212036819 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c85ace640 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca97c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 -- 172.21.15.113:0/212036819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca8be10 msgr2=0x56195ca8b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca8be10 0x56195ca8b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 -- 172.21.15.113:0/212036819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 msgr2=0x56195ca97c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca97c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 -- 172.21.15.113:0/212036819 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195ca9e810 con 0x56195ca83290 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c85ace640 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca97c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:39.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c852cd640 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 0x56195ca7eed0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009900 tx=0x7f0c7002c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645d43843c83bf59 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c6b7fe640 1 -- 172.21.15.113:0/212036819 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7002cad0 con 0x56195ca83290 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c6b7fe640 1 -- 172.21.15.113:0/212036819 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c7002cc70 con 0x56195ca83290 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.439+0000 7f0c6b7fe640 1 -- 172.21.15.113:0/212036819 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700049b0 con 0x56195ca83290 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 -- 172.21.15.113:0/212036819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 msgr2=0x56195ca7eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 0x56195ca7eed0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009900 tx=0x7f0c7002c710 comp rx=0 tx=0).stop 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 -- 172.21.15.113:0/212036819 shutdown_connections 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca8be10 0x56195ca8b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca83290 0x56195ca7eed0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/212036819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca97c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 -- 172.21.15.113:0/212036819 >> 172.21.15.113:0/212036819 conn(0x56195ca7cfb0 msgr2=0x56195ca87b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 -- 172.21.15.113:0/212036819 shutdown_connections 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 -- 172.21.15.113:0/212036819 wait complete. 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 -- start start 2024-04-22T07:18:39.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.440+0000 7f0c8aa02740 1 Processor -- start 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca94330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca83290 0x56195ca85380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801aa450 con 0x56195ca8be10 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ab8c0 con 0x56195ca828e0 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801aab20 con 0x56195ca83290 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c8aa02740 10 client.0 _ll_register_callbacks cb 0x7f0c7800faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58432/0 (socket says 172.21.15.113:58432) 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 -- 172.21.15.113:0/3488496922 learned_addr learned my addr 172.21.15.113:0/3488496922 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c852cd640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca83290 0x56195ca85380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c85ace640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca94330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 -- 172.21.15.113:0/3488496922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca83290 msgr2=0x56195ca85380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca83290 0x56195ca85380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 -- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 msgr2=0x56195ca94330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca94330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 -- 172.21.15.113:0/3488496922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56195ca85e80 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c862cf640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca858c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009a50 tx=0x7f0c78009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f5fef11091ade0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.441+0000 7f0c697fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c697fa640 1 -- 172.21.15.113:0/3488496922 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78034070 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c697fa640 1 -- 172.21.15.113:0/3488496922 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f0c7800fc60 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7002cc70 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c697fa640 1 -- 172.21.15.113:0/3488496922 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0c78022040 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 10 client.16521 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56195cb02010 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c697fa640 1 -- 172.21.15.113:0/3488496922 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c7002cc70 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c697fa640 1 -- 172.21.15.113:0/3488496922 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f0c7802e040 con 0x56195ca8be10 2024-04-22T07:18:39.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 10 client.16521 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 10 client.16521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 20 client.16521 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195cb027d0 0x56195cb04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 4 client.16521 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56195c9f0c10 con 0x56195cb027d0 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.442+0000 7f0c85ace640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195cb027d0 0x56195cb04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c697fa640 10 client.16521 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c85ace640 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195cb027d0 0x56195cb04bf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c923f1e389349b0 server_cookie=9b9ec35039d911f1 in_seq=0 out_seq=0 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c697fa640 1 -- 172.21.15.113:0/3488496922 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56195c9f0c10 con 0x56195cb027d0 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c697fa640 10 client.16521 handle_command_reply: tid=0 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 client.16521 shutdown 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 msgr2=0x56195ca858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca858c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009a50 tx=0x7f0c78009820 comp rx=0 tx=0).stop 2024-04-22T07:18:39.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 shutdown_connections 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x56195cb027d0 0x56195cb04bf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56195ca8be10 0x56195ca858c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56195ca83290 0x56195ca85380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 --2- 172.21.15.113:0/3488496922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56195ca828e0 0x56195ca94330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c697fa640 0 client.16521 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 >> 172.21.15.113:0/3488496922 conn(0x56195ca7cfb0 msgr2=0x56195ca77370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 shutdown_connections 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 1 -- 172.21.15.113:0/3488496922 wait complete. 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.443+0000 7f0c8aa02740 20 client.16521 trim_cache size 0 max 16384 2024-04-22T07:18:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:18:39.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c6c0a3d20 msgr2=0x7f0c6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c6c0a3d20 0x7f0c6c0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009fd0 tx=0x7f0c78009c60 comp rx=0 tx=0).stop 2024-04-22T07:18:39.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 msgr2=0x7f0c8019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:39.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 0x7f0c8019e5a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00d950 tx=0x7f0c7c00de10 comp rx=0 tx=0).stop 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 shutdown_connections 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c6c0a3d20 0x7f0c6c0a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c80198660 0x7f0c80198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c80106160 0x7f0c8019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 --2- 172.21.15.113:0/1477776544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c80101090 0x7f0c8019e5a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 >> 172.21.15.113:0/1477776544 conn(0x7f0c80077f50 msgr2=0x7f0c800fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 shutdown_connections 2024-04-22T07:18:39.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:39.444+0000 7f0c87d32640 1 -- 172.21.15.113:0/1477776544 wait complete. 2024-04-22T07:18:39.967 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:39 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3746071384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:39.967 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:39 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1505621373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:39 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3746071384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:40.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:39 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1505621373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:39 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3746071384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:40.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:39 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1505621373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:18:40.453 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:18:40.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:18:40.453 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:18:40.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 -- 172.21.15.113:0/44644127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 msgr2=0x7f51f406af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 --2- 172.21.15.113:0/44644127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f406af10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009900 tx=0x7f51f002bcd0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 -- 172.21.15.113:0/44644127 shutdown_connections 2024-04-22T07:18:40.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 --2- 172.21.15.113:0/44644127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f410e810 0x7f51f4110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 --2- 172.21.15.113:0/44644127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f406af10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 --2- 172.21.15.113:0/44644127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f4102f00 0x7f51f406a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 -- 172.21.15.113:0/44644127 >> 172.21.15.113:0/44644127 conn(0x7f51f4076c50 msgr2=0x7f51f4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 -- 172.21.15.113:0/44644127 shutdown_connections 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.570+0000 7f51fc55f640 1 -- 172.21.15.113:0/44644127 wait complete. 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 Processor -- start 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 -- start start 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f4102f00 0x7f51f41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f410e810 0x7f51f419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4114580 con 0x7f51f406aa90 2024-04-22T07:18:40.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4114400 con 0x7f51f4102f00 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fc55f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4114700 con 0x7f51f410e810 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f41a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58450/0 (socket says 172.21.15.113:58450) 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 -- 172.21.15.113:0/2454431134 learned_addr learned my addr 172.21.15.113:0/2454431134 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51faafc640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f410e810 0x7f51f419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51f9afa640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f4102f00 0x7f51f41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 -- 172.21.15.113:0/2454431134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f410e810 msgr2=0x7f51f419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f410e810 0x7f51f419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 -- 172.21.15.113:0/2454431134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f4102f00 msgr2=0x7f51f41a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f4102f00 0x7f51f41a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 -- 172.21.15.113:0/2454431134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f419b0e0 con 0x7f51f406aa90 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51f9afa640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f4102f00 0x7f51f41a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.571+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f41a07b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f51e400d770 tx=0x7f51e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.572+0000 7f51eb7fe640 1 -- 172.21.15.113:0/2454431134 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4017070 con 0x7f51f406aa90 2024-04-22T07:18:40.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.572+0000 7f51eb7fe640 1 -- 172.21.15.113:0/2454431134 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f51e40054f0 con 0x7f51f406aa90 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.572+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f419b3d0 con 0x7f51f406aa90 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.572+0000 7f51eb7fe640 1 -- 172.21.15.113:0/2454431134 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4004de0 con 0x7f51f406aa90 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.572+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f41a7670 con 0x7f51f406aa90 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.574+0000 7f51eb7fe640 1 -- 172.21.15.113:0/2454431134 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f51e4004f80 con 0x7f51f406aa90 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.574+0000 7f51ff22a740 1 Processor -- start 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.574+0000 7f51ff22a740 1 -- start start 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 0x562879f78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f79290 0x562879f81840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f84260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51ff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562879edcf00 con 0x562879f788e0 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51ff22a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562879e7a590 con 0x562879f79290 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51ff22a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562879ed8ec0 con 0x562879f81e10 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51eb7fe640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f51e00a3d20 0x7f51e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 0x562879f78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51eb7fe640 1 -- 172.21.15.113:0/2454431134 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f51e40d0540 con 0x7f51f406aa90 2024-04-22T07:18:40.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 0x562879f78cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58462/0 (socket says 172.21.15.113:58462) 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 -- 172.21.15.113:0/1733331555 learned_addr learned my addr 172.21.15.113:0/1733331555 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f51e00a3d20 0x7f51e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f79290 0x562879f81840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51f9afa640 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f84260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 -- 172.21.15.113:0/1733331555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 msgr2=0x562879f84260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f84260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 -- 172.21.15.113:0/1733331555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f79290 msgr2=0x562879f81840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f79290 0x562879f81840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 -- 172.21.15.113:0/1733331555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562879cb77e0 con 0x562879f788e0 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.575+0000 7f51faafc640 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 0x562879f78cc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00aef0 tx=0x7f51ec016640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2326653ed07f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51faafc640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f51e00a3d20 0x7f51e00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0070b0 tx=0x7f51ec01a000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51e8bfe640 1 -- 172.21.15.113:0/1733331555 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec026070 con 0x562879f788e0 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51e8bfe640 1 -- 172.21.15.113:0/1733331555 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f51ec009c50 con 0x562879f788e0 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51e8bfe640 1 -- 172.21.15.113:0/1733331555 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec013040 con 0x562879f788e0 2024-04-22T07:18:40.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 -- 172.21.15.113:0/1733331555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 msgr2=0x562879f78cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 0x562879f78cc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00aef0 tx=0x7f51ec016640 comp rx=0 tx=0).stop 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 -- 172.21.15.113:0/1733331555 shutdown_connections 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f84260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f79290 0x562879f81840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1733331555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f788e0 0x562879f78cc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.576+0000 7f51ff22a740 1 -- 172.21.15.113:0/1733331555 >> 172.21.15.113:0/1733331555 conn(0x562879f72fb0 msgr2=0x562879f73390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 -- 172.21.15.113:0/1733331555 shutdown_connections 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 -- 172.21.15.113:0/1733331555 wait complete. 2024-04-22T07:18:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 -- start start 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 Processor -- start 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f7ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f79290 0x562879f7f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 0x562879f7f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562879f84d20 con 0x562879f81e10 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562879e96580 con 0x562879f788e0 2024-04-22T07:18:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562879f84ea0 con 0x562879f79290 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51ff22a740 10 client.0 _ll_register_callbacks cb 0x562879e9c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51faafc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 0x562879f7f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51faafc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 0x562879f7f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58470/0 (socket says 172.21.15.113:58470) 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51faafc640 1 -- 172.21.15.113:0/1616009629 learned_addr learned my addr 172.21.15.113:0/1616009629 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f7ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.577+0000 7f51f9afa640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f79290 0x562879f7f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51faafc640 1 -- 172.21.15.113:0/1616009629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f79290 msgr2=0x562879f7f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51faafc640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f79290 0x562879f7f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51faafc640 1 -- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 msgr2=0x562879f7ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51faafc640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f7ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51faafc640 1 -- 172.21.15.113:0/1616009629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562879d17f70 con 0x562879f81e10 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51f9afa640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f79290 0x562879f7f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51faafc640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 0x562879f7f7f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00a670 tx=0x7f51ec00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba449eb7f510d571 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ddffb640 1 -- 172.21.15.113:0/1616009629 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec030070 con 0x562879f81e10 2024-04-22T07:18:40.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ddffb640 1 -- 172.21.15.113:0/1616009629 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f51ec021c90 con 0x562879f81e10 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ec009c50 con 0x562879f81e10 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ddffb640 1 -- 172.21.15.113:0/1616009629 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f51ec03c050 con 0x562879f81e10 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 10 client.16533 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562879baf170 con 0x562879f81e10 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ddffb640 1 -- 172.21.15.113:0/1616009629 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f51ec009c50 con 0x562879f81e10 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ddffb640 1 -- 172.21.15.113:0/1616009629 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f51ec02b700 con 0x562879f81e10 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 10 client.16533 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:40.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 20 client.16533 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879f921b0 0x562879f945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 4 client.16533 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.578+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562879eea920 con 0x562879f921b0 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.579+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879f921b0 0x562879f945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.579+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879f921b0 0x562879f945a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495fcee23c90c530 server_cookie=f4d0997482d6c66e in_seq=0 out_seq=0 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.579+0000 7f51ddffb640 10 client.16533 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ddffb640 1 -- 172.21.15.113:0/1616009629 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f51e4081ae0 con 0x562879f921b0 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ddffb640 10 client.16533 handle_command_reply: tid=0 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 client.16533 shutdown 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 msgr2=0x562879f7f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 0x562879f7f7f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00a670 tx=0x7f51ec00a6a0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 shutdown_connections 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879f921b0 0x562879f945a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f81e10 0x562879f7f7f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f79290 0x562879f7f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1616009629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f7ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ddffb640 0 client.16533 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 >> 172.21.15.113:0/1616009629 conn(0x562879f72fb0 msgr2=0x562879f74d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 shutdown_connections 2024-04-22T07:18:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 1 -- 172.21.15.113:0/1616009629 wait complete. 2024-04-22T07:18:40.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.580+0000 7f51ff22a740 20 client.16533 trim_cache size 0 max 16384 2024-04-22T07:18:40.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 Processor -- start 2024-04-22T07:18:40.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 -- start start 2024-04-22T07:18:40.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41b2b20 con 0x562879f79290 2024-04-22T07:18:40.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41b0390 con 0x562879f788e0 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51ff22a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41ae1e0 con 0x562879f81e10 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58484/0 (socket says 172.21.15.113:58484) 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 -- 172.21.15.113:0/1364115521 learned_addr learned my addr 172.21.15.113:0/1364115521 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 -- 172.21.15.113:0/1364115521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 msgr2=0x562879f99c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 -- 172.21.15.113:0/1364115521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 msgr2=0x562879f8dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f8dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 -- 172.21.15.113:0/1364115521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562879f738f0 con 0x562879f79290 2024-04-22T07:18:40.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f8dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:18:40.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.598+0000 7f51f9afa640 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7f610 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f51f0000c00 tx=0x7f51f002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904eab9ff892dbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51dffff640 1 -- 172.21.15.113:0/1364115521 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f002c9f0 con 0x562879f79290 2024-04-22T07:18:40.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51dffff640 1 -- 172.21.15.113:0/1364115521 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f51f002cb90 con 0x562879f79290 2024-04-22T07:18:40.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51dffff640 1 -- 172.21.15.113:0/1364115521 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004720 con 0x562879f79290 2024-04-22T07:18:40.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 -- 172.21.15.113:0/1364115521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 msgr2=0x562879f7f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7f610 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f51f0000c00 tx=0x7f51f002c7b0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 -- 172.21.15.113:0/1364115521 shutdown_connections 2024-04-22T07:18:40.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f81e10 0x562879f99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7f610 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 --2- 172.21.15.113:0/1364115521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f788e0 0x562879f8dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 -- 172.21.15.113:0/1364115521 >> 172.21.15.113:0/1364115521 conn(0x562879f72fb0 msgr2=0x562879f72010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 -- 172.21.15.113:0/1364115521 shutdown_connections 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.599+0000 7f51ff22a740 1 -- 172.21.15.113:0/1364115521 wait complete. 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 -- start start 2024-04-22T07:18:40.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 Processor -- start 2024-04-22T07:18:40.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f788e0 0x562879f7bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f81e10 0x562879f7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41ae850 con 0x562879f79290 2024-04-22T07:18:40.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41afcc0 con 0x562879f81e10 2024-04-22T07:18:40.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41aef20 con 0x562879f788e0 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 10 client.0 _ll_register_callbacks cb 0x7f51ec024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58496/0 (socket says 172.21.15.113:58496) 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 -- 172.21.15.113:0/2284019721 learned_addr learned my addr 172.21.15.113:0/2284019721 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f788e0 0x562879f7bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51faafc640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f81e10 0x562879f7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 -- 172.21.15.113:0/2284019721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f788e0 msgr2=0x562879f7bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f788e0 0x562879f7bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 -- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f81e10 msgr2=0x562879f7c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f81e10 0x562879f7c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 -- 172.21.15.113:0/2284019721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562879ff7740 con 0x562879f79290 2024-04-22T07:18:40.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51f9afa640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7c200 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036040 tx=0x7f51f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830d429b2aece3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ddffb640 1 -- 172.21.15.113:0/2284019721 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f002c9f0 con 0x562879f79290 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ddffb640 1 -- 172.21.15.113:0/2284019721 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f51f00037f0 con 0x562879f79290 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.600+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f002cb90 con 0x562879f79290 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ddffb640 1 -- 172.21.15.113:0/2284019721 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f51f005b050 con 0x562879f79290 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 10 client.16539 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562879ff8a60 con 0x562879f79290 2024-04-22T07:18:40.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ddffb640 1 -- 172.21.15.113:0/2284019721 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f51f002cb90 con 0x562879f79290 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ddffb640 1 -- 172.21.15.113:0/2284019721 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f51f003f410 con 0x562879f79290 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 20 client.16539 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879ff91f0 0x562879ffb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 4 client.16539 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562879ee4790 con 0x562879ff91f0 2024-04-22T07:18:40.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.601+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879ff91f0 0x562879ffb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.602+0000 7f51fa2fb640 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879ff91f0 0x562879ffb610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da4b47cf9f8d540 server_cookie=1b21d058a5642236 in_seq=0 out_seq=0 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.602+0000 7f51ddffb640 10 client.16539 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ddffb640 1 -- 172.21.15.113:0/2284019721 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562879ee4790 con 0x562879ff91f0 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ddffb640 10 client.16539 handle_command_reply: tid=0 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 client.16539 shutdown 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 msgr2=0x562879f7c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7c200 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036040 tx=0x7f51f0004270 comp rx=0 tx=0).stop 2024-04-22T07:18:40.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 shutdown_connections 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562879ff91f0 0x562879ffb610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562879f81e10 0x562879f7c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ddffb640 0 client.16539 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562879f79290 0x562879f7c200 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 --2- 172.21.15.113:0/2284019721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562879f788e0 0x562879f7bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 >> 172.21.15.113:0/2284019721 conn(0x562879f72fb0 msgr2=0x562879f742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 shutdown_connections 2024-04-22T07:18:40.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 1 -- 172.21.15.113:0/2284019721 wait complete. 2024-04-22T07:18:40.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51ff22a740 20 client.16539 trim_cache size 0 max 16384 2024-04-22T07:18:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:40.622 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:18:40.622 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "60592470-52d2-4d92-ad28-bb59bb968a1d", 2024-04-22T07:18:40.623 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:18:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:40.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f51e00a3d20 msgr2=0x7f51e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51fc55f640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f51e00a3d20 0x7f51e00a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0070b0 tx=0x7f51ec01a000 comp rx=0 tx=0).stop 2024-04-22T07:18:40.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 msgr2=0x7f51f41a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.603+0000 7f51fc55f640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f41a07b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f51e400d770 tx=0x7f51e400dc30 comp rx=0 tx=0).stop 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 shutdown_connections 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f51e00a3d20 0x7f51e00a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f410e810 0x7f51f419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51f4102f00 0x7f51f41a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 --2- 172.21.15.113:0/2454431134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f406aa90 0x7f51f41a07b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 >> 172.21.15.113:0/2454431134 conn(0x7f51f4076c50 msgr2=0x7f51f410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 shutdown_connections 2024-04-22T07:18:40.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.604+0000 7f51fc55f640 1 -- 172.21.15.113:0/2454431134 wait complete. 2024-04-22T07:18:40.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:18:40.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 -- 172.21.15.113:0/4027152744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e0101a40 msgr2=0x7f75e0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 --2- 172.21.15.113:0/4027152744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e0101a40 0x7f75e0105a80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009900 tx=0x7f75d002bc30 comp rx=0 tx=0).stop 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 -- 172.21.15.113:0/4027152744 shutdown_connections 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 --2- 172.21.15.113:0/4027152744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75e0106160 0x7f75e010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 --2- 172.21.15.113:0/4027152744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e0101a40 0x7f75e0105a80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 --2- 172.21.15.113:0/4027152744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0101090 0x7f75e0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 -- 172.21.15.113:0/4027152744 >> 172.21.15.113:0/4027152744 conn(0x7f75e00771d0 msgr2=0x7f75e00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 -- 172.21.15.113:0/4027152744 shutdown_connections 2024-04-22T07:18:40.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.730+0000 7f75e8306640 1 -- 172.21.15.113:0/4027152744 wait complete. 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 Processor -- start 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 -- start start 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75e0101090 0x7f75e01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0106160 0x7f75e01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 0x7f75e019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0112300 con 0x7f75e019a8c0 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0112180 con 0x7f75e0101090 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e8306640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75e0112480 con 0x7f75e0106160 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e68a3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 0x7f75e019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e68a3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 0x7f75e019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58514/0 (socket says 172.21.15.113:58514) 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.731+0000 7f75e68a3640 1 -- 172.21.15.113:0/1318317851 learned_addr learned my addr 172.21.15.113:0/1318317851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e60a2640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75e0101090 0x7f75e01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0106160 0x7f75e01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e68a3640 1 -- 172.21.15.113:0/1318317851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0106160 msgr2=0x7f75e01a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e68a3640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0106160 0x7f75e01a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e68a3640 1 -- 172.21.15.113:0/1318317851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75e0101090 msgr2=0x7f75e01a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e68a3640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75e0101090 0x7f75e01a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e68a3640 1 -- 172.21.15.113:0/1318317851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e019b3e0 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0106160 0x7f75e01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e68a3640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 0x7f75e019ad70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f75d800eaa0 tx=0x7f75d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75d77fe640 1 -- 172.21.15.113:0/1318317851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d800cdb0 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75d77fe640 1 -- 172.21.15.113:0/1318317851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f75d8005020 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75d77fe640 1 -- 172.21.15.113:0/1318317851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d80116d0 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e019b6d0 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.732+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01a7920 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.734+0000 7f75d77fe640 1 -- 172.21.15.113:0/1318317851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f75d80051d0 con 0x7f75e019a8c0 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.734+0000 7f75eafcc740 1 Processor -- start 2024-04-22T07:18:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 -- start start 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x562257399960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5622572fe1e0 con 0x5622573a2a80 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56225729b390 con 0x562257399580 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75eafcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5622572fa1a0 con 0x562257399ea0 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75d77fe640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75cc0a3da0 0x7f75cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58520/0 (socket says 172.21.15.113:58520) 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 -- 172.21.15.113:0/1896797770 learned_addr learned my addr 172.21.15.113:0/1896797770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e60a2640 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x562257399960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e68a3640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75cc0a3da0 0x7f75cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75d77fe640 1 -- 172.21.15.113:0/1318317851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f75d80d0bf0 con 0x7f75e019a8c0 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 -- 172.21.15.113:0/1896797770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 msgr2=0x5622573a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 -- 172.21.15.113:0/1896797770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 msgr2=0x562257399960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x562257399960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.735+0000 7f75e58a1640 1 -- 172.21.15.113:0/1896797770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622570da7d0 con 0x5622573a2a80 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573a4e70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005c20 tx=0x7f75d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95117015039da56 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75e68a3640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75cc0a3da0 0x7f75cc0a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75d800efd0 tx=0x7f75d8087040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75d4bfe640 1 -- 172.21.15.113:0/1896797770 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0048070 con 0x5622573a2a80 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75d4bfe640 1 -- 172.21.15.113:0/1896797770 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f75d002cc70 con 0x5622573a2a80 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75d4bfe640 1 -- 172.21.15.113:0/1896797770 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0003d80 con 0x5622573a2a80 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75eafcc740 1 -- 172.21.15.113:0/1896797770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 msgr2=0x5622573a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.736+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573a4e70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005c20 tx=0x7f75d0000bc0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- 172.21.15.113:0/1896797770 shutdown_connections 2024-04-22T07:18:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573a4e70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1896797770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x562257399960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- 172.21.15.113:0/1896797770 >> 172.21.15.113:0/1896797770 conn(0x562257393cb0 msgr2=0x562257394090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- 172.21.15.113:0/1896797770 shutdown_connections 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- 172.21.15.113:0/1896797770 wait complete. 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- start start 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 Processor -- start 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 0x56225739fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622573a2a80 0x5622573a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5622573a5810 con 0x562257399580 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5622572b72f0 con 0x5622573a2a80 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5622573a5990 con 0x562257399ea0 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75eafcc740 10 client.0 _ll_register_callbacks cb 0x5622572c4580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:40.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75e58a1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 0x56225739fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75e58a1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 0x56225739fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58536/0 (socket says 172.21.15.113:58536) 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75e58a1640 1 -- 172.21.15.113:0/3925625851 learned_addr learned my addr 172.21.15.113:0/3925625851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.737+0000 7f75e60a2640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622573a2a80 0x5622573a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 -- 172.21.15.113:0/3925625851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 msgr2=0x5622573a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 -- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622573a2a80 msgr2=0x5622573a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622573a2a80 0x5622573a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 -- 172.21.15.113:0/3925625851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622570103f0 con 0x562257399580 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75e58a1640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 0x56225739fb90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009610 tx=0x7f75d002c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289e18baa485926 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75c9ffb640 1 -- 172.21.15.113:0/3925625851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0048040 con 0x562257399580 2024-04-22T07:18:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d002cc70 con 0x562257399580 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75c9ffb640 1 -- 172.21.15.113:0/3925625851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f75d0005220 con 0x562257399580 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75c9ffb640 1 -- 172.21.15.113:0/3925625851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f75d0005420 con 0x562257399580 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75eafcc740 10 client.16551 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562256fd00f0 con 0x562257399580 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75c9ffb640 1 -- 172.21.15.113:0/3925625851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f75d002cc70 con 0x562257399580 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.738+0000 7f75c9ffb640 1 -- 172.21.15.113:0/3925625851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f75d004e030 con 0x562257399580 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75eafcc740 10 client.16551 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75eafcc740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75eafcc740 20 client.16551 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75eafcc740 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5622573b2980 0x5622573b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75eafcc740 4 client.16551 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56225730bc00 con 0x5622573b2980 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75e60a2640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5622573b2980 0x5622573b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75c9ffb640 10 client.16551 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.739+0000 7f75e60a2640 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5622573b2980 0x5622573b4d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f5d0279a9278717 server_cookie=e8b1c8733cfa8b10 in_seq=0 out_seq=0 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75c9ffb640 1 -- 172.21.15.113:0/3925625851 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56225730bc00 con 0x5622573b2980 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75c9ffb640 10 client.16551 handle_command_reply: tid=0 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 client.16551 shutdown 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 msgr2=0x56225739fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 0x56225739fb90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009610 tx=0x7f75d002c700 comp rx=0 tx=0).stop 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 shutdown_connections 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5622573b2980 0x5622573b4d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622573a2a80 0x5622573a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399ea0 0x5622573a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75c9ffb640 0 client.16551 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 --2- 172.21.15.113:0/3925625851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399580 0x56225739fb90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 >> 172.21.15.113:0/3925625851 conn(0x562257393cb0 msgr2=0x562257395a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 shutdown_connections 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 1 -- 172.21.15.113:0/3925625851 wait complete. 2024-04-22T07:18:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.740+0000 7f75eafcc740 20 client.16551 trim_cache size 0 max 16384 2024-04-22T07:18:40.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.757+0000 7f75eafcc740 1 Processor -- start 2024-04-22T07:18:40.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 -- start start 2024-04-22T07:18:40.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399580 0x5622573a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399ea0 0x562257392bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b2a70 con 0x5622573a2a80 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01b02e0 con 0x562257399ea0 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75eafcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01ae130 con 0x562257399580 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58538/0 (socket says 172.21.15.113:58538) 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 -- 172.21.15.113:0/1331433182 learned_addr learned my addr 172.21.15.113:0/1331433182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 -- 172.21.15.113:0/1331433182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399580 msgr2=0x5622573a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399580 0x5622573a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 -- 172.21.15.113:0/1331433182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399ea0 msgr2=0x562257392bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399ea0 0x562257392bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 -- 172.21.15.113:0/1331433182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56225739e0e0 con 0x5622573a2a80 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75e58a1640 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573b7d90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f75d00092c0 tx=0x7f75d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8841240fb84ab249 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75cbfff640 1 -- 172.21.15.113:0/1331433182 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0048040 con 0x5622573a2a80 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75cbfff640 1 -- 172.21.15.113:0/1331433182 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f75d0004420 con 0x5622573a2a80 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.758+0000 7f75cbfff640 1 -- 172.21.15.113:0/1331433182 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d00436b0 con 0x5622573a2a80 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 -- 172.21.15.113:0/1331433182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 msgr2=0x5622573b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573b7d90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f75d00092c0 tx=0x7f75d0000bc0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 -- 172.21.15.113:0/1331433182 shutdown_connections 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5622573a2a80 0x5622573b7d90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399ea0 0x562257392bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 --2- 172.21.15.113:0/1331433182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562257399580 0x5622573a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 -- 172.21.15.113:0/1331433182 >> 172.21.15.113:0/1331433182 conn(0x562257393cb0 msgr2=0x5622573924e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 -- 172.21.15.113:0/1331433182 shutdown_connections 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 -- 172.21.15.113:0/1331433182 wait complete. 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 -- start start 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.759+0000 7f75eafcc740 1 Processor -- start 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x5622573b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 0x5622573b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622573a2a80 0x5622573b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01ae7a0 con 0x562257399ea0 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01afc10 con 0x562257399580 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f75e01aee70 con 0x5622573a2a80 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 10 client.0 _ll_register_callbacks cb 0x7f75d0046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 0x5622573b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 0x5622573b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58546/0 (socket says 172.21.15.113:58546) 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 -- 172.21.15.113:0/332021963 learned_addr learned my addr 172.21.15.113:0/332021963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e58a1640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x5622573b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 -- 172.21.15.113:0/332021963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622573a2a80 msgr2=0x5622573b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e58a1640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622573a2a80 0x5622573b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622573a2a80 0x5622573b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 -- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 msgr2=0x5622573b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x5622573b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 -- 172.21.15.113:0/332021963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562257420370 con 0x562257399ea0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75e60a2640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 0x5622573b5ce0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f75dc005fd0 tx=0x7f75dc0108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e515afa476caa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75c9ffb640 1 -- 172.21.15.113:0/332021963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc020030 con 0x562257399ea0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d0004420 con 0x562257399ea0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75c9ffb640 1 -- 172.21.15.113:0/332021963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f75dc01c6f0 con 0x562257399ea0 2024-04-22T07:18:40.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75c9ffb640 1 -- 172.21.15.113:0/332021963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f75dc02a050 con 0x562257399ea0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 10 client.16557 fetch_fsmap learned FSMap version 47 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.760+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562257421670 con 0x562257399ea0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75c9ffb640 1 -- 172.21.15.113:0/332021963 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f75d0004420 con 0x562257399ea0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75c9ffb640 1 -- 172.21.15.113:0/332021963 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f75dc004c40 con 0x562257399ea0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 10 client.16557 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 20 client.16557 populate_metadata read hostname 'smithi113' 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562257421ed0 0x5622574242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 4 client.16557 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562257305a70 con 0x562257421ed0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75e58a1640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562257421ed0 0x5622574242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75e58a1640 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562257421ed0 0x5622574242f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3fdb8439ca537f3 server_cookie=5f5aa2e3718ee8ab in_seq=0 out_seq=0 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75c9ffb640 10 client.16557 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75c9ffb640 1 -- 172.21.15.113:0/332021963 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562257305a70 con 0x562257421ed0 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75c9ffb640 10 client.16557 handle_command_reply: tid=0 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.761+0000 7f75eafcc740 1 client.16557 shutdown 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 msgr2=0x5622573b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 0x5622573b5ce0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f75dc005fd0 tx=0x7f75dc0108c0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 shutdown_connections 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562257421ed0 0x5622574242f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5622573a2a80 0x5622573b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562257399ea0 0x5622573b5ce0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 --2- 172.21.15.113:0/332021963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562257399580 0x5622573b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75c9ffb640 0 client.16557 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 >> 172.21.15.113:0/332021963 conn(0x562257393cb0 msgr2=0x5622573a03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 shutdown_connections 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 1 -- 172.21.15.113:0/332021963 wait complete. 2024-04-22T07:18:40.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75eafcc740 20 client.16557 trim_cache size 0 max 16384 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: "60592470-52d2-4d92-ad28-bb59bb968a1d": { 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "60592470-52d2-4d92-ad28-bb59bb968a1d", 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:18:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:18:40.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75cc0a3da0 msgr2=0x7f75cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75cc0a3da0 0x7f75cc0a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75d800efd0 tx=0x7f75d8087040 comp rx=0 tx=0).stop 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 msgr2=0x7f75e019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 0x7f75e019ad70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f75d800eaa0 tx=0x7f75d800ef60 comp rx=0 tx=0).stop 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 shutdown_connections 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f75cc0a3da0 0x7f75cc0a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75e019a8c0 0x7f75e019ad70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75e0106160 0x7f75e01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 --2- 172.21.15.113:0/1318317851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75e0101090 0x7f75e01a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 >> 172.21.15.113:0/1318317851 conn(0x7f75e00771d0 msgr2=0x7f75e00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 shutdown_connections 2024-04-22T07:18:40.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:40.762+0000 7f75e8306640 1 -- 172.21.15.113:0/1318317851 wait complete. 2024-04-22T07:18:40.771 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:40 smithi113 ceph-mon[27933]: pgmap v122: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-04-22T07:18:40.771 INFO:tasks.cephfs.filesystem:scrub status for tag:60592470-52d2-4d92-ad28-bb59bb968a1d - {'path': '/', 'tag': '60592470-52d2-4d92-ad28-bb59bb968a1d', 'options': 'recursive,force'} 2024-04-22T07:18:40.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:40 smithi133 ceph-mon[31198]: pgmap v122: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-04-22T07:18:41.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:40 smithi154 ceph-mon[31862]: pgmap v122: 97 pgs: 97 active+clean; 278 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-04-22T07:18:42.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:41 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:18:42.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:41 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:42.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:41 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:18:42.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:41 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:42.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:41 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:18:42.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:41 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:42.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:41 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:18:42.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:41 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:42.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:41 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:18:42.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:41 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:42.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:41 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:18:42.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:41 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:18:43.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:42 smithi133 ceph-mon[31198]: pgmap v123: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-04-22T07:18:43.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:42 smithi154 ceph-mon[31862]: pgmap v123: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-04-22T07:18:43.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:42 smithi113 ceph-mon[27933]: pgmap v123: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 409 op/s 2024-04-22T07:18:45.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:44 smithi133 ceph-mon[31198]: pgmap v124: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-04-22T07:18:45.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:44 smithi154 ceph-mon[31862]: pgmap v124: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-04-22T07:18:45.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:44 smithi113 ceph-mon[27933]: pgmap v124: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-04-22T07:18:46.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:45 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:18:46.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:45 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:18:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:45 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:18:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:46 smithi133 ceph-mon[31198]: pgmap v125: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-22T07:18:47.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:46 smithi154 ceph-mon[31862]: pgmap v125: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-22T07:18:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:46 smithi113 ceph-mon[27933]: pgmap v125: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-22T07:18:47.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:47.247+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:18:47.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:47.247+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:18:47.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:47.247+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:18:47.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:47.247+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:18:47.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:18:47.247+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:18:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:48 smithi133 ceph-mon[31198]: pgmap v126: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 440 KiB/s wr, 133 op/s 2024-04-22T07:18:49.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:48 smithi154 ceph-mon[31862]: pgmap v126: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 440 KiB/s wr, 133 op/s 2024-04-22T07:18:49.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:48 smithi113 ceph-mon[27933]: pgmap v126: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 440 KiB/s wr, 133 op/s 2024-04-22T07:18:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:50 smithi133 ceph-mon[31198]: pgmap v127: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 440 KiB/s wr, 133 op/s 2024-04-22T07:18:51.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:50 smithi154 ceph-mon[31862]: pgmap v127: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 440 KiB/s wr, 133 op/s 2024-04-22T07:18:51.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:50 smithi113 ceph-mon[27933]: pgmap v127: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 440 KiB/s wr, 133 op/s 2024-04-22T07:18:53.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:52 smithi133 ceph-mon[31198]: pgmap v128: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 443 KiB/s wr, 137 op/s 2024-04-22T07:18:53.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:52 smithi154 ceph-mon[31862]: pgmap v128: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 443 KiB/s wr, 137 op/s 2024-04-22T07:18:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:52 smithi113 ceph-mon[27933]: pgmap v128: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 443 KiB/s wr, 137 op/s 2024-04-22T07:18:55.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:54 smithi133 ceph-mon[31198]: pgmap v129: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-22T07:18:55.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:54 smithi154 ceph-mon[31862]: pgmap v129: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-22T07:18:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:54 smithi113 ceph-mon[27933]: pgmap v129: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-22T07:18:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:56 smithi133 ceph-mon[31198]: pgmap v130: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-22T07:18:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:56 smithi154 ceph-mon[31862]: pgmap v130: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-22T07:18:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:56 smithi113 ceph-mon[27933]: pgmap v130: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-04-22T07:18:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:58 smithi133 ceph-mon[31198]: pgmap v131: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-22T07:18:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:18:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:18:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:18:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:18:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:18:59.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:58 smithi154 ceph-mon[31862]: pgmap v131: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-22T07:18:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:18:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:18:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:18:59.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:18:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:18:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:58 smithi113 ceph-mon[27933]: pgmap v131: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-22T07:18:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:18:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:18:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:18:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:18:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:19:01.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:00 smithi133 ceph-mon[31198]: pgmap v132: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-22T07:19:01.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:00 smithi154 ceph-mon[31862]: pgmap v132: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-22T07:19:01.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:00 smithi113 ceph-mon[27933]: pgmap v132: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-22T07:19:02.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:02.250+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:02.251+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:19:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:02.251+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:19:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:02.251+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:19:02.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:02.251+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:19:03.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:02 smithi133 ceph-mon[31198]: pgmap v133: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-22T07:19:03.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:02 smithi154 ceph-mon[31862]: pgmap v133: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-22T07:19:03.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:02 smithi113 ceph-mon[27933]: pgmap v133: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-22T07:19:05.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:04 smithi133 ceph-mon[31198]: pgmap v134: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-22T07:19:05.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:04 smithi154 ceph-mon[31862]: pgmap v134: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-22T07:19:05.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:04 smithi113 ceph-mon[27933]: pgmap v134: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-22T07:19:07.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:06 smithi133 ceph-mon[31198]: pgmap v135: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T07:19:07.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:06 smithi154 ceph-mon[31862]: pgmap v135: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T07:19:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:06 smithi113 ceph-mon[27933]: pgmap v135: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T07:19:09.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:08 smithi133 ceph-mon[31198]: pgmap v136: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-22T07:19:09.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:08 smithi154 ceph-mon[31862]: pgmap v136: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-22T07:19:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:08 smithi113 ceph-mon[27933]: pgmap v136: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-22T07:19:10.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:19:10.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 -- 172.21.15.113:0/3949140466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241021a0 msgr2=0x7f7b24102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/3949140466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241021a0 0x7f7b24102580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009900 tx=0x7f7b1402bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 -- 172.21.15.113:0/3949140466 shutdown_connections 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/3949140466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b24107270 0x7f7b2410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/3949140466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b24102b50 0x7f7b24106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/3949140466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241021a0 0x7f7b24102580 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 -- 172.21.15.113:0/3949140466 >> 172.21.15.113:0/3949140466 conn(0x7f7b240fda40 msgr2=0x7f7b240ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 -- 172.21.15.113:0/3949140466 shutdown_connections 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.886+0000 7f7b2c02e640 1 -- 172.21.15.113:0/3949140466 wait complete. 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 Processor -- start 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 -- start start 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b241021a0 0x7f7b2419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b24107270 0x7f7b2419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 0x7f7b24198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241122f0 con 0x7f7b241986d0 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24112170 con 0x7f7b24107270 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2c02e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24112470 con 0x7f7b241021a0 2024-04-22T07:19:10.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 0x7f7b24198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 0x7f7b24198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54060/0 (socket says 172.21.15.113:54060) 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 -- 172.21.15.113:0/1501414896 learned_addr learned my addr 172.21.15.113:0/1501414896 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b24107270 0x7f7b2419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b29dca640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b241021a0 0x7f7b2419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 -- 172.21.15.113:0/1501414896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b241021a0 msgr2=0x7f7b2419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b241021a0 0x7f7b2419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 -- 172.21.15.113:0/1501414896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b24107270 msgr2=0x7f7b2419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b24107270 0x7f7b2419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.887+0000 7f7b2a5cb640 1 -- 172.21.15.113:0/1501414896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b241992d0 con 0x7f7b241986d0 2024-04-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.888+0000 7f7b2a5cb640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 0x7f7b24198b50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00eb10 tx=0x7f7b1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.888+0000 7f7b1affd640 1 -- 172.21.15.113:0/1501414896 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c004b20 con 0x7f7b241986d0 2024-04-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.888+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b241a5360 con 0x7f7b241986d0 2024-04-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.888+0000 7f7b1affd640 1 -- 172.21.15.113:0/1501414896 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7b1c004cc0 con 0x7f7b241986d0 2024-04-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.888+0000 7f7b1affd640 1 -- 172.21.15.113:0/1501414896 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c010720 con 0x7f7b241986d0 2024-04-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.888+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b241a5710 con 0x7f7b241986d0 2024-04-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b1affd640 1 -- 172.21.15.113:0/1501414896 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7b1c005020 con 0x7f7b241986d0 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 Processor -- start 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 -- start start 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aad9e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aada7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a2aad031e0 con 0x55a2aad9e580 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a2aaca0390 con 0x55a2aada7a80 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.890+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a2aacff1a0 con 0x55a2aad9eea0 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aad9e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b1affd640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7b100a3d20 0x7f7b100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aad9e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54062/0 (socket says 172.21.15.113:54062) 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 -- 172.21.15.113:0/1366779653 learned_addr learned my addr 172.21.15.113:0/1366779653 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:10.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b29dca640 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aada7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b2a5cb640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7b100a3d20 0x7f7b100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b1affd640 1 -- 172.21.15.113:0/1501414896 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f7b1c0d0e70 con 0x7f7b241986d0 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 -- 172.21.15.113:0/1366779653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 msgr2=0x55a2aada7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aada7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 -- 172.21.15.113:0/1366779653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 msgr2=0x55a2aada9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 -- 172.21.15.113:0/1366779653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2aaadf7d0 con 0x55a2aad9e580 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b29dca640 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aada7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aad9e960 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7b200135f0 tx=0x7f7b200099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3dbfbf98fc2f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b2a5cb640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7b100a3d20 0x7f7b100a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0cea40 tx=0x7f7b1c06e6d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b0ffff640 1 -- 172.21.15.113:0/1366779653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20004050 con 0x55a2aad9e580 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b0ffff640 1 -- 172.21.15.113:0/1366779653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7b200041f0 con 0x55a2aad9e580 2024-04-22T07:19:10.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.891+0000 7f7b0ffff640 1 -- 172.21.15.113:0/1366779653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000f760 con 0x55a2aad9e580 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1366779653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 msgr2=0x55a2aad9e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aad9e960 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7b200135f0 tx=0x7f7b200099d0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1366779653 shutdown_connections 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aada7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1366779653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aad9e960 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1366779653 >> 172.21.15.113:0/1366779653 conn(0x55a2aad98cb0 msgr2=0x55a2aad99090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1366779653 shutdown_connections 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.892+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1366779653 wait complete. 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 -- start start 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 Processor -- start 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aada4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aada4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a2aadaa870 con 0x55a2aad9eea0 2024-04-22T07:19:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a2aacbc2f0 con 0x55a2aada7a80 2024-04-22T07:19:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a2aadaa9f0 con 0x55a2aad9e580 2024-04-22T07:19:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b2ec16740 10 client.0 _ll_register_callbacks cb 0x55a2aacc9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aada4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aada4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54066/0 (socket says 172.21.15.113:54066) 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 -- 172.21.15.113:0/3357526411 learned_addr learned my addr 172.21.15.113:0/3357526411 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b295c9640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aada4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b295c9640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 -- 172.21.15.113:0/3357526411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 msgr2=0x55a2aada4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aada4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 -- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 msgr2=0x55a2aada5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.893+0000 7f7b29dca640 1 -- 172.21.15.113:0/3357526411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2aaa153f0 con 0x55a2aad9eea0 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b295c9640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aada4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b29dca640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aada4ff0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7b1402c170 tx=0x7f7b1402cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d3c061feaa8c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:10.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/3357526411 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14004460 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b200041f0 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/3357526411 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f7b14005030 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/3357526411 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7b14005230 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 10 client.16569 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a2aa9d50f0 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/3357526411 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7b200041f0 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/3357526411 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f7b140035f0 con 0x55a2aad9eea0 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 10 client.16569 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 20 client.16569 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aadb7b10 0x55a2aadb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 4 client.16569 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.894+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2aad10c00 con 0x55a2aadb7b10 2024-04-22T07:19:10.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.895+0000 7f7b295c9640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aadb7b10 0x55a2aadb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.895+0000 7f7b0d7fa640 10 client.16569 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.895+0000 7f7b295c9640 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aadb7b10 0x55a2aadb9f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc2c9bd938daac6 server_cookie=8361950a697b8532 in_seq=0 out_seq=0 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/3357526411 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2aad10c00 con 0x55a2aadb7b10 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b0d7fa640 10 client.16569 handle_command_reply: tid=0 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 client.16569 shutdown 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 msgr2=0x55a2aada4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aada4ff0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7b1402c170 tx=0x7f7b1402cda0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 shutdown_connections 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aadb7b10 0x55a2aadb9f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aada7a80 0x55a2aada5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b0d7fa640 0 client.16569 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aada4ff0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/3357526411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aada4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 >> 172.21.15.113:0/3357526411 conn(0x55a2aad98cb0 msgr2=0x55a2aad9aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:10.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 shutdown_connections 2024-04-22T07:19:10.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 1 -- 172.21.15.113:0/3357526411 wait complete. 2024-04-22T07:19:10.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.896+0000 7f7b2ec16740 20 client.16569 trim_cache size 0 max 16384 2024-04-22T07:19:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 Processor -- start 2024-04-22T07:19:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 -- start start 2024-04-22T07:19:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aada39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aadb3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aadb0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241b08a0 con 0x55a2aad9eea0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241ae110 con 0x55a2aad98cb0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.913+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241abf60 con 0x55a2aad9e580 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aadb0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aadb0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54076/0 (socket says 172.21.15.113:54076) 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 -- 172.21.15.113:0/585728953 learned_addr learned my addr 172.21.15.113:0/585728953 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aada39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 -- 172.21.15.113:0/585728953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 msgr2=0x55a2aadb3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aadb3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 -- 172.21.15.113:0/585728953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 msgr2=0x55a2aada39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aada39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 -- 172.21.15.113:0/585728953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2aadba560 con 0x55a2aad9eea0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b295c9640 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aadb0020 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7b2001c000 tx=0x7f7b20005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd29c6768ea61938 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b0f7fe640 1 -- 172.21.15.113:0/585728953 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200052f0 con 0x55a2aad9eea0 2024-04-22T07:19:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b0f7fe640 1 -- 172.21.15.113:0/585728953 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7b2002d870 con 0x55a2aad9eea0 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.914+0000 7f7b0f7fe640 1 -- 172.21.15.113:0/585728953 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20003c30 con 0x55a2aad9eea0 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 -- 172.21.15.113:0/585728953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 msgr2=0x55a2aadb0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aadb0020 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7b2001c000 tx=0x7f7b20005f90 comp rx=0 tx=0).stop 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 -- 172.21.15.113:0/585728953 shutdown_connections 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9eea0 0x55a2aadb0020 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9e580 0x55a2aadb3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/585728953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aada39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 -- 172.21.15.113:0/585728953 >> 172.21.15.113:0/585728953 conn(0x55a2aada0480 msgr2=0x55a2aad97d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 -- 172.21.15.113:0/585728953 shutdown_connections 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 -- 172.21.15.113:0/585728953 wait complete. 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 -- start start 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.915+0000 7f7b2ec16740 1 Processor -- start 2024-04-22T07:19:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aae1a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aae1a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aae1b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241ac5d0 con 0x55a2aad9e580 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241ada40 con 0x55a2aad98cb0 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241acca0 con 0x55a2aad9eea0 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 10 client.0 _ll_register_callbacks cb 0x7f7b20005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aae1a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aae1a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54086/0 (socket says 172.21.15.113:54086) 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 -- 172.21.15.113:0/1062830790 learned_addr learned my addr 172.21.15.113:0/1062830790 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aae1b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 -- 172.21.15.113:0/1062830790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 msgr2=0x55a2aae1b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aae1a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aae1b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 -- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 msgr2=0x55a2aae1a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aae1a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 -- 172.21.15.113:0/1062830790 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2aada9e70 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b29dca640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aae1a8d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7b14003a10 tx=0x7f7b14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e955e774684d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/1062830790 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14004460 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/1062830790 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f7b140053d0 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2002d870 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/1062830790 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7b14003e10 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 10 client.16575 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.916+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a2aae22ab0 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/1062830790 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7b2002d870 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/1062830790 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f7b14054460 con 0x55a2aad9e580 2024-04-22T07:19:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b2ec16740 10 client.16575 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:10.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b2ec16740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:10.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b2ec16740 20 client.16575 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:10.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aada7a80 0x55a2aae251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:10.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b2ec16740 4 client.16575 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:19:10.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2aad0aa70 con 0x55a2aada7a80 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.917+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aada7a80 0x55a2aae251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b0d7fa640 10 client.16575 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b295c9640 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aada7a80 0x55a2aae251c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1802c85add54d596 server_cookie=6dc8a125e676bfd7 in_seq=0 out_seq=0 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b0d7fa640 1 -- 172.21.15.113:0/1062830790 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2aad0aa70 con 0x55a2aada7a80 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b0d7fa640 10 client.16575 handle_command_reply: tid=0 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 client.16575 shutdown 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 msgr2=0x55a2aae1a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aae1a8d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7b14003a10 tx=0x7f7b14009780 comp rx=0 tx=0).stop 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 shutdown_connections 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55a2aada7a80 0x55a2aae251c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a2aad9eea0 0x55a2aae1b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a2aad9e580 0x55a2aae1a8d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b0d7fa640 0 client.16575 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:10.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 --2- 172.21.15.113:0/1062830790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a2aad98cb0 0x55a2aae1a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 >> 172.21.15.113:0/1062830790 conn(0x55a2aada0480 msgr2=0x55a2aad99e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 shutdown_connections 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 1 -- 172.21.15.113:0/1062830790 wait complete. 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.918+0000 7f7b2ec16740 20 client.16575 trim_cache size 0 max 16384 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:19:10.923 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:19:10.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7b100a3d20 msgr2=0x7f7b100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7b100a3d20 0x7f7b100a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0cea40 tx=0x7f7b1c06e6d0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 msgr2=0x7f7b24198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:10.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 0x7f7b24198b50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00eb10 tx=0x7f7b1c00c2d0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 shutdown_connections 2024-04-22T07:19:10.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7b100a3d20 0x7f7b100a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b241986d0 0x7f7b24198b50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b24107270 0x7f7b2419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 --2- 172.21.15.113:0/1501414896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b241021a0 0x7f7b2419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 >> 172.21.15.113:0/1501414896 conn(0x7f7b240fda40 msgr2=0x7f7b240fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 shutdown_connections 2024-04-22T07:19:10.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:10.919+0000 7f7b2c02e640 1 -- 172.21.15.113:0/1501414896 wait complete. 2024-04-22T07:19:10.927 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:19:10.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:19:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 -- 172.21.15.113:0/3484925378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c102190 msgr2=0x7f681c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 --2- 172.21.15.113:0/3484925378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c102190 0x7f681c102570 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f680c009900 tx=0x7f680c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 -- 172.21.15.113:0/3484925378 shutdown_connections 2024-04-22T07:19:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 --2- 172.21.15.113:0/3484925378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 0x7f681c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 --2- 172.21.15.113:0/3484925378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f681c102b40 0x7f681c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 --2- 172.21.15.113:0/3484925378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c102190 0x7f681c102570 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 -- 172.21.15.113:0/3484925378 >> 172.21.15.113:0/3484925378 conn(0x7f681c0793e0 msgr2=0x7f681c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 -- 172.21.15.113:0/3484925378 shutdown_connections 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 -- 172.21.15.113:0/3484925378 wait complete. 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 Processor -- start 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.028+0000 7f6822c5b640 1 -- start start 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f681c102190 0x7f681c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 0x7f681c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 0x7f681c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f681c112300 con 0x7f681c198640 2024-04-22T07:19:11.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f681c112180 con 0x7f681c107260 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f681c112480 con 0x7f681c102190 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 0x7f681c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 0x7f681c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54098/0 (socket says 172.21.15.113:54098) 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 -- 172.21.15.113:0/4454082 learned_addr learned my addr 172.21.15.113:0/4454082 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68209f7640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f681c102190 0x7f681c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f681bfff640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 0x7f681c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 -- 172.21.15.113:0/4454082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f681c102190 msgr2=0x7f681c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f681c102190 0x7f681c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 -- 172.21.15.113:0/4454082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 msgr2=0x7f681c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 0x7f681c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 -- 172.21.15.113:0/4454082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c199240 con 0x7f681c198640 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f681bfff640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 0x7f681c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f68211f8640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 0x7f681c198ac0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f681400cad0 tx=0x7f681400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681400ae20 con 0x7f681c198640 2024-04-22T07:19:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6814005020 con 0x7f681c198640 2024-04-22T07:19:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c1a5320 con 0x7f681c198640 2024-04-22T07:19:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.029+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681400f6b0 con 0x7f681c198640 2024-04-22T07:19:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.030+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c1a56d0 con 0x7f681c198640 2024-04-22T07:19:11.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.030+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681c103660 con 0x7f681c198640 2024-04-22T07:19:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.031+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f68140051d0 con 0x7f681c198640 2024-04-22T07:19:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.032+0000 7f6819ffb640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f68080a3d20 0x7f68080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.032+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f68140d09d0 con 0x7f681c198640 2024-04-22T07:19:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.032+0000 7f68209f7640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f68080a3d20 0x7f68080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.032+0000 7f68209f7640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f68080a3d20 0x7f68080a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f680c009230 tx=0x7f680c005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.037+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6814014030 con 0x7f681c198640 2024-04-22T07:19:11.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:10 smithi133 ceph-mon[31198]: pgmap v137: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-22T07:19:11.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:10 smithi154 ceph-mon[31862]: pgmap v137: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-22T07:19:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:10 smithi113 ceph-mon[27933]: pgmap v137: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-22T07:19:11.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.255+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f681c10a490 con 0x7f681c198640 2024-04-22T07:19:11.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.256+0000 7f6819ffb640 1 -- 172.21.15.113:0/4454082 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11894 (secure 0 0 0) 0x7f681409cef0 con 0x7f681c198640 2024-04-22T07:19:11.254 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:19:11.254 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:18:21.107438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:19:11.254 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f68080a3d20 msgr2=0x7f68080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f68080a3d20 0x7f68080a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f680c009230 tx=0x7f680c005fd0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 msgr2=0x7f681c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 0x7f681c198ac0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f681400cad0 tx=0x7f681400cf90 comp rx=0 tx=0).stop 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 shutdown_connections 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f68080a3d20 0x7f68080a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f681c198640 0x7f681c198ac0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c107260 0x7f681c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 --2- 172.21.15.113:0/4454082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f681c102190 0x7f681c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 >> 172.21.15.113:0/4454082 conn(0x7f681c0793e0 msgr2=0x7f681c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 shutdown_connections 2024-04-22T07:19:11.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.258+0000 7f6822c5b640 1 -- 172.21.15.113:0/4454082 wait complete. 2024-04-22T07:19:11.271 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 -- 172.21.15.113:0/1617694415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781072b0 msgr2=0x7f907810eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 --2- 172.21.15.113:0/1617694415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781072b0 0x7f907810eec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9070009ca0 tx=0x7f907002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 -- 172.21.15.113:0/1617694415 shutdown_connections 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 --2- 172.21.15.113:0/1617694415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781072b0 0x7f907810eec0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 --2- 172.21.15.113:0/1617694415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90781014a0 0x7f9078106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 --2- 172.21.15.113:0/1617694415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 0x7f9078100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 -- 172.21.15.113:0/1617694415 >> 172.21.15.113:0/1617694415 conn(0x7f90780760e0 msgr2=0x7f90780764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 -- 172.21.15.113:0/1617694415 shutdown_connections 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 -- 172.21.15.113:0/1617694415 wait complete. 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.373+0000 7f90800e4640 1 Processor -- start 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 -- start start 2024-04-22T07:19:11.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 0x7f9078101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90781072b0 0x7f9078101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 0x7f90781a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90781123f0 con 0x7f90781024e0 2024-04-22T07:19:11.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9078112270 con 0x7f9078100af0 2024-04-22T07:19:11.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9078112570 con 0x7f90781072b0 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 0x7f90781a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 0x7f90781a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54118/0 (socket says 172.21.15.113:54118) 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 -- 172.21.15.113:0/1892264206 learned_addr learned my addr 172.21.15.113:0/1892264206 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907de80640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 0x7f9078101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 -- 172.21.15.113:0/1892264206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90781072b0 msgr2=0x7f9078101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907d67f640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90781072b0 0x7f9078101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90781072b0 0x7f9078101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 -- 172.21.15.113:0/1892264206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 msgr2=0x7f9078101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 0x7f9078101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 -- 172.21.15.113:0/1892264206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90781a7180 con 0x7f90781024e0 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907de80640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 0x7f9078101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f907e681640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 0x7f90781a6a80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f907002c700 tx=0x7f9070035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f906affd640 1 -- 172.21.15.113:0/1892264206 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070039050 con 0x7f90781024e0 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f906affd640 1 -- 172.21.15.113:0/1892264206 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9070045070 con 0x7f90781024e0 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781a7410 con 0x7f90781024e0 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.374+0000 7f906affd640 1 -- 172.21.15.113:0/1892264206 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070004680 con 0x7f90781024e0 2024-04-22T07:19:11.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.375+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90781a7900 con 0x7f90781024e0 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.377+0000 7f906affd640 1 -- 172.21.15.113:0/1892264206 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9070003970 con 0x7f90781024e0 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.377+0000 7f9082db4740 1 Processor -- start 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.377+0000 7f9082db4740 1 -- start start 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9082db4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d78d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9082db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d796430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9082db4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d798e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9082db4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55924d6f21f0 con 0x55924d796a00 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9082db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55924d68f4b0 con 0x55924d78de80 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9082db4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55924d6ee1b0 con 0x55924d78d4d0 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f906affd640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90640a3d50 0x7f90640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d798e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d798e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54126/0 (socket says 172.21.15.113:54126) 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 -- 172.21.15.113:0/3830409207 learned_addr learned my addr 172.21.15.113:0/3830409207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d78d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907e681640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90640a3d50 0x7f90640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907de80640 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d796430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f906affd640 1 -- 172.21.15.113:0/1892264206 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f90700f3300 con 0x7f90781024e0 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 -- 172.21.15.113:0/3830409207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 msgr2=0x55924d78d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d78d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 -- 172.21.15.113:0/3830409207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 msgr2=0x55924d796430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d796430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 -- 172.21.15.113:0/3830409207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924d6811b0 con 0x55924d796a00 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907e681640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90640a3d50 0x7f90640a6210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90700f0a90 tx=0x7f907002c0f0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f907d67f640 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d798e50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f907400cf30 tx=0x7f907400c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29ad442bb631d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9063fff640 1 -- 172.21.15.113:0/3830409207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907401d070 con 0x55924d796a00 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9063fff640 1 -- 172.21.15.113:0/3830409207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9074018b40 con 0x55924d796a00 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.378+0000 7f9063fff640 1 -- 172.21.15.113:0/3830409207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907400f040 con 0x55924d796a00 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 -- 172.21.15.113:0/3830409207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 msgr2=0x55924d798e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d798e50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f907400cf30 tx=0x7f907400c170 comp rx=0 tx=0).stop 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 -- 172.21.15.113:0/3830409207 shutdown_connections 2024-04-22T07:19:11.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d798e50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d796430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 --2- 172.21.15.113:0/3830409207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d78d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 -- 172.21.15.113:0/3830409207 >> 172.21.15.113:0/3830409207 conn(0x55924d787ba0 msgr2=0x55924d787f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 -- 172.21.15.113:0/3830409207 shutdown_connections 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.379+0000 7f9082db4740 1 -- 172.21.15.113:0/3830409207 wait complete. 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 -- start start 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 Processor -- start 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d796360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d7936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d793d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55924d799970 con 0x55924d78de80 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55924d6ab500 con 0x55924d796a00 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55924d799af0 con 0x55924d78d4d0 2024-04-22T07:19:11.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f9082db4740 10 client.0 _ll_register_callbacks cb 0x55924d679b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d7936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d7936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54128/0 (socket says 172.21.15.113:54128) 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 -- 172.21.15.113:0/1942123159 learned_addr learned my addr 172.21.15.113:0/1942123159 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907d67f640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d793d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 -- 172.21.15.113:0/1942123159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 msgr2=0x55924d796360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907d67f640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d796360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d796360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 -- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 msgr2=0x55924d793d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d793d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907de80640 1 -- 172.21.15.113:0/1942123159 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924d4043f0 con 0x55924d78de80 2024-04-22T07:19:11.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.380+0000 7f907d67f640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d796360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f907de80640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d7936f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f906c00bf00 tx=0x7f906c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a6712418036f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f90617fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f90617fa640 1 -- 172.21.15.113:0/1942123159 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c004440 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f90617fa640 1 -- 172.21.15.113:0/1942123159 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f906c0035f0 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9074018b40 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f90617fa640 1 -- 172.21.15.113:0/1942123159 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f906c0037f0 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 10 client.16593 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55924d4ce7d0 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f90617fa640 1 -- 172.21.15.113:0/1942123159 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9074018b40 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f90617fa640 1 -- 172.21.15.113:0/1942123159 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f906c01bcb0 con 0x55924d78de80 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 10 client.16593 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:11.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 20 client.16593 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d7a6d50 0x55924d7a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 4 client.16593 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.381+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55924d691930 con 0x55924d7a6d50 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.382+0000 7f907d67f640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d7a6d50 0x55924d7a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.382+0000 7f907d67f640 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d7a6d50 0x55924d7a9170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f474259a98d4bb0b server_cookie=923f53452decc280 in_seq=0 out_seq=0 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.382+0000 7f90617fa640 10 client.16593 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f90617fa640 1 -- 172.21.15.113:0/1942123159 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55924d691930 con 0x55924d7a6d50 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f90617fa640 10 client.16593 handle_command_reply: tid=0 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 client.16593 shutdown 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 msgr2=0x55924d7936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d7936f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f906c00bf00 tx=0x7f906c0117b0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 shutdown_connections 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d7a6d50 0x55924d7a9170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d793d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d7936f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 --2- 172.21.15.113:0/1942123159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d796360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f90617fa640 0 client.16593 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 >> 172.21.15.113:0/1942123159 conn(0x55924d787ba0 msgr2=0x55924d7898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 shutdown_connections 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 1 -- 172.21.15.113:0/1942123159 wait complete. 2024-04-22T07:19:11.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.383+0000 7f9082db4740 20 client.16593 trim_cache size 0 max 16384 2024-04-22T07:19:11.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 Processor -- start 2024-04-22T07:19:11.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 -- start start 2024-04-22T07:19:11.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d7a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d792850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d78eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90781b29b0 con 0x55924d796a00 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90781b0220 con 0x55924d78de80 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f9082db4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90781ae090 con 0x55924d78d4d0 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d78eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d78eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54138/0 (socket says 172.21.15.113:54138) 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 -- 172.21.15.113:0/4161869799 learned_addr learned my addr 172.21.15.113:0/4161869799 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d7a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 -- 172.21.15.113:0/4161869799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 msgr2=0x55924d7a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d7a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 -- 172.21.15.113:0/4161869799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 msgr2=0x55924d792850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.396+0000 7f907d67f640 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d792850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f907d67f640 1 -- 172.21.15.113:0/4161869799 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924d78dc20 con 0x55924d796a00 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f907d67f640 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d78eae0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f90740155f0 tx=0x7f907400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0e247b84c48a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f90637fe640 1 -- 172.21.15.113:0/4161869799 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907401d040 con 0x55924d796a00 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f90637fe640 1 -- 172.21.15.113:0/4161869799 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9074028e50 con 0x55924d796a00 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f90637fe640 1 -- 172.21.15.113:0/4161869799 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074018470 con 0x55924d796a00 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 -- 172.21.15.113:0/4161869799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 msgr2=0x55924d78eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d78eae0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f90740155f0 tx=0x7f907400ce50 comp rx=0 tx=0).stop 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 -- 172.21.15.113:0/4161869799 shutdown_connections 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d796a00 0x55924d78eae0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d78de80 0x55924d792850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 --2- 172.21.15.113:0/4161869799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d7a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 -- 172.21.15.113:0/4161869799 >> 172.21.15.113:0/4161869799 conn(0x55924d787ba0 msgr2=0x55924d78e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 -- 172.21.15.113:0/4161869799 shutdown_connections 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.397+0000 7f9082db4740 1 -- 172.21.15.113:0/4161869799 wait complete. 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 -- start start 2024-04-22T07:19:11.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 Processor -- start 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d79ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d79a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d79a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90781ae6e0 con 0x55924d78de80 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90781afb50 con 0x55924d796a00 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90781aedb0 con 0x55924d78d4d0 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f9082db4740 10 client.0 _ll_register_callbacks cb 0x7f906c00bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f907de80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d79a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f907de80640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d79a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54146/0 (socket says 172.21.15.113:54146) 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f907de80640 1 -- 172.21.15.113:0/826248388 learned_addr learned my addr 172.21.15.113:0/826248388 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f907d67f640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d79a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f907d67f640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d79ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.398+0000 7f907de80640 1 -- 172.21.15.113:0/826248388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 msgr2=0x55924d79ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f907de80640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d79ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f907de80640 1 -- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 msgr2=0x55924d79a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f907de80640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d79a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f907de80640 1 -- 172.21.15.113:0/826248388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924d79ae80 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f907d67f640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d79a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f907de80640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d79a300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f906c012690 tx=0x7f906c0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e12d903567fe603 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f90617fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f90617fa640 1 -- 172.21.15.113:0/826248388 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c003990 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f90617fa640 1 -- 172.21.15.113:0/826248388 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f906c003b30 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9074028e50 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f90617fa640 1 -- 172.21.15.113:0/826248388 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f906c037050 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 10 client.16599 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55924d80d140 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f90617fa640 1 -- 172.21.15.113:0/826248388 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9074028e50 con 0x55924d78de80 2024-04-22T07:19:11.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f90617fa640 1 -- 172.21.15.113:0/826248388 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f906c032030 con 0x55924d78de80 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 10 client.16599 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 20 client.16599 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d80d880 0x55924d80fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 4 client.16599 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.399+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55924d6ffc10 con 0x55924d80d880 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f907d67f640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d80d880 0x55924d80fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f90617fa640 10 client.16599 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f907d67f640 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d80d880 0x55924d80fca0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3cb0e3c4134eaa server_cookie=2af728f065659664 in_seq=0 out_seq=0 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f90617fa640 1 -- 172.21.15.113:0/826248388 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55924d6ffc10 con 0x55924d80d880 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f90617fa640 10 client.16599 handle_command_reply: tid=0 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 client.16599 shutdown 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 msgr2=0x55924d79a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d79a300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f906c012690 tx=0x7f906c0044f0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 shutdown_connections 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55924d80d880 0x55924d80fca0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924d796a00 0x55924d79a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55924d78de80 0x55924d79a300 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 --2- 172.21.15.113:0/826248388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55924d78d4d0 0x55924d79ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f90617fa640 0 client.16599 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 >> 172.21.15.113:0/826248388 conn(0x55924d787ba0 msgr2=0x55924d786c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 shutdown_connections 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 1 -- 172.21.15.113:0/826248388 wait complete. 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.400+0000 7f9082db4740 20 client.16599 trim_cache size 0 max 16384 2024-04-22T07:19:11.405 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:19:11.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90640a3d50 msgr2=0x7f90640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90640a3d50 0x7f90640a6210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90700f0a90 tx=0x7f907002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 msgr2=0x7f90781a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 0x7f90781a6a80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f907002c700 tx=0x7f9070035d80 comp rx=0 tx=0).stop 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 shutdown_connections 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90640a3d50 0x7f90640a6210 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90781024e0 0x7f90781a6a80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90781072b0 0x7f9078101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 --2- 172.21.15.113:0/1892264206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078100af0 0x7f9078101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 >> 172.21.15.113:0/1892264206 conn(0x7f90780760e0 msgr2=0x7f90780feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 shutdown_connections 2024-04-22T07:19:11.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.401+0000 7f90800e4640 1 -- 172.21.15.113:0/1892264206 wait complete. 2024-04-22T07:19:11.409 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:19:11.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 -- 172.21.15.113:0/355445549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 msgr2=0x7f5cbc102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/355445549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc102570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009900 tx=0x7f5cac02bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 -- 172.21.15.113:0/355445549 shutdown_connections 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/355445549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 0x7f5cbc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/355445549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc102b40 0x7f5cbc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/355445549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc102570 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 -- 172.21.15.113:0/355445549 >> 172.21.15.113:0/355445549 conn(0x7f5cbc0793a0 msgr2=0x7f5cbc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.512+0000 7f5cc1d84640 1 -- 172.21.15.113:0/355445549 shutdown_connections 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 -- 172.21.15.113:0/355445549 wait complete. 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 Processor -- start 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 -- start start 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 0x7f5cbc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc1986a0 0x7f5cbc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc112300 con 0x7f5cbc102190 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc112180 con 0x7f5cbc1986a0 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.513+0000 7f5cc1d84640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc112480 con 0x7f5cbc107260 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54162/0 (socket says 172.21.15.113:54162) 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/1854372441 learned_addr learned my addr 172.21.15.113:0/1854372441 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc1986a0 0x7f5cbc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 0x7f5cbc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/1854372441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 msgr2=0x7f5cbc19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 0x7f5cbc19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/1854372441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc1986a0 msgr2=0x7f5cbc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc1986a0 0x7f5cbc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/1854372441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc1992a0 con 0x7f5cbc102190 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 0x7f5cbc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc19e5e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006010 tx=0x7f5cac02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cb8ff9640 1 -- 172.21.15.113:0/1854372441 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac041070 con 0x7f5cbc102190 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cb8ff9640 1 -- 172.21.15.113:0/1854372441 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5cac0052f0 con 0x7f5cbc102190 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc199530 con 0x7f5cbc102190 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cb8ff9640 1 -- 172.21.15.113:0/1854372441 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003db0 con 0x7f5cbc102190 2024-04-22T07:19:11.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.514+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc1a56d0 con 0x7f5cbc102190 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.516+0000 7f5cb8ff9640 1 -- 172.21.15.113:0/1854372441 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5cac044070 con 0x7f5cbc102190 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 Processor -- start 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 -- start start 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73beef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73befae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c73be541f0 con 0x55c73beefe80 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c73bdf14b0 con 0x55c73bef8a00 2024-04-22T07:19:11.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cc4979740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c73be501b0 con 0x55c73beef4d0 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cb8ff9640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ca40a3d20 0x7f5ca40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54170/0 (socket says 172.21.15.113:54170) 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/166404247 learned_addr learned my addr 172.21.15.113:0/166404247 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73befae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ca40a3d20 0x7f5ca40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73beef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cb8ff9640 1 -- 172.21.15.113:0/1854372441 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5cac0f2770 con 0x7f5cbc102190 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbaffd640 1 -- 172.21.15.113:0/166404247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 msgr2=0x55c73beef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73beef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbaffd640 1 -- 172.21.15.113:0/166404247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 msgr2=0x55c73bef8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbaffd640 1 -- 172.21.15.113:0/166404247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73bde31b0 con 0x55c73bef8a00 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.517+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:19:11.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.518+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ca40a3d20 0x7f5ca40a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005cc0 tx=0x7f5cb4005bd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.518+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73befae50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc199ef0 tx=0x7f5cb000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bddd6ace2fab5b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.518+0000 7f5ca1ffb640 1 -- 172.21.15.113:0/166404247 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0015070 con 0x55c73bef8a00 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.518+0000 7f5ca1ffb640 1 -- 172.21.15.113:0/166404247 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5cb0011910 con 0x55c73bef8a00 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.518+0000 7f5ca1ffb640 1 -- 172.21.15.113:0/166404247 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00109c0 con 0x55c73bef8a00 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- 172.21.15.113:0/166404247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 msgr2=0x55c73befae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73befae50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc199ef0 tx=0x7f5cb000f730 comp rx=0 tx=0).stop 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- 172.21.15.113:0/166404247 shutdown_connections 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73befae50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- 172.21.15.113:0/166404247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73beef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- 172.21.15.113:0/166404247 >> 172.21.15.113:0/166404247 conn(0x55c73bee9ba0 msgr2=0x55c73bee9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- 172.21.15.113:0/166404247 shutdown_connections 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- 172.21.15.113:0/166404247 wait complete. 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- start start 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 Processor -- start 2024-04-22T07:19:11.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bef5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c73befb970 con 0x55c73beefe80 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c73be0d500 con 0x55c73bef8a00 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c73befbaf0 con 0x55c73beef4d0 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.519+0000 7f5cc4979740 10 client.0 _ll_register_callbacks cb 0x55c73bddbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54174/0 (socket says 172.21.15.113:54174) 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 -- 172.21.15.113:0/1913472965 learned_addr learned my addr 172.21.15.113:0/1913472965 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bef5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 -- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 msgr2=0x55c73bef8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 -- 172.21.15.113:0/1913472965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 msgr2=0x55c73bef5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bef5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 -- 172.21.15.113:0/1913472965 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73bb663f0 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef56d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000b9f0 tx=0x7f5cb000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a26411a8279d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5c9affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5c9affd640 1 -- 172.21.15.113:0/1913472965 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0015070 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5c9affd640 1 -- 172.21.15.113:0/1913472965 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5cb0011650 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb0011910 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5c9affd640 1 -- 172.21.15.113:0/1913472965 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5cb002e050 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cc4979740 10 client.16611 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c73bc307d0 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.520+0000 7f5c9affd640 1 -- 172.21.15.113:0/1913472965 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5cb0011910 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5c9affd640 1 -- 172.21.15.113:0/1913472965 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb0005160 con 0x55c73beefe80 2024-04-22T07:19:11.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cc4979740 10 client.16611 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cc4979740 10 client.16611 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cc4979740 20 client.16611 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cc4979740 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf08b70 0x55c73bf0af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cc4979740 4 client.16611 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73bdf3930 con 0x55c73bf08b70 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf08b70 0x55c73bf0af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5c9affd640 10 client.16611 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.521+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf08b70 0x55c73bf0af60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4722b0495068315 server_cookie=6056f9b2daec3029 in_seq=0 out_seq=0 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5c9affd640 1 -- 172.21.15.113:0/1913472965 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c73bdf3930 con 0x55c73bf08b70 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5c9affd640 10 client.16611 handle_command_reply: tid=0 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 client.16611 shutdown 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 msgr2=0x55c73bef56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef56d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000b9f0 tx=0x7f5cb000bd40 comp rx=0 tx=0).stop 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 shutdown_connections 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf08b70 0x55c73bf0af60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bef5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef56d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5c9affd640 0 client.16611 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 --2- 172.21.15.113:0/1913472965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 >> 172.21.15.113:0/1913472965 conn(0x55c73bee9ba0 msgr2=0x55c73beeb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.522+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 shutdown_connections 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.523+0000 7f5cc4979740 1 -- 172.21.15.113:0/1913472965 wait complete. 2024-04-22T07:19:11.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.523+0000 7f5cc4979740 20 client.16611 trim_cache size 0 max 16384 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 Processor -- start 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 -- start start 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bf10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1b0890 con 0x55c73beefe80 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1ae100 con 0x55c73bef8a00 2024-04-22T07:19:11.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cc4979740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1abf70 con 0x55c73beef4d0 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54182/0 (socket says 172.21.15.113:54182) 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 -- 172.21.15.113:0/923122990 learned_addr learned my addr 172.21.15.113:0/923122990 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bf10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 -- 172.21.15.113:0/923122990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 msgr2=0x55c73bef7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 -- 172.21.15.113:0/923122990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 msgr2=0x55c73bf10590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bf10590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 -- 172.21.15.113:0/923122990 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73bef6050 con 0x55c73beefe80 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8340 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000ba40 tx=0x7f5cb000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fea66598c3531b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5ca17fa640 1 -- 172.21.15.113:0/923122990 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0015040 con 0x55c73beefe80 2024-04-22T07:19:11.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.536+0000 7f5ca17fa640 1 -- 172.21.15.113:0/923122990 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5cb00106b0 con 0x55c73beefe80 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5ca17fa640 1 -- 172.21.15.113:0/923122990 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0004680 con 0x55c73beefe80 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 -- 172.21.15.113:0/923122990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 msgr2=0x55c73bef8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8340 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000ba40 tx=0x7f5cb000b560 comp rx=0 tx=0).stop 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 -- 172.21.15.113:0/923122990 shutdown_connections 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73bef8a00 0x55c73bf10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beefe80 0x55c73bef8340 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 --2- 172.21.15.113:0/923122990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73beef4d0 0x55c73bef7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 -- 172.21.15.113:0/923122990 >> 172.21.15.113:0/923122990 conn(0x55c73bee9ba0 msgr2=0x55c73bef0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 -- 172.21.15.113:0/923122990 shutdown_connections 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 -- 172.21.15.113:0/923122990 wait complete. 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.537+0000 7f5cc4979740 1 -- start start 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 Processor -- start 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 0x55c73befc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73beefe80 0x55c73befcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73bef8a00 0x55c73befd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1ac5c0 con 0x55c73beef4d0 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1ada30 con 0x55c73beefe80 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1acc90 con 0x55c73bef8a00 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cc4979740 10 client.0 _ll_register_callbacks cb 0x7f5cb000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 0x55c73befc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 0x55c73befc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54186/0 (socket says 172.21.15.113:54186) 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/3235325486 learned_addr learned my addr 172.21.15.113:0/3235325486 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73beefe80 0x55c73befcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73bef8a00 0x55c73befd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73bef8a00 msgr2=0x55c73befd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73bef8a00 0x55c73befd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/3235325486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73beefe80 msgr2=0x55c73befcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73beefe80 0x55c73befcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 -- 172.21.15.113:0/3235325486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73bf6de30 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbbfff640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73bef8a00 0x55c73befd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5cbb7fe640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 0x55c73befc7c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5cac005940 tx=0x7f5cac02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898c7ed0278370c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5c9affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5c9affd640 1 -- 172.21.15.113:0/3235325486 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac04b040 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.538+0000 7f5c9affd640 1 -- 172.21.15.113:0/3235325486 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5cac0054a0 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb00106b0 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5c9affd640 1 -- 172.21.15.113:0/3235325486 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5cac0be640 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 10 client.16617 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c73bf6f0f0 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5c9affd640 1 -- 172.21.15.113:0/3235325486 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5cac0d5020 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5c9affd640 1 -- 172.21.15.113:0/3235325486 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f5cac0be8a0 con 0x55c73beef4d0 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 10 client.16617 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 10 client.16617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 20 client.16617 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:11.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf6f820 0x55c73bf71c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 4 client.16617 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c73be61c10 con 0x55c73bf6f820 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.539+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf6f820 0x55c73bf71c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5c9affd640 10 client.16617 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cbaffd640 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf6f820 0x55c73bf71c40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44b4841dbbca9619 server_cookie=2adc0262f8d4162 in_seq=0 out_seq=0 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5c9affd640 1 -- 172.21.15.113:0/3235325486 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c73be61c10 con 0x55c73bf6f820 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5c9affd640 10 client.16617 handle_command_reply: tid=0 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 client.16617 shutdown 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 msgr2=0x55c73befc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 0x55c73befc7c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5cac005940 tx=0x7f5cac02bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 shutdown_connections 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55c73bf6f820 0x55c73bf71c40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73bef8a00 0x55c73befd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5c9affd640 0 client.16617 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73beefe80 0x55c73befcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 --2- 172.21.15.113:0/3235325486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73beef4d0 0x55c73befc7c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 >> 172.21.15.113:0/3235325486 conn(0x55c73bee9ba0 msgr2=0x55c73bef0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 shutdown_connections 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 1 -- 172.21.15.113:0/3235325486 wait complete. 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.540+0000 7f5cc4979740 20 client.16617 trim_cache size 0 max 16384 2024-04-22T07:19:11.545 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ca40a3d20 msgr2=0x7f5ca40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ca40a3d20 0x7f5ca40a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4005cc0 tx=0x7f5cb4005bd0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 msgr2=0x7f5cbc19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc19e5e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006010 tx=0x7f5cac02c710 comp rx=0 tx=0).stop 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 shutdown_connections 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5ca40a3d20 0x7f5ca40a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc1986a0 0x7f5cbc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cbc107260 0x7f5cbc19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 --2- 172.21.15.113:0/1854372441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cbc102190 0x7f5cbc19e5e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 >> 172.21.15.113:0/1854372441 conn(0x7f5cbc0793a0 msgr2=0x7f5cbc0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 shutdown_connections 2024-04-22T07:19:11.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.541+0000 7f5cc1d84640 1 -- 172.21.15.113:0/1854372441 wait complete. 2024-04-22T07:19:11.550 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:19:11.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3514636954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102bc0 msgr2=0x7fa1e8106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3514636954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102bc0 0x7fa1e8106c00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009900 tx=0x7fa1d802bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:11.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3514636954 shutdown_connections 2024-04-22T07:19:11.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3514636954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e81072e0 0x7fa1e810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3514636954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102bc0 0x7fa1e8106c00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3514636954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e8102210 0x7fa1e81025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3514636954 >> 172.21.15.113:0/3514636954 conn(0x7fa1e80fdab0 msgr2=0x7fa1e80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3514636954 shutdown_connections 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3514636954 wait complete. 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 Processor -- start 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.653+0000 7fa1ef45c640 1 -- start start 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 0x7fa1e819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e81072e0 0x7fa1e819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e8198760 0x7fa1e8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8112340 con 0x7fa1e8102210 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81121c0 con 0x7fa1e81072e0 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81124c0 con 0x7fa1e8198760 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 0x7fa1e819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 0x7fa1e819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54208/0 (socket says 172.21.15.113:54208) 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 -- 172.21.15.113:0/3632812937 learned_addr learned my addr 172.21.15.113:0/3632812937 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e8198760 0x7fa1e8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 -- 172.21.15.113:0/3632812937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e8198760 msgr2=0x7fa1e8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e8198760 0x7fa1e8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 -- 172.21.15.113:0/3632812937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e81072e0 msgr2=0x7fa1e819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e81072e0 0x7fa1e819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 -- 172.21.15.113:0/3632812937 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e8199360 con 0x7fa1e8102210 2024-04-22T07:19:11.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ed1f8640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 0x7fa1e819e650 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400c920 tx=0x7fa1e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1de7fc640 1 -- 172.21.15.113:0/3632812937 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4007c70 con 0x7fa1e8102210 2024-04-22T07:19:11.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e81a53f0 con 0x7fa1e8102210 2024-04-22T07:19:11.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1de7fc640 1 -- 172.21.15.113:0/3632812937 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1e4007e10 con 0x7fa1e8102210 2024-04-22T07:19:11.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1de7fc640 1 -- 172.21.15.113:0/3632812937 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40188c0 con 0x7fa1e8102210 2024-04-22T07:19:11.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.654+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a57f0 con 0x7fa1e8102210 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.656+0000 7fa1de7fc640 1 -- 172.21.15.113:0/3632812937 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa1e4018a60 con 0x7fa1e8102210 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 Processor -- start 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 -- start start 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1de7fc640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561ae54411f0 con 0x561ae54dce80 2024-04-22T07:19:11.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561ae53de4b0 con 0x561ae54e5a00 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.657+0000 7fa1f2053740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561ae543d1b0 con 0x561ae54dc4d0 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ed1f8640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54210/0 (socket says 172.21.15.113:54210) 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/2039661083 learned_addr learned my addr 172.21.15.113:0/2039661083 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1de7fc640 1 -- 172.21.15.113:0/3632812937 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fa1e40d0be0 con 0x7fa1e8102210 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/2039661083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 msgr2=0x561ae54dc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54dc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/2039661083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 msgr2=0x561ae54e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/2039661083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ae53d01b0 con 0x561ae54dce80 2024-04-22T07:19:11.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ed1f8640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4099370 tx=0x7fa1e409c7e0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54e5430 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa1d802c170 tx=0x7fa1d8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7706ef40f55421 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1d37fe640 1 -- 172.21.15.113:0/2039661083 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8041020 con 0x561ae54dce80 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1d37fe640 1 -- 172.21.15.113:0/2039661083 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1d8004040 con 0x561ae54dce80 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.658+0000 7fa1d37fe640 1 -- 172.21.15.113:0/2039661083 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8003af0 con 0x561ae54dce80 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 -- 172.21.15.113:0/2039661083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 msgr2=0x561ae54e5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54e5430 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa1d802c170 tx=0x7fa1d8009560 comp rx=0 tx=0).stop 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 -- 172.21.15.113:0/2039661083 shutdown_connections 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54e5430 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 --2- 172.21.15.113:0/2039661083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 -- 172.21.15.113:0/2039661083 >> 172.21.15.113:0/2039661083 conn(0x561ae54d6ba0 msgr2=0x561ae54d6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 -- 172.21.15.113:0/2039661083 shutdown_connections 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 -- 172.21.15.113:0/2039661083 wait complete. 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 -- start start 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.659+0000 7fa1f2053740 1 Processor -- start 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dc4d0 0x561ae54e5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dce80 0x561ae54e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 0x561ae54e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561ae54e8910 con 0x561ae54e5a00 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561ae53fa500 con 0x561ae54dc4d0 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561ae54e8a90 con 0x561ae54dce80 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 10 client.0 _ll_register_callbacks cb 0x561ae53c8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 0x561ae54e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 0x561ae54e2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54218/0 (socket says 172.21.15.113:54218) 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/1121135605 learned_addr learned my addr 172.21.15.113:0/1121135605 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dc4d0 0x561ae54e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dce80 0x561ae54e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/1121135605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dce80 msgr2=0x561ae54e2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dce80 0x561ae54e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dc4d0 msgr2=0x561ae54e5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dc4d0 0x561ae54e5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/1121135605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ae51533f0 con 0x561ae54e5a00 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 0x561ae54e2b40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00106b0 tx=0x7fa1e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca05a7214a484ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/1121135605 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0018030 con 0x561ae54e5a00 2024-04-22T07:19:11.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/1121135605 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fa1e0004d20 con 0x561ae54e5a00 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.660+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8004040 con 0x561ae54e5a00 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/1121135605 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa1e002b050 con 0x561ae54e5a00 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 10 client.16629 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561ae521d7d0 con 0x561ae54e5a00 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/1121135605 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1d8004040 con 0x561ae54e5a00 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/1121135605 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7fa1e0005090 con 0x561ae54e5a00 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 10 client.16629 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 10 client.16629 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 20 client.16629 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae54f8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 4 client.16629 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:11.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ae53e0930 con 0x561ae54f5c10 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.661+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae54f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.662+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae54f8060 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a6e4eed9647fe28 server_cookie=6b7263d1182683df in_seq=0 out_seq=0 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.662+0000 7fa1d0ff9640 10 client.16629 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.662+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/1121135605 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ae53e0930 con 0x561ae54f5c10 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.662+0000 7fa1d0ff9640 10 client.16629 handle_command_reply: tid=0 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.662+0000 7fa1f2053740 1 client.16629 shutdown 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 msgr2=0x561ae54e2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 0x561ae54e2b40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00106b0 tx=0x7fa1e0004a40 comp rx=0 tx=0).stop 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 shutdown_connections 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae54f8060 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54e5a00 0x561ae54e2b40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1d0ff9640 0 client.16629 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:11.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dce80 0x561ae54e2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 --2- 172.21.15.113:0/1121135605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dc4d0 0x561ae54e5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 >> 172.21.15.113:0/1121135605 conn(0x561ae54d6ba0 msgr2=0x561ae54d88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 shutdown_connections 2024-04-22T07:19:11.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 1 -- 172.21.15.113:0/1121135605 wait complete. 2024-04-22T07:19:11.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.663+0000 7fa1f2053740 20 client.16629 trim_cache size 0 max 16384 2024-04-22T07:19:11.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 Processor -- start 2024-04-22T07:19:11.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 -- start start 2024-04-22T07:19:11.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81b0740 con 0x561ae54dce80 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81adfb0 con 0x561ae54e5a00 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.676+0000 7fa1f2053740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81abe20 con 0x561ae54dc4d0 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54220/0 (socket says 172.21.15.113:54220) 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/3396309067 learned_addr learned my addr 172.21.15.113:0/3396309067 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/3396309067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 msgr2=0x561ae54f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/3396309067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 msgr2=0x561ae54e4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 -- 172.21.15.113:0/3396309067 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ae54e3140 con 0x561ae54dce80 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54d5a80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8003700 tx=0x7fa1d8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d3047f4c4d94da server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1d2ffd640 1 -- 172.21.15.113:0/3396309067 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8052070 con 0x561ae54dce80 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1d2ffd640 1 -- 172.21.15.113:0/3396309067 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1d8005850 con 0x561ae54dce80 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.677+0000 7fa1d2ffd640 1 -- 172.21.15.113:0/3396309067 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8041040 con 0x561ae54dce80 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 -- 172.21.15.113:0/3396309067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 msgr2=0x561ae54d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54d5a80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8003700 tx=0x7fa1d8003730 comp rx=0 tx=0).stop 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 -- 172.21.15.113:0/3396309067 shutdown_connections 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54e5a00 0x561ae54e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dce80 0x561ae54d5a80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 --2- 172.21.15.113:0/3396309067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54dc4d0 0x561ae54f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 -- 172.21.15.113:0/3396309067 >> 172.21.15.113:0/3396309067 conn(0x561ae54d6ba0 msgr2=0x561ae54d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 -- 172.21.15.113:0/3396309067 shutdown_connections 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 -- 172.21.15.113:0/3396309067 wait complete. 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 -- start start 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 Processor -- start 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.678+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 0x561ae54edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dce80 0x561ae54e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54e5a00 0x561ae54e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81ac470 con 0x561ae54dc4d0 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81ad8e0 con 0x561ae54dce80 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81acb40 con 0x561ae54e5a00 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 10 client.0 _ll_register_callbacks cb 0x7fa1e0012e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 0x561ae54edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 0x561ae54edee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54236/0 (socket says 172.21.15.113:54236) 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/631620640 learned_addr learned my addr 172.21.15.113:0/631620640 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dce80 0x561ae54e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/631620640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54e5a00 msgr2=0x561ae54e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54e5a00 0x561ae54e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dce80 msgr2=0x561ae54e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dce80 0x561ae54e92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 -- 172.21.15.113:0/631620640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ae54e9da0 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1ed9f9640 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 0x561ae54edee0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009270 tx=0x7fa1e0012dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70560db681e615ba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/631620640 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0018040 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/631620640 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fa1e00058e0 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8005850 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/631620640 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa1e003c050 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 10 client.16635 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561ae5560330 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.679+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/631620640 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa1d8005850 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/631620640 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7fa1e0004650 con 0x561ae54dc4d0 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 10 client.16635 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 10 client.16635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 20 client.16635 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:11.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae5562b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 4 client.16635 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ae54e2ed0 con 0x561ae54f5c10 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae5562b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1ec9f7640 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae5562b30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=769aa4a3eaf5454f server_cookie=2c6931c82ae26caa in_seq=0 out_seq=0 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1d0ff9640 10 client.16635 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1d0ff9640 1 -- 172.21.15.113:0/631620640 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ae54e2ed0 con 0x561ae54f5c10 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1d0ff9640 10 client.16635 handle_command_reply: tid=0 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 1 client.16635 shutdown 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 msgr2=0x561ae54edee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.680+0000 7fa1f2053740 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 0x561ae54edee0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009270 tx=0x7fa1e0012dc0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 shutdown_connections 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x561ae54f5c10 0x561ae5562b30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561ae54e5a00 0x561ae54e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561ae54dce80 0x561ae54e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1d0ff9640 0 client.16635 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:11.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 --2- 172.21.15.113:0/631620640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561ae54dc4d0 0x561ae54edee0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 >> 172.21.15.113:0/631620640 conn(0x561ae54d6ba0 msgr2=0x561ae54d5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 shutdown_connections 2024-04-22T07:19:11.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 1 -- 172.21.15.113:0/631620640 wait complete. 2024-04-22T07:19:11.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1f2053740 20 client.16635 trim_cache size 0 max 16384 2024-04-22T07:19:11.689 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa1d40a3d20 msgr2=0x7fa1d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4099370 tx=0x7fa1e409c7e0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 msgr2=0x7fa1e819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 0x7fa1e819e650 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400c920 tx=0x7fa1e400cde0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 shutdown_connections 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e8198760 0x7fa1e8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e81072e0 0x7fa1e819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 --2- 172.21.15.113:0/3632812937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1e8102210 0x7fa1e819e650 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 >> 172.21.15.113:0/3632812937 conn(0x7fa1e80fdab0 msgr2=0x7fa1e80ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 shutdown_connections 2024-04-22T07:19:11.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:11.681+0000 7fa1ef45c640 1 -- 172.21.15.113:0/3632812937 wait complete. 2024-04-22T07:19:11.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:11 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4454082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:19:12.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:11 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4454082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:19:12.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:11 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4454082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:19:12.693 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:19:12.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:19:12.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 -- 172.21.15.113:0/2503746930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4106160 msgr2=0x7f9cb410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 --2- 172.21.15.113:0/2503746930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4106160 0x7f9cb410dd30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009ca0 tx=0x7f9cac02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 -- 172.21.15.113:0/2503746930 shutdown_connections 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 --2- 172.21.15.113:0/2503746930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4106160 0x7f9cb410dd30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 --2- 172.21.15.113:0/2503746930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cb4101a40 0x7f9cb4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 --2- 172.21.15.113:0/2503746930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb4101090 0x7f9cb4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 -- 172.21.15.113:0/2503746930 >> 172.21.15.113:0/2503746930 conn(0x7f9cb4077f50 msgr2=0x7f9cb4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 -- 172.21.15.113:0/2503746930 shutdown_connections 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 -- 172.21.15.113:0/2503746930 wait complete. 2024-04-22T07:19:12.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 Processor -- start 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.811+0000 7f9cbc014640 1 -- start start 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cb4101090 0x7f9cb4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb4106160 0x7f9cb41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 0x7f9cb41a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4111200 con 0x7f9cb4197ef0 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4111080 con 0x7f9cb4106160 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4111380 con 0x7f9cb4101090 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 0x7f9cb41a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 0x7f9cb41a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54256/0 (socket says 172.21.15.113:54256) 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 -- 172.21.15.113:0/389600963 learned_addr learned my addr 172.21.15.113:0/389600963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cb95af640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb4106160 0x7f9cb41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cb4101090 0x7f9cb4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 -- 172.21.15.113:0/389600963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cb4101090 msgr2=0x7f9cb4197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cb4101090 0x7f9cb4197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 -- 172.21.15.113:0/389600963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb4106160 msgr2=0x7f9cb41979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb4106160 0x7f9cb41979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 -- 172.21.15.113:0/389600963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb41a70b0 con 0x7f9cb4197ef0 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cba5b1640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 0x7f9cb41a6af0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009dc0 tx=0x7f9cac035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9caaffd640 1 -- 172.21.15.113:0/389600963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac039040 con 0x7f9cb4197ef0 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9caaffd640 1 -- 172.21.15.113:0/389600963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9cac044070 con 0x7f9cb4197ef0 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb41a7340 con 0x7f9cb4197ef0 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9caaffd640 1 -- 172.21.15.113:0/389600963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac004330 con 0x7f9cb4197ef0 2024-04-22T07:19:12.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.812+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb41a77d0 con 0x7f9cb4197ef0 2024-04-22T07:19:12.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9caaffd640 1 -- 172.21.15.113:0/389600963 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9cac0044d0 con 0x7f9cb4197ef0 2024-04-22T07:19:12.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 Processor -- start 2024-04-22T07:19:12.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 -- start start 2024-04-22T07:19:12.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae87cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae90840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae90e10 0x55c73ae93260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c73adebf00 con 0x55c73ae878e0 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c73ad89590 con 0x55c73ae88290 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cbec08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c73ade7ec0 con 0x55c73ae90e10 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae87cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae87cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54270/0 (socket says 172.21.15.113:54270) 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cb95af640 1 -- 172.21.15.113:0/3370275007 learned_addr learned my addr 172.21.15.113:0/3370275007 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.815+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae90840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9caaffd640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ca00a3d20 0x7f9ca00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae90e10 0x55c73ae93260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cba5b1640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ca00a3d20 0x7f9ca00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 -- 172.21.15.113:0/3370275007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae90e10 msgr2=0x55c73ae93260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae90e10 0x55c73ae93260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 -- 172.21.15.113:0/3370275007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 msgr2=0x55c73ae90840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae90840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 -- 172.21.15.113:0/3370275007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73abc67e0 con 0x55c73ae878e0 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae90840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9caaffd640 1 -- 172.21.15.113:0/389600963 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f9cac0f2da0 con 0x7f9cb4197ef0 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9cb95af640 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae87cc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400afd0 tx=0x7f9ca400a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5363ccfa30984b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9c9ffff640 1 -- 172.21.15.113:0/3370275007 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4019030 con 0x55c73ae878e0 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9c9ffff640 1 -- 172.21.15.113:0/3370275007 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ca40054f0 con 0x55c73ae878e0 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.816+0000 7f9c9ffff640 1 -- 172.21.15.113:0/3370275007 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4010860 con 0x55c73ae878e0 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 -- 172.21.15.113:0/3370275007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 msgr2=0x55c73ae87cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae87cc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400afd0 tx=0x7f9ca400a7c0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cba5b1640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ca00a3d20 0x7f9ca00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0f0970 tx=0x7f9cac0a9800 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 -- 172.21.15.113:0/3370275007 shutdown_connections 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae90e10 0x55c73ae93260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae90840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 --2- 172.21.15.113:0/3370275007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae87cc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 -- 172.21.15.113:0/3370275007 >> 172.21.15.113:0/3370275007 conn(0x55c73ae81fb0 msgr2=0x55c73ae82390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 -- 172.21.15.113:0/3370275007 shutdown_connections 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 -- 172.21.15.113:0/3370275007 wait complete. 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 -- start start 2024-04-22T07:19:12.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.817+0000 7f9cbec08740 1 Processor -- start 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c73ae93d80 con 0x55c73ae90e10 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c73ada5580 con 0x55c73ae88290 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c73ae93f00 con 0x55c73ae878e0 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cbec08740 10 client.0 _ll_register_callbacks cb 0x55c73adab650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54272/0 (socket says 172.21.15.113:54272) 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 -- 172.21.15.113:0/608353992 learned_addr learned my addr 172.21.15.113:0/608353992 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 -- 172.21.15.113:0/608353992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 msgr2=0x55c73ae8de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 -- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 msgr2=0x55c73ae8e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 -- 172.21.15.113:0/608353992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73ac26f70 con 0x55c73ae90e10 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9cb95af640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8e890 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a500 tx=0x7f9ca400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e0d787434b6858 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.818+0000 7f9c9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/608353992 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4019040 con 0x55c73ae90e10 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/608353992 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f9ca4010640 con 0x55c73ae90e10 2024-04-22T07:19:12.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca40054f0 con 0x55c73ae90e10 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/608353992 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9ca402f050 con 0x55c73ae90e10 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 10 client.16647 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c73aabe170 con 0x55c73ae90e10 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/608353992 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ca40054f0 con 0x55c73ae90e10 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/608353992 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f9ca401e790 con 0x55c73ae90e10 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 10 client.16647 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 20 client.16647 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73aea1300 0x55c73aea36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 4 client.16647 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:12.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.819+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73adf9920 con 0x55c73aea1300 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.820+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73aea1300 0x55c73aea36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.820+0000 7f9c9d7fa640 10 client.16647 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.820+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73aea1300 0x55c73aea36f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beacf693a034f49 server_cookie=301d7ccaff62a855 in_seq=0 out_seq=0 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.820+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/608353992 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c73adf9920 con 0x55c73aea1300 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.820+0000 7f9c9d7fa640 10 client.16647 handle_command_reply: tid=0 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.820+0000 7f9cbec08740 1 client.16647 shutdown 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 msgr2=0x55c73ae8e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8e890 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a500 tx=0x7f9ca400a490 comp rx=0 tx=0).stop 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 shutdown_connections 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73aea1300 0x55c73aea36f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8e890 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9c9d7fa640 0 client.16647 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 --2- 172.21.15.113:0/608353992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 >> 172.21.15.113:0/608353992 conn(0x55c73ae81fb0 msgr2=0x55c73ae83d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 shutdown_connections 2024-04-22T07:19:12.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 1 -- 172.21.15.113:0/608353992 wait complete. 2024-04-22T07:19:12.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.821+0000 7f9cbec08740 20 client.16647 trim_cache size 0 max 16384 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.838+0000 7f9cbec08740 1 Processor -- start 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.838+0000 7f9cbec08740 1 -- start start 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae9cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae88290 0x55c73ae8d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae90e10 0x55c73ae8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cbec08740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb41b29c0 con 0x55c73ae878e0 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cbec08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb41b0230 con 0x55c73ae90e10 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cbec08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb41ae080 con 0x55c73ae88290 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae9cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae9cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54288/0 (socket says 172.21.15.113:54288) 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 -- 172.21.15.113:0/1240690314 learned_addr learned my addr 172.21.15.113:0/1240690314 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae90e10 0x55c73ae8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae88290 0x55c73ae8d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 -- 172.21.15.113:0/1240690314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae88290 msgr2=0x55c73ae8d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae88290 0x55c73ae8d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 -- 172.21.15.113:0/1240690314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae90e10 msgr2=0x55c73ae8fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae90e10 0x55c73ae8fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 -- 172.21.15.113:0/1240690314 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73ae88c20 con 0x55c73ae878e0 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae88290 0x55c73ae8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9cb95af640 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae9cc40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a860 tx=0x7f9ca40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e268ed6d273bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9c9f7fe640 1 -- 172.21.15.113:0/1240690314 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4019040 con 0x55c73ae878e0 2024-04-22T07:19:12.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9c9f7fe640 1 -- 172.21.15.113:0/1240690314 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ca4004a40 con 0x55c73ae878e0 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.839+0000 7f9c9f7fe640 1 -- 172.21.15.113:0/1240690314 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca401e6c0 con 0x55c73ae878e0 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 -- 172.21.15.113:0/1240690314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 msgr2=0x55c73ae9cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae9cc40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400a860 tx=0x7f9ca40058a0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 -- 172.21.15.113:0/1240690314 shutdown_connections 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae90e10 0x55c73ae8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae88290 0x55c73ae8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 --2- 172.21.15.113:0/1240690314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae878e0 0x55c73ae9cc40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 -- 172.21.15.113:0/1240690314 >> 172.21.15.113:0/1240690314 conn(0x55c73ae81fb0 msgr2=0x55c73ae8f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 -- 172.21.15.113:0/1240690314 shutdown_connections 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 -- 172.21.15.113:0/1240690314 wait complete. 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 -- start start 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.840+0000 7f9cbec08740 1 Processor -- start 2024-04-22T07:19:12.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb41ae6f0 con 0x55c73ae90e10 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb41afb60 con 0x55c73ae88290 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb41aedc0 con 0x55c73ae878e0 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cbec08740 10 client.0 _ll_register_callbacks cb 0x7f9cb00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54298/0 (socket says 172.21.15.113:54298) 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 -- 172.21.15.113:0/4076297031 learned_addr learned my addr 172.21.15.113:0/4076297031 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 -- 172.21.15.113:0/4076297031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 msgr2=0x55c73ae8aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 -- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 msgr2=0x55c73ae8b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 -- 172.21.15.113:0/4076297031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73af06950 con 0x55c73ae90e10 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9cb95af640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8b720 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9ca401ec00 tx=0x7f9ca4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ebc7609d2ead97 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9c9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.841+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/4076297031 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4039030 con 0x55c73ae90e10 2024-04-22T07:19:12.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/4076297031 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f9ca403e070 con 0x55c73ae90e10 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca4004a40 con 0x55c73ae90e10 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/4076297031 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9ca4049050 con 0x55c73ae90e10 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 10 client.16653 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c73af07bc0 con 0x55c73ae90e10 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/4076297031 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ca4004a40 con 0x55c73ae90e10 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/4076297031 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f9ca4010640 con 0x55c73ae90e10 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 10 client.16653 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 20 client.16653 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73af08270 0x55c73af0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 4 client.16653 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c73adf3790 con 0x55c73af08270 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.842+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73af08270 0x55c73af0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9c9d7fa640 10 client.16653 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9cb9db0640 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73af08270 0x55c73af0a690 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feab2ae0b46f5f67 server_cookie=730701a355dee1ff in_seq=0 out_seq=0 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9c9d7fa640 1 -- 172.21.15.113:0/4076297031 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c73adf3790 con 0x55c73af08270 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9c9d7fa640 10 client.16653 handle_command_reply: tid=0 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9cbec08740 1 client.16653 shutdown 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 msgr2=0x55c73ae8b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.843+0000 7f9cbec08740 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8b720 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9ca401ec00 tx=0x7f9ca4005a10 comp rx=0 tx=0).stop 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 shutdown_connections 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c73af08270 0x55c73af0a690 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73ae90e10 0x55c73ae8b720 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9c9d7fa640 0 client.16653 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73ae88290 0x55c73ae8b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 --2- 172.21.15.113:0/4076297031 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c73ae878e0 0x55c73ae8aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 >> 172.21.15.113:0/4076297031 conn(0x55c73ae81fb0 msgr2=0x55c73ae8fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 shutdown_connections 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 1 -- 172.21.15.113:0/4076297031 wait complete. 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbec08740 20 client.16653 trim_cache size 0 max 16384 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "1616afa6-6280-4529-bb76-62f8ebadfdfd", 2024-04-22T07:19:12.850 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:19:12.851 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:19:12.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ca00a3d20 msgr2=0x7f9ca00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ca00a3d20 0x7f9ca00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0f0970 tx=0x7f9cac0a9800 comp rx=0 tx=0).stop 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 msgr2=0x7f9cb41a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 0x7f9cb41a6af0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009dc0 tx=0x7f9cac035d80 comp rx=0 tx=0).stop 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 shutdown_connections 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9ca00a3d20 0x7f9ca00a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb4197ef0 0x7f9cb41a6af0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb4106160 0x7f9cb41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 --2- 172.21.15.113:0/389600963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cb4101090 0x7f9cb4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 >> 172.21.15.113:0/389600963 conn(0x7f9cb4077f50 msgr2=0x7f9cb40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 shutdown_connections 2024-04-22T07:19:12.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.844+0000 7f9cbc014640 1 -- 172.21.15.113:0/389600963 wait complete. 2024-04-22T07:19:12.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:19:12.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:19:12.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- 172.21.15.113:0/564573485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc106150 msgr2=0x7f2bdc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- 172.21.15.113:0/564573485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc106150 0x7f2bdc10dd20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009900 tx=0x7f2bd402bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:12.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- 172.21.15.113:0/564573485 shutdown_connections 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- 172.21.15.113:0/564573485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc106150 0x7f2bdc10dd20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- 172.21.15.113:0/564573485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bdc101a30 0x7f2bdc105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- 172.21.15.113:0/564573485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc101080 0x7f2bdc101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- 172.21.15.113:0/564573485 >> 172.21.15.113:0/564573485 conn(0x7f2bdc078000 msgr2=0x7f2bdc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- 172.21.15.113:0/564573485 shutdown_connections 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- 172.21.15.113:0/564573485 wait complete. 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 Processor -- start 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- start start 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bdc101080 0x7f2bdc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc106150 0x7f2bdc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 0x7f2bdc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1111f0 con 0x7f2bdc198680 2024-04-22T07:19:12.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111070 con 0x7f2bdc106150 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.956+0000 7f2be359d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111370 con 0x7f2bdc101080 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 0x7f2bdc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be0b38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc106150 0x7f2bdc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 0x7f2bdc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54318/0 (socket says 172.21.15.113:54318) 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 -- 172.21.15.113:0/3274884905 learned_addr learned my addr 172.21.15.113:0/3274884905 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1339640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bdc101080 0x7f2bdc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 -- 172.21.15.113:0/3274884905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bdc101080 msgr2=0x7f2bdc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bdc101080 0x7f2bdc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 -- 172.21.15.113:0/3274884905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc106150 msgr2=0x7f2bdc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc106150 0x7f2bdc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 -- 172.21.15.113:0/3274884905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc199280 con 0x7f2bdc198680 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 0x7f2bdc198b00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4005940 tx=0x7f2bd402cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2bd27fc640 1 -- 172.21.15.113:0/3274884905 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4041070 con 0x7f2bdc198680 2024-04-22T07:19:12.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2bd27fc640 1 -- 172.21.15.113:0/3274884905 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2bd40044d0 con 0x7f2bdc198680 2024-04-22T07:19:12.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc199510 con 0x7f2bdc198680 2024-04-22T07:19:12.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2bd27fc640 1 -- 172.21.15.113:0/3274884905 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4003d40 con 0x7f2bdc198680 2024-04-22T07:19:12.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.957+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a56b0 con 0x7f2bdc198680 2024-04-22T07:19:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.959+0000 7f2bd27fc640 1 -- 172.21.15.113:0/3274884905 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2bd4044070 con 0x7f2bdc198680 2024-04-22T07:19:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 Processor -- start 2024-04-22T07:19:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 -- start start 2024-04-22T07:19:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eab7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eac0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2bd27fc640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bc80a3d20 0x7f2bc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561c1ea1c1e0 con 0x561c1eab7580 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561c1e9b9390 con 0x561c1eac0a80 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be618c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561c1ea181a0 con 0x561c1eab7ea0 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be0b38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eab7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be0b38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eab7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54332/0 (socket says 172.21.15.113:54332) 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be0b38640 1 -- 172.21.15.113:0/3440273588 learned_addr learned my addr 172.21.15.113:0/3440273588 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be0b38640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bc80a3d20 0x7f2bc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2be1339640 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eac0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.960+0000 7f2bd27fc640 1 -- 172.21.15.113:0/3274884905 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2bd40f2250 con 0x7f2bdc198680 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 -- 172.21.15.113:0/3440273588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 msgr2=0x561c1eac0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eac0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 -- 172.21.15.113:0/3440273588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 msgr2=0x561c1eac2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 -- 172.21.15.113:0/3440273588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1e7f87d0 con 0x561c1eab7580 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eab7960 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc004650 tx=0x7f2bcc012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8ac433208479fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2be0b38640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bc80a3d20 0x7f2bc80a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc010740 tx=0x7f2bcc017000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2bc77fe640 1 -- 172.21.15.113:0/3440273588 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc024070 con 0x561c1eab7580 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2bc77fe640 1 -- 172.21.15.113:0/3440273588 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2bcc00b9d0 con 0x561c1eab7580 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.961+0000 7f2bc77fe640 1 -- 172.21.15.113:0/3440273588 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc004d00 con 0x561c1eab7580 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 -- 172.21.15.113:0/3440273588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 msgr2=0x561c1eab7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eab7960 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc004650 tx=0x7f2bcc012e70 comp rx=0 tx=0).stop 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 -- 172.21.15.113:0/3440273588 shutdown_connections 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eac0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 --2- 172.21.15.113:0/3440273588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eab7960 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 -- 172.21.15.113:0/3440273588 >> 172.21.15.113:0/3440273588 conn(0x561c1eab1cb0 msgr2=0x561c1eab2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 -- 172.21.15.113:0/3440273588 shutdown_connections 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 -- 172.21.15.113:0/3440273588 wait complete. 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 -- start start 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.962+0000 7f2be618c740 1 Processor -- start 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7580 0x561c1eabdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eabe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 0x561c1eabe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561c1eac3810 con 0x561c1eac0a80 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561c1e9d52f0 con 0x561c1eab7580 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561c1eac3990 con 0x561c1eab7ea0 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 10 client.0 _ll_register_callbacks cb 0x561c1e9e2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 0x561c1eabe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 0x561c1eabe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54344/0 (socket says 172.21.15.113:54344) 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 -- 172.21.15.113:0/1359329296 learned_addr learned my addr 172.21.15.113:0/1359329296 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1339640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7580 0x561c1eabdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be0b38640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eabe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 -- 172.21.15.113:0/1359329296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 msgr2=0x561c1eabe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eabe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 -- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7580 msgr2=0x561c1eabdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7580 0x561c1eabdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 -- 172.21.15.113:0/1359329296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1e72e3f0 con 0x561c1eac0a80 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 0x561c1eabe610 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40ef990 tx=0x7f2bd403acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a90a264d0f5dff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/1359329296 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd404b040 con 0x561c1eac0a80 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/1359329296 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2bd40ba930 con 0x561c1eac0a80 2024-04-22T07:19:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bcc00b9d0 con 0x561c1eac0a80 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/1359329296 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2bd40bdc20 con 0x561c1eac0a80 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 10 client.16665 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.963+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c1e6ee0f0 con 0x561c1eac0a80 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/1359329296 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2bcc00b9d0 con 0x561c1eac0a80 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/1359329296 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f2bd40d8410 con 0x561c1eac0a80 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be618c740 10 client.16665 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be618c740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be618c740 20 client.16665 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be618c740 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1ead0ae0 0x561c1ead2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be618c740 4 client.16665 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1ea29c00 con 0x561c1ead0ae0 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be1339640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1ead0ae0 0x561c1ead2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2be1339640 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1ead0ae0 0x561c1ead2f40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82a939385d6f3d4 server_cookie=faa03e447232e084 in_seq=0 out_seq=0 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.964+0000 7f2bc4ff9640 10 client.16665 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/1359329296 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c1ea29c00 con 0x561c1ead0ae0 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2bc4ff9640 10 client.16665 handle_command_reply: tid=0 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 client.16665 shutdown 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 msgr2=0x561c1eabe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 0x561c1eabe610 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40ef990 tx=0x7f2bd403acd0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 shutdown_connections 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1ead0ae0 0x561c1ead2f40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eac0a80 0x561c1eabe610 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2bc4ff9640 0 client.16665 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7ea0 0x561c1eabe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 --2- 172.21.15.113:0/1359329296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7580 0x561c1eabdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 >> 172.21.15.113:0/1359329296 conn(0x561c1eab1cb0 msgr2=0x561c1eab3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 shutdown_connections 2024-04-22T07:19:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 1 -- 172.21.15.113:0/1359329296 wait complete. 2024-04-22T07:19:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.965+0000 7f2be618c740 20 client.16665 trim_cache size 0 max 16384 2024-04-22T07:19:12.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.982+0000 7f2be618c740 1 Processor -- start 2024-04-22T07:19:12.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.982+0000 7f2be618c740 1 -- start start 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be618c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eac02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be618c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7ea0 0x561c1eacc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be618c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eac0a80 0x561c1eab0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be618c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1b0860 con 0x561c1eab7580 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be618c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1ae0d0 con 0x561c1eab7ea0 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be618c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1abf40 con 0x561c1eac0a80 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eac02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eac02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54350/0 (socket says 172.21.15.113:54350) 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 -- 172.21.15.113:0/495170248 learned_addr learned my addr 172.21.15.113:0/495170248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be0b38640 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7ea0 0x561c1eacc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eac0a80 0x561c1eab0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 -- 172.21.15.113:0/495170248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eac0a80 msgr2=0x561c1eab0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eac0a80 0x561c1eab0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 -- 172.21.15.113:0/495170248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7ea0 msgr2=0x561c1eacc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7ea0 0x561c1eacc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 -- 172.21.15.113:0/495170248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1eac31e0 con 0x561c1eab7580 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eac0a80 0x561c1eab0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2be1339640 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eac02c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x561c1ead37b0 tx=0x7f2bd8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae033f85f698954 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2bc6ffd640 1 -- 172.21.15.113:0/495170248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801ca50 con 0x561c1eab7580 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2bc6ffd640 1 -- 172.21.15.113:0/495170248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2bd801cbf0 con 0x561c1eab7580 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.983+0000 7f2bc6ffd640 1 -- 172.21.15.113:0/495170248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801d770 con 0x561c1eab7580 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 -- 172.21.15.113:0/495170248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 msgr2=0x561c1eac02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eac02c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x561c1ead37b0 tx=0x7f2bd8009500 comp rx=0 tx=0).stop 2024-04-22T07:19:12.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 -- 172.21.15.113:0/495170248 shutdown_connections 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eac0a80 0x561c1eab0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eab7ea0 0x561c1eacc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 --2- 172.21.15.113:0/495170248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7580 0x561c1eac02c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 -- 172.21.15.113:0/495170248 >> 172.21.15.113:0/495170248 conn(0x561c1eab1cb0 msgr2=0x561c1eab04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 -- 172.21.15.113:0/495170248 shutdown_connections 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 -- 172.21.15.113:0/495170248 wait complete. 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 -- start start 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.984+0000 7f2be618c740 1 Processor -- start 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7580 0x561c1eac44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 0x561c1eac4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1ac590 con 0x561c1eab7ea0 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1ada00 con 0x561c1eac0a80 2024-04-22T07:19:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc1acc60 con 0x561c1eab7580 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 10 client.0 _ll_register_callbacks cb 0x7f2bd4009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 0x561c1eac4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 0x561c1eac4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54364/0 (socket says 172.21.15.113:54364) 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 -- 172.21.15.113:0/2106652705 learned_addr learned my addr 172.21.15.113:0/2106652705 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be1b3a640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be1339640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7580 0x561c1eac44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 -- 172.21.15.113:0/2106652705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7580 msgr2=0x561c1eac44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7580 0x561c1eac44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 -- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 msgr2=0x561c1eac4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 -- 172.21.15.113:0/2106652705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1eb36080 con 0x561c1eab7ea0 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be0b38640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 0x561c1eac4a30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00b750 tx=0x7f2bcc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8599f861e89e0544 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:12.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/2106652705 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc024040 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/2106652705 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2bcc01b040 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.985+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd801cbf0 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/2106652705 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2bcc00d720 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 10 client.16671 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561c1eb372a0 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/2106652705 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2bd801cbf0 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/2106652705 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f2bcc033910 con 0x561c1eab7ea0 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 10 client.16671 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:12.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 20 client.16671 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1eb377f0 0x561c1eb39c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 4 client.16671 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c1ea23a70 con 0x561c1eb377f0 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.986+0000 7f2be1339640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1eb377f0 0x561c1eb39c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be1339640 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1eb377f0 0x561c1eb39c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0ccfa993f97b99 server_cookie=5ff456f9452439a5 in_seq=0 out_seq=0 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2bc4ff9640 10 client.16671 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2bc4ff9640 1 -- 172.21.15.113:0/2106652705 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561c1ea23a70 con 0x561c1eb377f0 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2bc4ff9640 10 client.16671 handle_command_reply: tid=0 2024-04-22T07:19:12.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 client.16671 shutdown 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 msgr2=0x561c1eac4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 0x561c1eac4a30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00b750 tx=0x7f2bcc004680 comp rx=0 tx=0).stop 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 shutdown_connections 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561c1eb377f0 0x561c1eb39c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c1eac0a80 0x561c1eac4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561c1eab7ea0 0x561c1eac4a30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2bc4ff9640 0 client.16671 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 --2- 172.21.15.113:0/2106652705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561c1eab7580 0x561c1eac44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 >> 172.21.15.113:0/2106652705 conn(0x561c1eab1cb0 msgr2=0x561c1eab0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 shutdown_connections 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 1 -- 172.21.15.113:0/2106652705 wait complete. 2024-04-22T07:19:12.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be618c740 20 client.16671 trim_cache size 0 max 16384 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: "1616afa6-6280-4529-bb76-62f8ebadfdfd": { 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "1616afa6-6280-4529-bb76-62f8ebadfdfd", 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:19:12.994 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:19:12.995 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:19:12.995 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:19:12.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bc80a3d20 msgr2=0x7f2bc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be359d640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bc80a3d20 0x7f2bc80a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc010740 tx=0x7f2bcc017000 comp rx=0 tx=0).stop 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 msgr2=0x7f2bdc198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.987+0000 7f2be359d640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 0x7f2bdc198b00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4005940 tx=0x7f2bd402cdf0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 shutdown_connections 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bc80a3d20 0x7f2bc80a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bdc198680 0x7f2bdc198b00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc106150 0x7f2bdc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 --2- 172.21.15.113:0/3274884905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bdc101080 0x7f2bdc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 >> 172.21.15.113:0/3274884905 conn(0x7f2bdc078000 msgr2=0x7f2bdc069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 shutdown_connections 2024-04-22T07:19:12.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:12.988+0000 7f2be359d640 1 -- 172.21.15.113:0/3274884905 wait complete. 2024-04-22T07:19:12.997 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:12 smithi113 ceph-mon[27933]: pgmap v138: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T07:19:12.998 INFO:tasks.cephfs.filesystem:scrub status for tag:1616afa6-6280-4529-bb76-62f8ebadfdfd - {'path': '/', 'tag': '1616afa6-6280-4529-bb76-62f8ebadfdfd', 'options': 'recursive,force'} 2024-04-22T07:19:13.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:12 smithi133 ceph-mon[31198]: pgmap v138: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T07:19:13.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:12 smithi154 ceph-mon[31862]: pgmap v138: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-22T07:19:14.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:13 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:19:14.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:14.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:13 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:19:14.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:13 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:14.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:13 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:19:14.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:14.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:13 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:19:14.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:13 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:14.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:13 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:19:14.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:14.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:13 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:19:14.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:13 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:15.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:14 smithi133 ceph-mon[31198]: pgmap v139: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:19:15.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:14 smithi154 ceph-mon[31862]: pgmap v139: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:19:15.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:14 smithi113 ceph-mon[27933]: pgmap v139: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:19:16.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:15 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:19:16.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:15 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:19:16.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:15 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:19:17.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:16 smithi133 ceph-mon[31198]: pgmap v140: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T07:19:17.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:16 smithi154 ceph-mon[31862]: pgmap v140: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T07:19:17.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:16 smithi113 ceph-mon[27933]: pgmap v140: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T07:19:17.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:17.259+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:17.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:17.259+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:19:17.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:17.259+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:19:17.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:17.259+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:19:17.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:17.259+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:19:19.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:18 smithi133 ceph-mon[31198]: pgmap v141: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:19:19.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:18 smithi154 ceph-mon[31862]: pgmap v141: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:19:19.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:18 smithi113 ceph-mon[27933]: pgmap v141: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:19:21.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:20 smithi133 ceph-mon[31198]: pgmap v142: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:19:21.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:20 smithi154 ceph-mon[31862]: pgmap v142: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:19:21.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:20 smithi113 ceph-mon[27933]: pgmap v142: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:19:23.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:22 smithi133 ceph-mon[31198]: pgmap v143: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T07:19:23.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:22 smithi154 ceph-mon[31862]: pgmap v143: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T07:19:23.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:22 smithi113 ceph-mon[27933]: pgmap v143: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T07:19:25.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:24 smithi133 ceph-mon[31198]: pgmap v144: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-22T07:19:25.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:24 smithi154 ceph-mon[31862]: pgmap v144: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-22T07:19:25.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:24 smithi113 ceph-mon[27933]: pgmap v144: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-22T07:19:27.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:26 smithi154 ceph-mon[31862]: pgmap v145: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:19:27.067 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:26 smithi133 ceph-mon[31198]: pgmap v145: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:19:27.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:26 smithi113 ceph-mon[27933]: pgmap v145: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:19:29.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:28 smithi133 ceph-mon[31198]: pgmap v146: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:29.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:28 smithi154 ceph-mon[31862]: pgmap v146: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:29.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:28 smithi113 ceph-mon[27933]: pgmap v146: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:31.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:30 smithi133 ceph-mon[31198]: pgmap v147: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:31.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:30 smithi154 ceph-mon[31862]: pgmap v147: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:31.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:30 smithi113 ceph-mon[27933]: pgmap v147: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:32.261+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:32.262+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:19:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:32.262+0000 7f22f9f82640 1 -- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 msgr2=0x55d5e6753a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:19:32.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:32.262+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:19:32.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:32.262+0000 7f22f9f82640 1 --2- 172.21.15.113:0/83262998 >> [v2:172.21.15.133:6838/982201245,v1:172.21.15.133:6839/982201245] conn(0x55d5e66ea7f0 0x55d5e6753a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-22T07:19:33.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:32 smithi133 ceph-mon[31198]: pgmap v148: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:33.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:32 smithi154 ceph-mon[31862]: pgmap v148: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:32 smithi113 ceph-mon[27933]: pgmap v148: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:35.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:34 smithi133 ceph-mon[31198]: pgmap v149: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:35.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:34 smithi154 ceph-mon[31862]: pgmap v149: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:34 smithi113 ceph-mon[27933]: pgmap v149: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:36.664 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T07:19:36.665 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 240, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 211, in do_quiesce if self.check_canceled(stdout, set_id): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 156, in check_canceled if 'CANCELED' == self.get_set_state_name(response, set_id): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 141, in get_set_state_name response = json.loads(response) File "/usr/lib/python3.8/json/__init__.py", line 357, in loads return _default_decoder.decode(s) File "/usr/lib/python3.8/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2024-04-22T07:19:36.667 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 263, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Expecting value: line 1 column 1 (char 0) 2024-04-22T07:19:36.668 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-22T07:19:36.678 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-22T07:19:36.678 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T07:19:36.679 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:19:36.679 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-22T07:19:36.709 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-22T07:19:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:36 smithi133 ceph-mon[31198]: pgmap v150: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:37.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:36 smithi154 ceph-mon[31862]: pgmap v150: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:36 smithi113 ceph-mon[27933]: pgmap v150: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:38 smithi133 ceph-mon[31198]: pgmap v151: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:39.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:38 smithi154 ceph-mon[31862]: pgmap v151: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:39.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:38 smithi113 ceph-mon[27933]: pgmap v151: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:41.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:40 smithi133 ceph-mon[31198]: pgmap v152: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:41.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:40 smithi154 ceph-mon[31862]: pgmap v152: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:41.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:40 smithi113 ceph-mon[27933]: pgmap v152: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:42.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:19:42.999 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:19:43.024 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:42 smithi113 ceph-mon[27933]: pgmap v153: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:43.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:42 smithi133 ceph-mon[31198]: pgmap v153: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:43.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:42 smithi154 ceph-mon[31862]: pgmap v153: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:19:43.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 -- 172.21.15.113:0/3153077818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec10c500 msgr2=0x7f73ec10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/3153077818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec10c500 0x7f73ec10e9a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ca0 tx=0x7f73e402c210 comp rx=0 tx=0).stop 2024-04-22T07:19:43.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 -- 172.21.15.113:0/3153077818 shutdown_connections 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/3153077818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec10c500 0x7f73ec10e9a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/3153077818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec103ae0 0x7f73ec10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/3153077818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec1031c0 0x7f73ec1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 -- 172.21.15.113:0/3153077818 >> 172.21.15.113:0/3153077818 conn(0x7f73ec078000 msgr2=0x7f73ec078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 -- 172.21.15.113:0/3153077818 shutdown_connections 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 -- 172.21.15.113:0/3153077818 wait complete. 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 Processor -- start 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.114+0000 7f73f3ac8640 1 -- start start 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec1031c0 0x7f73ec198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 0x7f73ec198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec10c500 0x7f73ec19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1122d0 con 0x7f73ec103ae0 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec112150 con 0x7f73ec10c500 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec112450 con 0x7f73ec1031c0 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 0x7f73ec198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 0x7f73ec198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57950/0 (socket says 172.21.15.113:57950) 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 -- 172.21.15.113:0/807273510 learned_addr learned my addr 172.21.15.113:0/807273510 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1864640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec1031c0 0x7f73ec198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f2065640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec10c500 0x7f73ec19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 -- 172.21.15.113:0/807273510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec1031c0 msgr2=0x7f73ec198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec1031c0 0x7f73ec198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 -- 172.21.15.113:0/807273510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec10c500 msgr2=0x7f73ec19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec10c500 0x7f73ec19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 -- 172.21.15.113:0/807273510 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19d640 con 0x7f73ec103ae0 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f2065640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec10c500 0x7f73ec19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f1063640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 0x7f73ec198b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f73e800c920 tx=0x7f73e800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73e2ffd640 1 -- 172.21.15.113:0/807273510 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8007bb0 con 0x7f73ec103ae0 2024-04-22T07:19:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19d930 con 0x7f73ec103ae0 2024-04-22T07:19:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73e2ffd640 1 -- 172.21.15.113:0/807273510 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73e8007d50 con 0x7f73ec103ae0 2024-04-22T07:19:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73e2ffd640 1 -- 172.21.15.113:0/807273510 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80187c0 con 0x7f73ec103ae0 2024-04-22T07:19:43.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.115+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a51c0 con 0x7f73ec103ae0 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.118+0000 7f73e2ffd640 1 -- 172.21.15.113:0/807273510 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f73e800f450 con 0x7f73ec103ae0 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.118+0000 7f73f6799740 1 Processor -- start 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.118+0000 7f73f6799740 1 -- start start 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f6799740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160580 0x55c7c8160960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f6799740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8169540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f6799740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 0x55c7c816be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f6799740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c7c80c51e0 con 0x55c7c8169a80 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f6799740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c7c8062390 con 0x55c7c8160580 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f6799740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c7c80c11a0 con 0x55c7c8160ea0 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73e2ffd640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f73d80a3d20 0x7f73d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 0x55c7c816be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f2065640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f73d80a3d20 0x7f73d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 0x55c7c816be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57966/0 (socket says 172.21.15.113:57966) 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 -- 172.21.15.113:0/3532170234 learned_addr learned my addr 172.21.15.113:0/3532170234 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f2065640 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160580 0x55c7c8160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1864640 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73e2ffd640 1 -- 172.21.15.113:0/807273510 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f73e80d0a40 con 0x7f73ec103ae0 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 -- 172.21.15.113:0/3532170234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 msgr2=0x55c7c8169540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8169540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 -- 172.21.15.113:0/3532170234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160580 msgr2=0x55c7c8160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160580 0x55c7c8160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 -- 172.21.15.113:0/3532170234 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c7ea17d0 con 0x55c7c8169a80 2024-04-22T07:19:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f2065640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f73d80a3d20 0x7f73d80a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004af0 tx=0x7f73e4032450 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.119+0000 7f73f1063640 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 0x55c7c816be70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f73e800c8f0 tx=0x7f73e806f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9b2cf0a0db663f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73d7fff640 1 -- 172.21.15.113:0/3532170234 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8067370 con 0x55c7c8169a80 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73d7fff640 1 -- 172.21.15.113:0/3532170234 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73e809c860 con 0x55c7c8169a80 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73d7fff640 1 -- 172.21.15.113:0/3532170234 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e800f740 con 0x55c7c8169a80 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 -- 172.21.15.113:0/3532170234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 msgr2=0x55c7c816be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 0x55c7c816be70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f73e800c8f0 tx=0x7f73e806f350 comp rx=0 tx=0).stop 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 -- 172.21.15.113:0/3532170234 shutdown_connections 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8169a80 0x55c7c816be70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 --2- 172.21.15.113:0/3532170234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160580 0x55c7c8160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.120+0000 7f73f6799740 1 -- 172.21.15.113:0/3532170234 >> 172.21.15.113:0/3532170234 conn(0x55c7c815acb0 msgr2=0x55c7c815b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 -- 172.21.15.113:0/3532170234 shutdown_connections 2024-04-22T07:19:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 -- 172.21.15.113:0/3532170234 wait complete. 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 -- start start 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 Processor -- start 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8166b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8167060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c81675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c7c816c870 con 0x55c7c8160580 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c7c807e2f0 con 0x55c7c8169a80 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c7c816c9f0 con 0x55c7c8160ea0 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f6799740 10 client.0 _ll_register_callbacks cb 0x55c7c808b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f1864640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f1864640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8166b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57974/0 (socket says 172.21.15.113:57974) 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.121+0000 7f73f1864640 1 -- 172.21.15.113:0/1475009065 learned_addr learned my addr 172.21.15.113:0/1475009065 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f2065640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c81675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1063640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1864640 1 -- 172.21.15.113:0/1475009065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 msgr2=0x55c7c8167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1864640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8167060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1864640 1 -- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 msgr2=0x55c7c81675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1864640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c81675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1864640 1 -- 172.21.15.113:0/1475009065 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c7dd73f0 con 0x55c7c8160580 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1063640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8167060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f1864640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8166b20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f73dc010a50 tx=0x7f73dc010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60eaff7244d0a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73d57fa640 1 -- 172.21.15.113:0/1475009065 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc01e060 con 0x55c7c8160580 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73d57fa640 1 -- 172.21.15.113:0/1475009065 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f73dc00fbd0 con 0x55c7c8160580 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e809c860 con 0x55c7c8160580 2024-04-22T07:19:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73d57fa640 1 -- 172.21.15.113:0/1475009065 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f73dc025050 con 0x55c7c8160580 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f6799740 10 client.16683 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c7c7d970f0 con 0x55c7c8160580 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.122+0000 7f73d57fa640 1 -- 172.21.15.113:0/1475009065 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73e809c860 con 0x55c7c8160580 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73d57fa640 1 -- 172.21.15.113:0/1475009065 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f73dc00e3b0 con 0x55c7c8160580 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f6799740 10 client.16683 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f6799740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f6799740 20 client.16683 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f6799740 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c8179b80 0x55c7c817bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f6799740 4 client.16683 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c80d2c00 con 0x55c7c8179b80 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f1063640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c8179b80 0x55c7c817bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73d57fa640 10 client.16683 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.123+0000 7f73f1063640 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c8179b80 0x55c7c817bff0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07ab53d4e3aa516 server_cookie=c6d8274d969b615b in_seq=0 out_seq=0 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73d57fa640 1 -- 172.21.15.113:0/1475009065 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7c80d2c00 con 0x55c7c8179b80 2024-04-22T07:19:43.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73d57fa640 10 client.16683 handle_command_reply: tid=0 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 client.16683 shutdown 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 msgr2=0x55c7c8166b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8166b20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f73dc010a50 tx=0x7f73dc010f10 comp rx=0 tx=0).stop 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 shutdown_connections 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c8179b80 0x55c7c817bff0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c81675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160ea0 0x55c7c8167060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 --2- 172.21.15.113:0/1475009065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8166b20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73d57fa640 0 client.16683 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.124+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 >> 172.21.15.113:0/1475009065 conn(0x55c7c815acb0 msgr2=0x55c7c815ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.125+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 shutdown_connections 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.125+0000 7f73f6799740 1 -- 172.21.15.113:0/1475009065 wait complete. 2024-04-22T07:19:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.125+0000 7f73f6799740 20 client.16683 trim_cache size 0 max 16384 2024-04-22T07:19:43.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 Processor -- start 2024-04-22T07:19:43.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 -- start start 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160580 0x55c7c8169250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 0x55c7c8165a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c8175730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1b06f0 con 0x55c7c8160ea0 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1adf60 con 0x55c7c8169a80 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f6799740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1abdb0 con 0x55c7c8160580 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 0x55c7c8165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 0x55c7c8165a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57990/0 (socket says 172.21.15.113:57990) 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 -- 172.21.15.113:0/1144831570 learned_addr learned my addr 172.21.15.113:0/1144831570 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 -- 172.21.15.113:0/1144831570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160580 msgr2=0x55c7c8169250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160580 0x55c7c8169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 -- 172.21.15.113:0/1144831570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 msgr2=0x55c7c8175730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f2065640 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c8175730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c8175730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.142+0000 7f73f1063640 1 -- 172.21.15.113:0/1144831570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c81786a0 con 0x55c7c8160ea0 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f1063640 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 0x55c7c8165a60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f73e8098cc0 tx=0x7f73e80c7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667262c5259d0d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73d77fe640 1 -- 172.21.15.113:0/1144831570 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e809cf60 con 0x55c7c8160ea0 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73d77fe640 1 -- 172.21.15.113:0/1144831570 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73e80a0c10 con 0x55c7c8160ea0 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73d77fe640 1 -- 172.21.15.113:0/1144831570 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80ce290 con 0x55c7c8160ea0 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 -- 172.21.15.113:0/1144831570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 msgr2=0x55c7c8165a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 0x55c7c8165a60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f73e8098cc0 tx=0x7f73e80c7970 comp rx=0 tx=0).stop 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 -- 172.21.15.113:0/1144831570 shutdown_connections 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8169a80 0x55c7c8175730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160ea0 0x55c7c8165a60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 --2- 172.21.15.113:0/1144831570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8160580 0x55c7c8169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 -- 172.21.15.113:0/1144831570 >> 172.21.15.113:0/1144831570 conn(0x55c7c815acb0 msgr2=0x55c7c8159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 -- 172.21.15.113:0/1144831570 shutdown_connections 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.143+0000 7f73f6799740 1 -- 172.21.15.113:0/1144831570 wait complete. 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 -- start start 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 Processor -- start 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8171de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160ea0 0x55c7c816e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8169a80 0x55c7c816e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ac420 con 0x55c7c8160580 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ad890 con 0x55c7c8160ea0 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1acaf0 con 0x55c7c8169a80 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f6799740 10 client.0 _ll_register_callbacks cb 0x7f73dc00a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8171de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8171de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57996/0 (socket says 172.21.15.113:57996) 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 -- 172.21.15.113:0/90365617 learned_addr learned my addr 172.21.15.113:0/90365617 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1063640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160ea0 0x55c7c816e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f2065640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8169a80 0x55c7c816e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 -- 172.21.15.113:0/90365617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8169a80 msgr2=0x55c7c816e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8169a80 0x55c7c816e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 -- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160ea0 msgr2=0x55c7c816e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160ea0 0x55c7c816e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f1864640 1 -- 172.21.15.113:0/90365617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c816ec60 con 0x55c7c8160580 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.144+0000 7f73f2065640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8169a80 0x55c7c816e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f1864640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8171de0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00b010 tx=0x7f73dc00a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f914306d54a1d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73d57fa640 1 -- 172.21.15.113:0/90365617 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc02e060 con 0x55c7c8160580 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73d57fa640 1 -- 172.21.15.113:0/90365617 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f73dc00e9f0 con 0x55c7c8160580 2024-04-22T07:19:43.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e80a0c10 con 0x55c7c8160580 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73d57fa640 1 -- 172.21.15.113:0/90365617 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f73dc01e020 con 0x55c7c8160580 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 10 client.16689 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c7c81e0fc0 con 0x55c7c8160580 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73d57fa640 1 -- 172.21.15.113:0/90365617 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73e80a0c10 con 0x55c7c8160580 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73d57fa640 1 -- 172.21.15.113:0/90365617 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f73dc01a950 con 0x55c7c8160580 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 10 client.16689 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 20 client.16689 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c81e17a0 0x55c7c81e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 4 client.16689 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.145+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7c80cca70 con 0x55c7c81e17a0 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73f1063640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c81e17a0 0x55c7c81e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73d57fa640 10 client.16689 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73f1063640 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c81e17a0 0x55c7c81e3bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a776d4edb751ce1e server_cookie=29ddb5f80e352fee in_seq=0 out_seq=0 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73d57fa640 1 -- 172.21.15.113:0/90365617 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c7c80cca70 con 0x55c7c81e17a0 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73d57fa640 10 client.16689 handle_command_reply: tid=0 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73f6799740 1 client.16689 shutdown 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 msgr2=0x55c7c8171de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.146+0000 7f73f6799740 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8171de0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00b010 tx=0x7f73dc00a750 comp rx=0 tx=0).stop 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 shutdown_connections 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c7c81e17a0 0x55c7c81e3bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7c8169a80 0x55c7c816e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7c8160ea0 0x55c7c816e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 --2- 172.21.15.113:0/90365617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c7c8160580 0x55c7c8171de0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73d57fa640 0 client.16689 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 >> 172.21.15.113:0/90365617 conn(0x55c7c815acb0 msgr2=0x55c7c8166a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 shutdown_connections 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 1 -- 172.21.15.113:0/90365617 wait complete. 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f6799740 20 client.16689 trim_cache size 0 max 16384 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:19:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:19:43.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f73d80a3d20 msgr2=0x7f73d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f73d80a3d20 0x7f73d80a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004af0 tx=0x7f73e4032450 comp rx=0 tx=0).stop 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 msgr2=0x7f73ec198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 0x7f73ec198b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f73e800c920 tx=0x7f73e800cd20 comp rx=0 tx=0).stop 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 shutdown_connections 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f73d80a3d20 0x7f73d80a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73ec10c500 0x7f73ec19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73ec103ae0 0x7f73ec198b90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 --2- 172.21.15.113:0/807273510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73ec1031c0 0x7f73ec198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 >> 172.21.15.113:0/807273510 conn(0x7f73ec078000 msgr2=0x7f73ec0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 shutdown_connections 2024-04-22T07:19:43.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.147+0000 7f73f3ac8640 1 -- 172.21.15.113:0/807273510 wait complete. 2024-04-22T07:19:43.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:19:43.156 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- 172.21.15.113:0/4203290376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c0690e0 msgr2=0x7f786c069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- 172.21.15.113:0/4203290376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c0690e0 0x7f786c069540 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f785c009900 tx=0x7f785c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- 172.21.15.113:0/4203290376 shutdown_connections 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- 172.21.15.113:0/4203290376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 0x7f786c10eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- 172.21.15.113:0/4203290376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c0690e0 0x7f786c069540 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- 172.21.15.113:0/4203290376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c107fe0 0x7f786c1083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- 172.21.15.113:0/4203290376 >> 172.21.15.113:0/4203290376 conn(0x7f786c077e30 msgr2=0x7f786c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- 172.21.15.113:0/4203290376 shutdown_connections 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- 172.21.15.113:0/4203290376 wait complete. 2024-04-22T07:19:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 Processor -- start 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- start start 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c0690e0 0x7f786c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 0x7f786c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 0x7f786c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f786c112300 con 0x7f786c107fe0 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f786c112180 con 0x7f786c069a80 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.297+0000 7f78745ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f786c112480 con 0x7f786c0690e0 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 0x7f786c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 0x7f786c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58012/0 (socket says 172.21.15.113:58012) 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 -- 172.21.15.113:0/3226986166 learned_addr learned my addr 172.21.15.113:0/3226986166 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7871b65640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 0x7f786c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872366640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c0690e0 0x7f786c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 -- 172.21.15.113:0/3226986166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c0690e0 msgr2=0x7f786c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c0690e0 0x7f786c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 -- 172.21.15.113:0/3226986166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 msgr2=0x7f786c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 0x7f786c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 -- 172.21.15.113:0/3226986166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786c19b010 con 0x7f786c107fe0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872366640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c0690e0 0x7f786c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7871b65640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 0x7f786c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f7872b67640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 0x7f786c19a890 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f786400cad0 tx=0x7f786400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786400ae20 con 0x7f786c107fe0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786c19b300 con 0x7f786c107fe0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7864005020 con 0x7f786c107fe0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786400f6b0 con 0x7f786c107fe0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.298+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786c1a75a0 con 0x7f786c107fe0 2024-04-22T07:19:43.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.299+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786c068de0 con 0x7f786c107fe0 2024-04-22T07:19:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.300+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f78640051d0 con 0x7f786c107fe0 2024-04-22T07:19:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.301+0000 7f78637fe640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f78580a3d20 0x7f78580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.301+0000 7f7872366640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f78580a3d20 0x7f78580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.301+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f78640d0e30 con 0x7f786c107fe0 2024-04-22T07:19:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.301+0000 7f7872366640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f78580a3d20 0x7f78580a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.305+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7864014030 con 0x7f786c107fe0 2024-04-22T07:19:43.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.521+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f786c1a78f0 con 0x7f786c107fe0 2024-04-22T07:19:43.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.522+0000 7f78637fe640 1 -- 172.21.15.113:0/3226986166 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11894 (secure 0 0 0) 0x7f786409d350 con 0x7f786c107fe0 2024-04-22T07:19:43.520 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:19:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:18:21.107438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:19:43.520 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f78580a3d20 msgr2=0x7f78580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f78580a3d20 0x7f78580a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).stop 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 msgr2=0x7f786c19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 0x7f786c19a890 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f786400cad0 tx=0x7f786400cf90 comp rx=0 tx=0).stop 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 shutdown_connections 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f78580a3d20 0x7f78580a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f786c107fe0 0x7f786c19a890 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f786c069a80 0x7f786c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 --2- 172.21.15.113:0/3226986166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f786c0690e0 0x7f786c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 >> 172.21.15.113:0/3226986166 conn(0x7f786c077e30 msgr2=0x7f786c10d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 shutdown_connections 2024-04-22T07:19:43.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.524+0000 7f78745ca640 1 -- 172.21.15.113:0/3226986166 wait complete. 2024-04-22T07:19:43.533 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:19:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 -- 172.21.15.113:0/270713023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074102b40 msgr2=0x7f0074106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 --2- 172.21.15.113:0/270713023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074102b40 0x7f0074106b80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0070009900 tx=0x7f007002bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 -- 172.21.15.113:0/270713023 shutdown_connections 2024-04-22T07:19:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 --2- 172.21.15.113:0/270713023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074107260 0x7f007410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 --2- 172.21.15.113:0/270713023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074102b40 0x7f0074106b80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 --2- 172.21.15.113:0/270713023 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0074102190 0x7f0074102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 -- 172.21.15.113:0/270713023 >> 172.21.15.113:0/270713023 conn(0x7f00740793a0 msgr2=0x7f00740797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 -- 172.21.15.113:0/270713023 shutdown_connections 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.635+0000 7f007c36d640 1 -- 172.21.15.113:0/270713023 wait complete. 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 Processor -- start 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 -- start start 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074102190 0x7f007419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 0x7f007419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0074198680 0x7f0074198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00741122e0 con 0x7f0074107260 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0074112160 con 0x7f0074102190 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007c36d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0074112460 con 0x7f0074198680 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f0079908640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 0x7f007419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f0079908640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 0x7f007419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58038/0 (socket says 172.21.15.113:58038) 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f0079908640 1 -- 172.21.15.113:0/4170383280 learned_addr learned my addr 172.21.15.113:0/4170383280 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007a109640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074102190 0x7f007419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.636+0000 7f007a90a640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0074198680 0x7f0074198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f0079908640 1 -- 172.21.15.113:0/4170383280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0074198680 msgr2=0x7f0074198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f0079908640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0074198680 0x7f0074198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f0079908640 1 -- 172.21.15.113:0/4170383280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074102190 msgr2=0x7f007419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f0079908640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074102190 0x7f007419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f0079908640 1 -- 172.21.15.113:0/4170383280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0074199280 con 0x7f0074107260 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f0079908640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 0x7f007419eb00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0070000c00 tx=0x7f007002cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f006b7fe640 1 -- 172.21.15.113:0/4170383280 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070041070 con 0x7f0074107260 2024-04-22T07:19:43.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0074199510 con 0x7f0074107260 2024-04-22T07:19:43.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f006b7fe640 1 -- 172.21.15.113:0/4170383280 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f00700044d0 con 0x7f0074107260 2024-04-22T07:19:43.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f006b7fe640 1 -- 172.21.15.113:0/4170383280 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070003d50 con 0x7f0074107260 2024-04-22T07:19:43.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.637+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00741a56b0 con 0x7f0074107260 2024-04-22T07:19:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f006b7fe640 1 -- 172.21.15.113:0/4170383280 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0070044070 con 0x7f0074107260 2024-04-22T07:19:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 Processor -- start 2024-04-22T07:19:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 -- start start 2024-04-22T07:19:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb4d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb4de80 0x558f8eb56430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb56a00 0x558f8eb58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558f8eab21f0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558f8ea4f4b0 con 0x558f8eb4de80 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.640+0000 7f007f033740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558f8eaae1b0 con 0x558f8eb56a00 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb4d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb4d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58054/0 (socket says 172.21.15.113:58054) 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 -- 172.21.15.113:0/2283315911 learned_addr learned my addr 172.21.15.113:0/2283315911 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f006b7fe640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f00600a3d20 0x7f00600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a109640 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb4de80 0x558f8eb56430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f0079908640 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb56a00 0x558f8eb58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f00600a3d20 0x7f00600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f006b7fe640 1 -- 172.21.15.113:0/4170383280 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f00700f21a0 con 0x7f0074107260 2024-04-22T07:19:43.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 -- 172.21.15.113:0/2283315911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb56a00 msgr2=0x558f8eb58e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb56a00 0x558f8eb58e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 -- 172.21.15.113:0/2283315911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb4de80 msgr2=0x558f8eb56430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb4de80 0x558f8eb56430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 -- 172.21.15.113:0/2283315911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8ea411b0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f0079908640 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb56a00 0x558f8eb58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb4d8b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f006c005c40 tx=0x7f006c013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1fb6bf67b086f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f0068bfe640 1 -- 172.21.15.113:0/2283315911 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c022020 con 0x558f8eb4d4d0 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f0068bfe640 1 -- 172.21.15.113:0/2283315911 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f006c0134e0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f007a90a640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f00600a3d20 0x7f00600a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f006c023000 tx=0x7f006c021000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.641+0000 7f0068bfe640 1 -- 172.21.15.113:0/2283315911 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c0109b0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 -- 172.21.15.113:0/2283315911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 msgr2=0x558f8eb4d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb4d8b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f006c005c40 tx=0x7f006c013f50 comp rx=0 tx=0).stop 2024-04-22T07:19:43.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 -- 172.21.15.113:0/2283315911 shutdown_connections 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb56a00 0x558f8eb58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb4de80 0x558f8eb56430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 --2- 172.21.15.113:0/2283315911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb4d8b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 -- 172.21.15.113:0/2283315911 >> 172.21.15.113:0/2283315911 conn(0x558f8eb47ba0 msgr2=0x558f8eb47f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 -- 172.21.15.113:0/2283315911 shutdown_connections 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.642+0000 7f007f033740 1 -- 172.21.15.113:0/2283315911 wait complete. 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 -- start start 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 Processor -- start 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4de80 0x558f8eb536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb56a00 0x558f8eb53cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558f8eb59970 con 0x558f8eb4d4d0 2024-04-22T07:19:43.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558f8ea6b500 con 0x558f8eb56a00 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558f8eb59af0 con 0x558f8eb4de80 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007f033740 10 client.0 _ll_register_callbacks cb 0x558f8ea39b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58058/0 (socket says 172.21.15.113:58058) 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 -- 172.21.15.113:0/2727564045 learned_addr learned my addr 172.21.15.113:0/2727564045 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 -- 172.21.15.113:0/2727564045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4de80 msgr2=0x558f8eb536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4de80 0x558f8eb536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 -- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb56a00 msgr2=0x558f8eb53cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a90a640 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb56a00 0x558f8eb53cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb56a00 0x558f8eb53cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 -- 172.21.15.113:0/2727564045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8e7c43f0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f007a109640 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f006400b920 tx=0x7f006400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c155badac3db22 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f005dffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f005dffb640 1 -- 172.21.15.113:0/2727564045 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064005020 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.643+0000 7f005dffb640 1 -- 172.21.15.113:0/2727564045 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f00640051c0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f8ea51930 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f005dffb640 1 -- 172.21.15.113:0/2727564045 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f00640053c0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 10 client.16707 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558f8e88e7d0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f005dffb640 1 -- 172.21.15.113:0/2727564045 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x558f8ea51930 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f005dffb640 1 -- 172.21.15.113:0/2727564045 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f006401b500 con 0x558f8eb4d4d0 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 10 client.16707 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 20 client.16707 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb66d30 0x558f8eb69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 4 client.16707 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f8eabfc10 con 0x558f8eb66d30 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.644+0000 7f0079908640 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb66d30 0x558f8eb69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.645+0000 7f005dffb640 10 client.16707 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.645+0000 7f0079908640 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb66d30 0x558f8eb69160 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2fbbe630f347666 server_cookie=47d3e80cb300c963 in_seq=0 out_seq=0 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.645+0000 7f005dffb640 1 -- 172.21.15.113:0/2727564045 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f8eabfc10 con 0x558f8eb66d30 2024-04-22T07:19:43.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.645+0000 7f005dffb640 10 client.16707 handle_command_reply: tid=0 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.645+0000 7f007f033740 1 client.16707 shutdown 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 msgr2=0x558f8eb56330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f006400b920 tx=0x7f006400bde0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 shutdown_connections 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb66d30 0x558f8eb69160 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb56a00 0x558f8eb53cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f005dffb640 0 client.16707 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4de80 0x558f8eb536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 --2- 172.21.15.113:0/2727564045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 >> 172.21.15.113:0/2727564045 conn(0x558f8eb47ba0 msgr2=0x558f8eb498f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 shutdown_connections 2024-04-22T07:19:43.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 1 -- 172.21.15.113:0/2727564045 wait complete. 2024-04-22T07:19:43.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.646+0000 7f007f033740 20 client.16707 trim_cache size 0 max 16384 2024-04-22T07:19:43.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 Processor -- start 2024-04-22T07:19:43.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 -- start start 2024-04-22T07:19:43.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb55df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb6ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00741b08e0 con 0x558f8eb4de80 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00741ae150 con 0x558f8eb47ba0 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007f033740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00741abfc0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb6ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb6ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58068/0 (socket says 172.21.15.113:58068) 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 -- 172.21.15.113:0/3689708840 learned_addr learned my addr 172.21.15.113:0/3689708840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 -- 172.21.15.113:0/3689708840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 msgr2=0x558f8eb56330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a109640 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb55df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 -- 172.21.15.113:0/3689708840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 msgr2=0x558f8eb55df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb55df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.659+0000 7f007a90a640 1 -- 172.21.15.113:0/3689708840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8eb65800 con 0x558f8eb4de80 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007a90a640 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb6ea70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f006c014fd0 tx=0x7f006c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2309a6a8111662f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f005ffff640 1 -- 172.21.15.113:0/3689708840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c022040 con 0x558f8eb4de80 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f005ffff640 1 -- 172.21.15.113:0/3689708840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f006c009c00 con 0x558f8eb4de80 2024-04-22T07:19:43.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f005ffff640 1 -- 172.21.15.113:0/3689708840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c012820 con 0x558f8eb4de80 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 -- 172.21.15.113:0/3689708840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 msgr2=0x558f8eb6ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb6ea70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f006c014fd0 tx=0x7f006c009480 comp rx=0 tx=0).stop 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 -- 172.21.15.113:0/3689708840 shutdown_connections 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb6ea70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb56330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 --2- 172.21.15.113:0/3689708840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb55df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 -- 172.21.15.113:0/3689708840 >> 172.21.15.113:0/3689708840 conn(0x558f8eb46fa0 msgr2=0x558f8eb49860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 -- 172.21.15.113:0/3689708840 shutdown_connections 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.660+0000 7f007f033740 1 -- 172.21.15.113:0/3689708840 wait complete. 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 -- start start 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 Processor -- start 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb5a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb5abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb5b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00741ac610 con 0x558f8eb4de80 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00741ada80 con 0x558f8eb47ba0 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f00741acce0 con 0x558f8eb4d4d0 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007f033740 10 client.0 _ll_register_callbacks cb 0x7f0064004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb5b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb5b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58084/0 (socket says 172.21.15.113:58084) 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 -- 172.21.15.113:0/3312286851 learned_addr learned my addr 172.21.15.113:0/3312286851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a109640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb5a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f0079908640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb5abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 -- 172.21.15.113:0/3312286851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 msgr2=0x558f8eb5abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb5abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 -- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 msgr2=0x558f8eb5a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb5a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 -- 172.21.15.113:0/3312286851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f8ebcc2e0 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a109640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb5a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f0079908640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb5abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f007a90a640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb5b0f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f006c013800 tx=0x7f006c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c74816177e4022d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.661+0000 7f005dffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f005dffb640 1 -- 172.21.15.113:0/3312286851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c010630 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f005dffb640 1 -- 172.21.15.113:0/3312286851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f006c012820 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c009c00 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f005dffb640 1 -- 172.21.15.113:0/3312286851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f006c040050 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 10 client.16713 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558f8ebcd4e0 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f005dffb640 1 -- 172.21.15.113:0/3312286851 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f006c009c00 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f005dffb640 1 -- 172.21.15.113:0/3312286851 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f006c02f020 con 0x558f8eb4de80 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 10 client.16713 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 20 client.16713 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb56a00 0x558f8ebcfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 4 client.16713 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f8eab9a80 con 0x558f8eb56a00 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.662+0000 7f007a109640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb56a00 0x558f8ebcfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f005dffb640 10 client.16713 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007a109640 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb56a00 0x558f8ebcfa50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b58ec35a093e4d70 server_cookie=4b2cf247baf9f2e0 in_seq=0 out_seq=0 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f005dffb640 1 -- 172.21.15.113:0/3312286851 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f8eab9a80 con 0x558f8eb56a00 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f005dffb640 10 client.16713 handle_command_reply: tid=0 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 client.16713 shutdown 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 msgr2=0x558f8eb5b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb5b0f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f006c013800 tx=0x7f006c005c40 comp rx=0 tx=0).stop 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 shutdown_connections 2024-04-22T07:19:43.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558f8eb56a00 0x558f8ebcfa50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558f8eb4de80 0x558f8eb5b0f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f005dffb640 0 client.16713 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558f8eb4d4d0 0x558f8eb5abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 --2- 172.21.15.113:0/3312286851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f8eb47ba0 0x558f8eb5a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 >> 172.21.15.113:0/3312286851 conn(0x558f8eb46fa0 msgr2=0x558f8eb4e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 shutdown_connections 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 1 -- 172.21.15.113:0/3312286851 wait complete. 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.663+0000 7f007f033740 20 client.16713 trim_cache size 0 max 16384 2024-04-22T07:19:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:19:43.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f00600a3d20 msgr2=0x7f00600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f00600a3d20 0x7f00600a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f006c023000 tx=0x7f006c021000 comp rx=0 tx=0).stop 2024-04-22T07:19:43.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 msgr2=0x7f007419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 0x7f007419eb00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0070000c00 tx=0x7f007002cdf0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 shutdown_connections 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f00600a3d20 0x7f00600a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0074198680 0x7f0074198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0074107260 0x7f007419eb00 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 --2- 172.21.15.113:0/4170383280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0074102190 0x7f007419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 >> 172.21.15.113:0/4170383280 conn(0x7f00740793a0 msgr2=0x7f00740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 shutdown_connections 2024-04-22T07:19:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.664+0000 7f007c36d640 1 -- 172.21.15.113:0/4170383280 wait complete. 2024-04-22T07:19:43.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/1561662050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 msgr2=0x7f5dc010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/1561662050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc010dd30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009ca0 tx=0x7f5db802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/1561662050 shutdown_connections 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/1561662050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc010dd30 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/1561662050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc0101a40 0x7f5dc0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/1561662050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc0101090 0x7f5dc0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/1561662050 >> 172.21.15.113:0/1561662050 conn(0x7f5dc0077f50 msgr2=0x7f5dc0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/1561662050 shutdown_connections 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.778+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/1561662050 wait complete. 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 Processor -- start 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 -- start start 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc0101090 0x7f5dc0102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc0103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc01035a0 0x7f5dc01a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0111260 con 0x7f5dc0106160 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01110e0 con 0x7f5dc01035a0 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc7b0d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01113e0 con 0x7f5dc0101090 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc50a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc0103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc50a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc0103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58096/0 (socket says 172.21.15.113:58096) 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.779+0000 7f5dc50a8640 1 -- 172.21.15.113:0/4019083475 learned_addr learned my addr 172.21.15.113:0/4019083475 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc0101090 0x7f5dc0102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc01035a0 0x7f5dc01a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc50a8640 1 -- 172.21.15.113:0/4019083475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc0101090 msgr2=0x7f5dc0102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc0101090 0x7f5dc0102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc50a8640 1 -- 172.21.15.113:0/4019083475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc01035a0 msgr2=0x7f5dc01a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc01035a0 0x7f5dc01a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc50a8640 1 -- 172.21.15.113:0/4019083475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01a71a0 con 0x7f5dc0106160 2024-04-22T07:19:43.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.780+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc0103060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00d770 tx=0x7f5dbc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.781+0000 7f5db6ffd640 1 -- 172.21.15.113:0/4019083475 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dc0106160 2024-04-22T07:19:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.781+0000 7f5db6ffd640 1 -- 172.21.15.113:0/4019083475 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5dbc010460 con 0x7f5dc0106160 2024-04-22T07:19:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.781+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01a7490 con 0x7f5dc0106160 2024-04-22T07:19:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.781+0000 7f5db6ffd640 1 -- 172.21.15.113:0/4019083475 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00f620 con 0x7f5dc0106160 2024-04-22T07:19:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.781+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a79e0 con 0x7f5dc0106160 2024-04-22T07:19:43.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.783+0000 7f5db6ffd640 1 -- 172.21.15.113:0/4019083475 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5dbc01a070 con 0x7f5dc0106160 2024-04-22T07:19:43.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.784+0000 7f5dca7ce740 1 Processor -- start 2024-04-22T07:19:43.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.784+0000 7f5dca7ce740 1 -- start start 2024-04-22T07:19:43.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.784+0000 7f5db6ffd640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5dac0a3d50 0x7f5dac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.784+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed70a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.784+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed713430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.784+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 0x5570ed715e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5570ed66f1f0 con 0x5570ed70a4d0 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5570ed60c4b0 con 0x5570ed70ae80 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5570ed66b1b0 con 0x5570ed713a00 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5dac0a3d50 0x7f5dac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5db6ffd640 1 -- 172.21.15.113:0/4019083475 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5dbc0d0360 con 0x7f5dc0106160 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed70a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed70a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58104/0 (socket says 172.21.15.113:58104) 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 -- 172.21.15.113:0/1120424811 learned_addr learned my addr 172.21.15.113:0/1120424811 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed713430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 -- 172.21.15.113:0/1120424811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 msgr2=0x5570ed715e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 0x5570ed715e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 -- 172.21.15.113:0/1120424811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 msgr2=0x5570ed713430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed713430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 -- 172.21.15.113:0/1120424811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ed5fe1b0 con 0x5570ed70a4d0 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5dac0a3d50 0x7f5dac0a6210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5db8005e30 tx=0x7f5db8032420 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed70a8b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009a70 tx=0x7f5db0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7196fca918f1409c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dabfff640 1 -- 172.21.15.113:0/1120424811 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017070 con 0x5570ed70a4d0 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dabfff640 1 -- 172.21.15.113:0/1120424811 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5db0013470 con 0x5570ed70a4d0 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.785+0000 7f5dabfff640 1 -- 172.21.15.113:0/1120424811 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0012610 con 0x5570ed70a4d0 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1120424811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 msgr2=0x5570ed70a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed70a8b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009a70 tx=0x7f5db0009f30 comp rx=0 tx=0).stop 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1120424811 shutdown_connections 2024-04-22T07:19:43.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 0x5570ed715e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed713430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1120424811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed70a8b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1120424811 >> 172.21.15.113:0/1120424811 conn(0x5570ed704ba0 msgr2=0x5570ed704f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1120424811 shutdown_connections 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1120424811 wait complete. 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.786+0000 7f5dca7ce740 1 -- start start 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 Processor -- start 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed7132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed710640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 0x5570ed710b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5570ed716970 con 0x5570ed713a00 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5570ed628500 con 0x5570ed70ae80 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5570ed716af0 con 0x5570ed70a4d0 2024-04-22T07:19:43.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dca7ce740 10 client.0 _ll_register_callbacks cb 0x5570ed5f6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 0x5570ed710b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 0x5570ed710b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58120/0 (socket says 172.21.15.113:58120) 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 -- 172.21.15.113:0/1773350136 learned_addr learned my addr 172.21.15.113:0/1773350136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed710640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed7132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 -- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 msgr2=0x5570ed7132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed7132b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 -- 172.21.15.113:0/1773350136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 msgr2=0x5570ed710640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed710640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.787+0000 7f5dc60aa640 1 -- 172.21.15.113:0/1773350136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ed3813f0 con 0x5570ed713a00 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 0x5570ed710b80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009ca0 tx=0x7f5db802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc171884b077306 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5da97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5da97fa640 1 -- 172.21.15.113:0/1773350136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8051060 con 0x5570ed713a00 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5da97fa640 1 -- 172.21.15.113:0/1773350136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5db8004030 con 0x5570ed713a00 2024-04-22T07:19:43.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db0013470 con 0x5570ed713a00 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5da97fa640 1 -- 172.21.15.113:0/1773350136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5db8058050 con 0x5570ed713a00 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 10 client.16725 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5570ed44b7d0 con 0x5570ed713a00 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5da97fa640 1 -- 172.21.15.113:0/1773350136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5db0013470 con 0x5570ed713a00 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5da97fa640 1 -- 172.21.15.113:0/1773350136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f5db803f040 con 0x5570ed713a00 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 10 client.16725 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 10 client.16725 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 20 client.16725 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed723c50 0x5570ed7260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 4 client.16725 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.788+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570ed60e930 con 0x5570ed723c50 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.789+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed723c50 0x5570ed7260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.789+0000 7f5da97fa640 10 client.16725 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.789+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed723c50 0x5570ed7260d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf2c241bd93f9561 server_cookie=cfed66bf963717a0 in_seq=0 out_seq=0 2024-04-22T07:19:43.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5da97fa640 1 -- 172.21.15.113:0/1773350136 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570ed60e930 con 0x5570ed723c50 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5da97fa640 10 client.16725 handle_command_reply: tid=0 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 client.16725 shutdown 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 msgr2=0x5570ed710b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 0x5570ed710b80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009ca0 tx=0x7f5db802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 shutdown_connections 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed723c50 0x5570ed7260d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed713a00 0x5570ed710b80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed710640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/1773350136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed7132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5da97fa640 0 client.16725 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 >> 172.21.15.113:0/1773350136 conn(0x5570ed704ba0 msgr2=0x5570ed7068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 shutdown_connections 2024-04-22T07:19:43.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 1 -- 172.21.15.113:0/1773350136 wait complete. 2024-04-22T07:19:43.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.790+0000 7f5dca7ce740 20 client.16725 trim_cache size 0 max 16384 2024-04-22T07:19:43.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 Processor -- start 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 -- start start 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed71f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed703a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 0x5570ed712d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01b2a50 con 0x5570ed70a4d0 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01b02c0 con 0x5570ed70ae80 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01ae110 con 0x5570ed713a00 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dc58a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed71f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dc58a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed71f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58122/0 (socket says 172.21.15.113:58122) 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.803+0000 7f5dc58a9640 1 -- 172.21.15.113:0/3409449703 learned_addr learned my addr 172.21.15.113:0/3409449703 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dc58a9640 1 -- 172.21.15.113:0/3409449703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 msgr2=0x5570ed712d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 0x5570ed712d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dc58a9640 1 -- 172.21.15.113:0/3409449703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 msgr2=0x5570ed703a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed703a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dc58a9640 1 -- 172.21.15.113:0/3409449703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ed70eb90 con 0x5570ed70a4d0 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed71f830 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009630 tx=0x7f5db0009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3db5db33e5fbca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dab7fe640 1 -- 172.21.15.113:0/3409449703 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017040 con 0x5570ed70a4d0 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dab7fe640 1 -- 172.21.15.113:0/3409449703 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5db00093a0 con 0x5570ed70a4d0 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.804+0000 7f5dab7fe640 1 -- 172.21.15.113:0/3409449703 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00262f0 con 0x5570ed70a4d0 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- 172.21.15.113:0/3409449703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 msgr2=0x5570ed71f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed71f830 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009630 tx=0x7f5db0009110 comp rx=0 tx=0).stop 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- 172.21.15.113:0/3409449703 shutdown_connections 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed713a00 0x5570ed712d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed70ae80 0x5570ed703a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/3409449703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70a4d0 0x5570ed71f830 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- 172.21.15.113:0/3409449703 >> 172.21.15.113:0/3409449703 conn(0x5570ed704ba0 msgr2=0x5570ed703370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- 172.21.15.113:0/3409449703 shutdown_connections 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- 172.21.15.113:0/3409449703 wait complete. 2024-04-22T07:19:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- start start 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 Processor -- start 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed71c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 0x5570ed70d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed713a00 0x5570ed70d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01ae780 con 0x5570ed70ae80 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01afbf0 con 0x5570ed713a00 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01aee50 con 0x5570ed70a4d0 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dca7ce740 10 client.0 _ll_register_callbacks cb 0x7f5db8005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dc50a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 0x5570ed70d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dc50a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 0x5570ed70d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58132/0 (socket says 172.21.15.113:58132) 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.805+0000 7f5dc50a8640 1 -- 172.21.15.113:0/688599348 learned_addr learned my addr 172.21.15.113:0/688599348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc50a8640 1 -- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 msgr2=0x5570ed71c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed71c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc50a8640 1 -- 172.21.15.113:0/688599348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed713a00 msgr2=0x5570ed70d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed713a00 0x5570ed70d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed713a00 0x5570ed70d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc50a8640 1 -- 172.21.15.113:0/688599348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ed70deb0 con 0x5570ed70ae80 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc60aa640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed713a00 0x5570ed70d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dc50a8640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 0x5570ed70d070 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00dfd0 tx=0x7f5dbc09c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa8180858e70ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5da97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5da97fa640 1 -- 172.21.15.113:0/688599348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc021070 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5da97fa640 1 -- 172.21.15.113:0/688599348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5dbc0987f0 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db00093a0 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5da97fa640 1 -- 172.21.15.113:0/688599348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5dbc0d0180 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dca7ce740 10 client.16731 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5570ed78a2e0 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5da97fa640 1 -- 172.21.15.113:0/688599348 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5db00093a0 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5da97fa640 1 -- 172.21.15.113:0/688599348 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f5dbc054140 con 0x5570ed70ae80 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dca7ce740 10 client.16731 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dca7ce740 10 client.16731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.806+0000 7f5dca7ce740 20 client.16731 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed78aaf0 0x5570ed78cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 4 client.16731 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570ed710f10 con 0x5570ed78aaf0 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed78aaf0 0x5570ed78cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dc58a9640 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed78aaf0 0x5570ed78cee0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f6c3106125df845 server_cookie=44dc0dd302b04539 in_seq=0 out_seq=0 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5da97fa640 10 client.16731 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5da97fa640 1 -- 172.21.15.113:0/688599348 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570ed710f10 con 0x5570ed78aaf0 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5da97fa640 10 client.16731 handle_command_reply: tid=0 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 client.16731 shutdown 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 msgr2=0x5570ed70d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 0x5570ed70d070 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00dfd0 tx=0x7f5dbc09c2c0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 shutdown_connections 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5570ed78aaf0 0x5570ed78cee0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570ed713a00 0x5570ed70d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5570ed70ae80 0x5570ed70d070 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 --2- 172.21.15.113:0/688599348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5570ed70a4d0 0x5570ed71c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5da97fa640 0 client.16731 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:19:43.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 >> 172.21.15.113:0/688599348 conn(0x5570ed704ba0 msgr2=0x5570ed70baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.807+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 shutdown_connections 2024-04-22T07:19:43.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dca7ce740 1 -- 172.21.15.113:0/688599348 wait complete. 2024-04-22T07:19:43.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dca7ce740 20 client.16731 trim_cache size 0 max 16384 2024-04-22T07:19:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:19:43.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5dac0a3d50 msgr2=0x7f5dac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5dac0a3d50 0x7f5dac0a6210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5db8005e30 tx=0x7f5db8032420 comp rx=0 tx=0).stop 2024-04-22T07:19:43.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 msgr2=0x7f5dc0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc0103060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00d770 tx=0x7f5dbc00db70 comp rx=0 tx=0).stop 2024-04-22T07:19:43.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 shutdown_connections 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5dac0a3d50 0x7f5dac0a6210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc01035a0 0x7f5dc01a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5dc0106160 0x7f5dc0103060 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 --2- 172.21.15.113:0/4019083475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dc0101090 0x7f5dc0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 >> 172.21.15.113:0/4019083475 conn(0x7f5dc0077f50 msgr2=0x7f5dc0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 shutdown_connections 2024-04-22T07:19:43.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.808+0000 7f5dc7b0d640 1 -- 172.21.15.113:0/4019083475 wait complete. 2024-04-22T07:19:43.816 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:43 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3226986166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:19:43.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 -- 172.21.15.113:0/771338383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f694810e810 msgr2=0x7f6948110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 --2- 172.21.15.113:0/771338383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f694810e810 0x7f6948110c00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6940009ca0 tx=0x7f694002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 -- 172.21.15.113:0/771338383 shutdown_connections 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 --2- 172.21.15.113:0/771338383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f694810e810 0x7f6948110c00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 --2- 172.21.15.113:0/771338383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f694806aa90 0x7f694806aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 --2- 172.21.15.113:0/771338383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6948102e30 0x7f694806a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 -- 172.21.15.113:0/771338383 >> 172.21.15.113:0/771338383 conn(0x7f6948076c50 msgr2=0x7f6948077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 -- 172.21.15.113:0/771338383 shutdown_connections 2024-04-22T07:19:43.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 -- 172.21.15.113:0/771338383 wait complete. 2024-04-22T07:19:43.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.919+0000 7f694e1c7640 1 Processor -- start 2024-04-22T07:19:43.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 -- start start 2024-04-22T07:19:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694806aa90 0x7f69481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 0x7f69481a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f694810e810 0x7f694819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69481144e0 con 0x7f6948102e30 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6948114360 con 0x7f694806aa90 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f694e1c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6948114660 con 0x7f694810e810 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 0x7f69481a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 0x7f69481a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58144/0 (socket says 172.21.15.113:58144) 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 -- 172.21.15.113:0/3548728798 learned_addr learned my addr 172.21.15.113:0/3548728798 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f69477fe640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694806aa90 0x7f69481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 -- 172.21.15.113:0/3548728798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f694810e810 msgr2=0x7f694819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f694810e810 0x7f694819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 -- 172.21.15.113:0/3548728798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694806aa90 msgr2=0x7f69481a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694806aa90 0x7f69481a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 -- 172.21.15.113:0/3548728798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694819af90 con 0x7f6948102e30 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.920+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 0x7f69481a0b90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d770 tx=0x7f693c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.921+0000 7f6944ff9640 1 -- 172.21.15.113:0/3548728798 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c017070 con 0x7f6948102e30 2024-04-22T07:19:43.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.921+0000 7f6944ff9640 1 -- 172.21.15.113:0/3548728798 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f693c0054f0 con 0x7f6948102e30 2024-04-22T07:19:43.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.921+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694819b280 con 0x7f6948102e30 2024-04-22T07:19:43.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.921+0000 7f6944ff9640 1 -- 172.21.15.113:0/3548728798 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c004de0 con 0x7f6948102e30 2024-04-22T07:19:43.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.921+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69481a74a0 con 0x7f6948102e30 2024-04-22T07:19:43.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.923+0000 7f6950db8740 1 Processor -- start 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.923+0000 7f6950db8740 1 -- start start 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6944ff9640 1 -- 172.21.15.113:0/3548728798 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f693c004f80 con 0x7f6948102e30 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6950db8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6950db8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e749430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6950db8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e74be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6950db8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b75e6a51f0 con 0x55b75e749a00 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6950db8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b75e6424b0 con 0x55b75e740e80 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6950db8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b75e6a11b0 con 0x55b75e7404d0 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e74be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e74be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58152/0 (socket says 172.21.15.113:58152) 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 -- 172.21.15.113:0/3780540077 learned_addr learned my addr 172.21.15.113:0/3780540077 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f69477fe640 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e749430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6947fff640 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 -- 172.21.15.113:0/3780540077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 msgr2=0x55b75e7408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 -- 172.21.15.113:0/3780540077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 msgr2=0x55b75e749430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e749430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 -- 172.21.15.113:0/3780540077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75e6341b0 con 0x55b75e749a00 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6944ff9640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f69300a3d20 0x7f69300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6947fff640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f69300a3d20 0x7f69300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.924+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e74be50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f693c0cda80 tx=0x7f693c0d6bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f446b712cc29572 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6944ff9640 1 -- 172.21.15.113:0/3548728798 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f693c0d0540 con 0x7f6948102e30 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f692dffb640 1 -- 172.21.15.113:0/3780540077 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c098600 con 0x55b75e749a00 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f692dffb640 1 -- 172.21.15.113:0/3780540077 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f693c09ca60 con 0x55b75e749a00 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f692dffb640 1 -- 172.21.15.113:0/3780540077 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c09bcc0 con 0x55b75e749a00 2024-04-22T07:19:43.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6947fff640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f69300a3d20 0x7f69300a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6940005e30 tx=0x7f6940005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 -- 172.21.15.113:0/3780540077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 msgr2=0x55b75e74be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e74be50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f693c0cda80 tx=0x7f693c0d6bb0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 -- 172.21.15.113:0/3780540077 shutdown_connections 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e74be50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e749430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 --2- 172.21.15.113:0/3780540077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 -- 172.21.15.113:0/3780540077 >> 172.21.15.113:0/3780540077 conn(0x55b75e73aba0 msgr2=0x55b75e73af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 -- 172.21.15.113:0/3780540077 shutdown_connections 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.925+0000 7f6950db8740 1 -- 172.21.15.113:0/3780540077 wait complete. 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 -- start start 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 Processor -- start 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e746680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e746c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b75e74c970 con 0x55b75e749a00 2024-04-22T07:19:43.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b75e65e500 con 0x55b75e740e80 2024-04-22T07:19:43.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b75e74caf0 con 0x55b75e7404d0 2024-04-22T07:19:43.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6950db8740 10 client.0 _ll_register_callbacks cb 0x55b75e62cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6947fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e746c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6947fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e746c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58162/0 (socket says 172.21.15.113:58162) 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6947fff640 1 -- 172.21.15.113:0/3620667665 learned_addr learned my addr 172.21.15.113:0/3620667665 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f69477fe640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.926+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e746680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6947fff640 1 -- 172.21.15.113:0/3620667665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 msgr2=0x55b75e7492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6947fff640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6947fff640 1 -- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 msgr2=0x55b75e746680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6947fff640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e746680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6947fff640 1 -- 172.21.15.113:0/3620667665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75e3b73f0 con 0x55b75e749a00 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6946ffd640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e746680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6947fff640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e746c00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6940009ca0 tx=0x7f6940005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d861972c7c9a497 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6926ffd640 1 -- 172.21.15.113:0/3620667665 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940044070 con 0x55b75e749a00 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6926ffd640 1 -- 172.21.15.113:0/3620667665 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f694004b670 con 0x55b75e749a00 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f693c09ca60 con 0x55b75e749a00 2024-04-22T07:19:43.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6926ffd640 1 -- 172.21.15.113:0/3620667665 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6940059050 con 0x55b75e749a00 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 10 client.16743 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b75e4817d0 con 0x55b75e749a00 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6926ffd640 1 -- 172.21.15.113:0/3620667665 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f693c09ca60 con 0x55b75e749a00 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6926ffd640 1 -- 172.21.15.113:0/3620667665 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f6940034a90 con 0x55b75e749a00 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 10 client.16743 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 10 client.16743 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 20 client.16743 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e759c90 0x55b75e75c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 4 client.16743 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.927+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b75e644930 con 0x55b75e759c90 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.928+0000 7f69477fe640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e759c90 0x55b75e75c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.928+0000 7f69477fe640 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e759c90 0x55b75e75c0d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1089ad0d3b59ff5f server_cookie=6adbdf1266b6a8b4 in_seq=0 out_seq=0 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.928+0000 7f6926ffd640 10 client.16743 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:43.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6926ffd640 1 -- 172.21.15.113:0/3620667665 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b75e644930 con 0x55b75e759c90 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6926ffd640 10 client.16743 handle_command_reply: tid=0 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 client.16743 shutdown 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 msgr2=0x55b75e746c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e746c00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6940009ca0 tx=0x7f6940005dc0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 shutdown_connections 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e759c90 0x55b75e75c0d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e749a00 0x55b75e746c00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6926ffd640 0 client.16743 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e740e80 0x55b75e746680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 --2- 172.21.15.113:0/3620667665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e7492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 >> 172.21.15.113:0/3620667665 conn(0x55b75e73aba0 msgr2=0x55b75e73c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 shutdown_connections 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 1 -- 172.21.15.113:0/3620667665 wait complete. 2024-04-22T07:19:43.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.929+0000 7f6950db8740 20 client.16743 trim_cache size 0 max 16384 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.942+0000 7f6950db8740 1 Processor -- start 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.942+0000 7f6950db8740 1 -- start start 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.942+0000 7f6950db8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e755830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.942+0000 7f6950db8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e748db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.942+0000 7f6950db8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e749a00 0x55b75e7492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.942+0000 7f6950db8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b29d0 con 0x55b75e740e80 2024-04-22T07:19:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6950db8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f69481b0240 con 0x55b75e749a00 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6950db8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69481ae090 con 0x55b75e7404d0 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e748db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e748db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58168/0 (socket says 172.21.15.113:58168) 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 -- 172.21.15.113:0/1038906363 learned_addr learned my addr 172.21.15.113:0/1038906363 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 -- 172.21.15.113:0/1038906363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 msgr2=0x55b75e755830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e755830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 -- 172.21.15.113:0/1038906363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e749a00 msgr2=0x55b75e7492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e749a00 0x55b75e7492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 -- 172.21.15.113:0/1038906363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75e744bd0 con 0x55b75e740e80 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f6946ffd640 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e748db0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f693c0ce110 tx=0x7f693c005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ba2960d4feff31 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f692d7fa640 1 -- 172.21.15.113:0/1038906363 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c020040 con 0x55b75e740e80 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f692d7fa640 1 -- 172.21.15.113:0/1038906363 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f693c09ca60 con 0x55b75e740e80 2024-04-22T07:19:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.943+0000 7f692d7fa640 1 -- 172.21.15.113:0/1038906363 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c09bea0 con 0x55b75e740e80 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- 172.21.15.113:0/1038906363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 msgr2=0x55b75e748db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e748db0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f693c0ce110 tx=0x7f693c005ab0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- 172.21.15.113:0/1038906363 shutdown_connections 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e749a00 0x55b75e7492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e748db0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- 172.21.15.113:0/1038906363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e7404d0 0x55b75e755830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- 172.21.15.113:0/1038906363 >> 172.21.15.113:0/1038906363 conn(0x55b75e73aba0 msgr2=0x55b75e739c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- 172.21.15.113:0/1038906363 shutdown_connections 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- 172.21.15.113:0/1038906363 wait complete. 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- start start 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 Processor -- start 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e7404d0 0x55b75e7435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e743ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e749a00 0x55b75e744020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f69481ae700 con 0x55b75e740e80 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f69481afb70 con 0x55b75e7404d0 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.944+0000 7f6950db8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69481aedd0 con 0x55b75e749a00 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6950db8740 10 client.0 _ll_register_callbacks cb 0x7f69400325a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e743ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e743ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58180/0 (socket says 172.21.15.113:58180) 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 -- 172.21.15.113:0/836401568 learned_addr learned my addr 172.21.15.113:0/836401568 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 -- 172.21.15.113:0/836401568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e749a00 msgr2=0x55b75e744020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f69477fe640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e7404d0 0x55b75e7435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e749a00 0x55b75e744020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 -- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e7404d0 msgr2=0x55b75e7435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e7404d0 0x55b75e7435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 -- 172.21.15.113:0/836401568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75e7bf460 con 0x55b75e740e80 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f69477fe640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e7404d0 0x55b75e7435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6946ffd640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e743ae0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f693c01d000 tx=0x7f693c0d3c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200bb6ebc1fe06be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6926ffd640 1 -- 172.21.15.113:0/836401568 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c0b4070 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6926ffd640 1 -- 172.21.15.113:0/836401568 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f693c0d60d0 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f693c09ca60 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6926ffd640 1 -- 172.21.15.113:0/836401568 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f693c020050 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6950db8740 10 client.16749 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b75e7c0740 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.945+0000 7f6926ffd640 1 -- 172.21.15.113:0/836401568 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f693c07c080 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6926ffd640 1 -- 172.21.15.113:0/836401568 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f693c0cdd90 con 0x55b75e740e80 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 10 client.16749 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 10 client.16749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 20 client.16749 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e7c09b0 0x55b75e7c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 4 client.16749 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:19:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b75e6b2c10 con 0x55b75e7c09b0 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f69477fe640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e7c09b0 0x55b75e7c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f69477fe640 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e7c09b0 0x55b75e7c2dd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=161e3f655f2f9c2e server_cookie=370372c12ab8ec2a in_seq=0 out_seq=0 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6926ffd640 10 client.16749 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6926ffd640 1 -- 172.21.15.113:0/836401568 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b75e6b2c10 con 0x55b75e7c09b0 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6926ffd640 10 client.16749 handle_command_reply: tid=0 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.946+0000 7f6950db8740 1 client.16749 shutdown 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 msgr2=0x55b75e743ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e743ae0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f693c01d000 tx=0x7f693c0d3c20 comp rx=0 tx=0).stop 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 shutdown_connections 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55b75e7c09b0 0x55b75e7c2dd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b75e749a00 0x55b75e744020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b75e740e80 0x55b75e743ae0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6926ffd640 0 client.16749 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:19:43.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 --2- 172.21.15.113:0/836401568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b75e7404d0 0x55b75e7435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 >> 172.21.15.113:0/836401568 conn(0x55b75e73aba0 msgr2=0x55b75e7412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 shutdown_connections 2024-04-22T07:19:43.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 1 -- 172.21.15.113:0/836401568 wait complete. 2024-04-22T07:19:43.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f6950db8740 20 client.16749 trim_cache size 0 max 16384 2024-04-22T07:19:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f69300a3d20 msgr2=0x7f69300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f69300a3d20 0x7f69300a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6940005e30 tx=0x7f6940005fd0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 msgr2=0x7f69481a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 0x7f69481a0b90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d770 tx=0x7f693c00dc30 comp rx=0 tx=0).stop 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 shutdown_connections 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f69300a3d20 0x7f69300a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f694810e810 0x7f694819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6948102e30 0x7f69481a0b90 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 --2- 172.21.15.113:0/3548728798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694806aa90 0x7f69481a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 >> 172.21.15.113:0/3548728798 conn(0x7f6948076c50 msgr2=0x7f6948100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.947+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 shutdown_connections 2024-04-22T07:19:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:43.948+0000 7f694e1c7640 1 -- 172.21.15.113:0/3548728798 wait complete. 2024-04-22T07:19:44.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:43 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3226986166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:19:44.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:43 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3226986166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:19:44.957 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:19:44.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:19:44.957 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:19:45.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:44 smithi133 ceph-mon[31198]: pgmap v154: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:45.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:44 smithi154 ceph-mon[31862]: pgmap v154: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 -- 172.21.15.113:0/1469114546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb25806a840 msgr2=0x7fb25806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/1469114546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb25806a840 0x7fb25806ac20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb248009900 tx=0x7fb24802bc30 comp rx=0 tx=0).stop 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 -- 172.21.15.113:0/1469114546 shutdown_connections 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/1469114546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb258107270 0x7fb25810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/1469114546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb25806b1f0 0x7fb258106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/1469114546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb25806a840 0x7fb25806ac20 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 -- 172.21.15.113:0/1469114546 >> 172.21.15.113:0/1469114546 conn(0x7fb258077eb0 msgr2=0x7fb2580782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 -- 172.21.15.113:0/1469114546 shutdown_connections 2024-04-22T07:19:45.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.073+0000 7fb25e0c8640 1 -- 172.21.15.113:0/1469114546 wait complete. 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.074+0000 7fb25e0c8640 1 Processor -- start 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.074+0000 7fb25e0c8640 1 -- start start 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.074+0000 7fb25e0c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb25806a840 0x7fb25819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 0x7fb25819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2581985f0 0x7fb258198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581122f0 con 0x7fb258107270 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb258112170 con 0x7fb2581985f0 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb258112470 con 0x7fb25806a840 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 0x7fb25819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 0x7fb25819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58204/0 (socket says 172.21.15.113:58204) 2024-04-22T07:19:45.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 -- 172.21.15.113:0/104949203 learned_addr learned my addr 172.21.15.113:0/104949203 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb2577fe640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb25806a840 0x7fb25819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 -- 172.21.15.113:0/104949203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb25806a840 msgr2=0x7fb25819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb25806a840 0x7fb25819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 -- 172.21.15.113:0/104949203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2581985f0 msgr2=0x7fb258198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2581985f0 0x7fb258198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 -- 172.21.15.113:0/104949203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb258199110 con 0x7fb258107270 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb256ffd640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 0x7fb25819e980 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005fd0 tx=0x7fb24c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb254ff9640 1 -- 172.21.15.113:0/104949203 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c017070 con 0x7fb258107270 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb258199400 con 0x7fb258107270 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb254ff9640 1 -- 172.21.15.113:0/104949203 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb24c009c30 con 0x7fb258107270 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb254ff9640 1 -- 172.21.15.113:0/104949203 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c003de0 con 0x7fb258107270 2024-04-22T07:19:45.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.075+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2581a5650 con 0x7fb258107270 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb254ff9640 1 -- 172.21.15.113:0/104949203 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb24c003f80 con 0x7fb258107270 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 Processor -- start 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 -- start start 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e766acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7673840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7676260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5598e75cef00 con 0x5598e7673e10 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5598e756c590 con 0x5598e766a8e0 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb260d97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5598e75caec0 con 0x5598e766b290 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7676260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7676260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58218/0 (socket says 172.21.15.113:58218) 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb257fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e766acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 -- 172.21.15.113:0/294643161 learned_addr learned my addr 172.21.15.113:0/294643161 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb2577fe640 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7673840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb254ff9640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2400a3d20 0x7fb2400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 -- 172.21.15.113:0/294643161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 msgr2=0x5598e7673840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7673840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 -- 172.21.15.113:0/294643161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 msgr2=0x5598e766acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e766acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb256ffd640 1 -- 172.21.15.113:0/294643161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e73a97e0 con 0x5598e7673e10 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.078+0000 7fb257fff640 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e766acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb257fff640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2400a3d20 0x7fb2400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb254ff9640 1 -- 172.21.15.113:0/104949203 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb24c0d06d0 con 0x7fb258107270 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb256ffd640 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7676260 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0cdc10 tx=0x7fb24c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb93c6aad7643f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb23dffb640 1 -- 172.21.15.113:0/294643161 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c09c2c0 con 0x5598e7673e10 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb257fff640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2400a3d20 0x7fb2400a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb250005fd0 tx=0x7fb250004a40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb23dffb640 1 -- 172.21.15.113:0/294643161 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb24c009de0 con 0x5598e7673e10 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.079+0000 7fb23dffb640 1 -- 172.21.15.113:0/294643161 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c09be50 con 0x5598e7673e10 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- 172.21.15.113:0/294643161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 msgr2=0x5598e7676260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7676260 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0cdc10 tx=0x7fb24c021000 comp rx=0 tx=0).stop 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- 172.21.15.113:0/294643161 shutdown_connections 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7676260 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7673840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- 172.21.15.113:0/294643161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e766acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- 172.21.15.113:0/294643161 >> 172.21.15.113:0/294643161 conn(0x5598e7664fb0 msgr2=0x5598e7665390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- 172.21.15.113:0/294643161 shutdown_connections 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- 172.21.15.113:0/294643161 wait complete. 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- start start 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 Processor -- start 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e7670dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7671310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7671850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5598e7676d20 con 0x5598e7673e10 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5598e7588580 con 0x5598e766a8e0 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.080+0000 7fb260d97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5598e7676ea0 con 0x5598e766b290 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb260d97740 10 client.0 _ll_register_callbacks cb 0x5598e758e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7671850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7671850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58226/0 (socket says 172.21.15.113:58226) 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 -- 172.21.15.113:0/2289939372 learned_addr learned my addr 172.21.15.113:0/2289939372 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb256ffd640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7671310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb2577fe640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e7670dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 -- 172.21.15.113:0/2289939372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 msgr2=0x5598e7671310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7671310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 -- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 msgr2=0x5598e7670dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e7670dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 -- 172.21.15.113:0/2289939372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e7409f70 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb257fff640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7671850 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb25000ecc0 tx=0x7fb250009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12904a05b3968ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb236ffd640 1 -- 172.21.15.113:0/2289939372 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25002a060 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb236ffd640 1 -- 172.21.15.113:0/2289939372 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb250017070 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb24c009de0 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb236ffd640 1 -- 172.21.15.113:0/2289939372 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb250031050 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb260d97740 10 client.16761 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.081+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5598e72a1170 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb236ffd640 1 -- 172.21.15.113:0/2289939372 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb24c009de0 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb236ffd640 1 -- 172.21.15.113:0/2289939372 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7fb25001aa90 con 0x5598e7673e10 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb260d97740 10 client.16761 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb260d97740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb260d97740 20 client.16761 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:45.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb260d97740 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7684330 0x5598e76867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb260d97740 4 client.16761 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e75dc920 con 0x5598e7684330 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb2577fe640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7684330 0x5598e76867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb2577fe640 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7684330 0x5598e76867b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25840d29eae8f5ec server_cookie=a73ad7c5f36f69f6 in_seq=0 out_seq=0 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.082+0000 7fb236ffd640 10 client.16761 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb236ffd640 1 -- 172.21.15.113:0/2289939372 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598e75dc920 con 0x5598e7684330 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb236ffd640 10 client.16761 handle_command_reply: tid=0 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 client.16761 shutdown 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 msgr2=0x5598e7671850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7671850 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb25000ecc0 tx=0x7fb250009500 comp rx=0 tx=0).stop 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 shutdown_connections 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7684330 0x5598e76867b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7673e10 0x5598e7671850 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766b290 0x5598e7671310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb236ffd640 0 client.16761 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 --2- 172.21.15.113:0/2289939372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766a8e0 0x5598e7670dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.083+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 >> 172.21.15.113:0/2289939372 conn(0x5598e7664fb0 msgr2=0x5598e7666d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.084+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 shutdown_connections 2024-04-22T07:19:45.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.084+0000 7fb260d97740 1 -- 172.21.15.113:0/2289939372 wait complete. 2024-04-22T07:19:45.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.084+0000 7fb260d97740 20 client.16761 trim_cache size 0 max 16384 2024-04-22T07:19:45.088 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:44 smithi113 ceph-mon[27933]: pgmap v154: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:45.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.101+0000 7fb260d97740 1 Processor -- start 2024-04-22T07:19:45.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 -- start start 2024-04-22T07:19:45.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e7673500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e767fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e768c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581b08e0 con 0x5598e7664fb0 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581ae150 con 0x5598e766b290 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb260d97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581abfc0 con 0x5598e766a8e0 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e7673500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e7673500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58232/0 (socket says 172.21.15.113:58232) 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 -- 172.21.15.113:0/3169560034 learned_addr learned my addr 172.21.15.113:0/3169560034 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 -- 172.21.15.113:0/3169560034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 msgr2=0x5598e767fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e767fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 -- 172.21.15.113:0/3169560034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 msgr2=0x5598e768c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e768c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 -- 172.21.15.113:0/3169560034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e766c9b0 con 0x5598e7664fb0 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb2577fe640 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e7673500 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb24802c170 tx=0x7fb248005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06a6da8f4e4f698 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb23d7fa640 1 -- 172.21.15.113:0/3169560034 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248045dc0 con 0x5598e7664fb0 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb23d7fa640 1 -- 172.21.15.113:0/3169560034 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb24804a070 con 0x5598e7664fb0 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.102+0000 7fb23d7fa640 1 -- 172.21.15.113:0/3169560034 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248005040 con 0x5598e7664fb0 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 -- 172.21.15.113:0/3169560034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 msgr2=0x5598e7673500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e7673500 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb24802c170 tx=0x7fb248005a20 comp rx=0 tx=0).stop 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 -- 172.21.15.113:0/3169560034 shutdown_connections 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e768c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e767fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 --2- 172.21.15.113:0/3169560034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e7673500 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 -- 172.21.15.113:0/3169560034 >> 172.21.15.113:0/3169560034 conn(0x5598e766b9e0 msgr2=0x5598e7671620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 -- 172.21.15.113:0/3169560034 shutdown_connections 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 -- 172.21.15.113:0/3169560034 wait complete. 2024-04-22T07:19:45.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.103+0000 7fb260d97740 1 -- start start 2024-04-22T07:19:45.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 Processor -- start 2024-04-22T07:19:45.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e766ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e766e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e76e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581ac610 con 0x5598e7664fb0 2024-04-22T07:19:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581ada80 con 0x5598e766b290 2024-04-22T07:19:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb2581acce0 con 0x5598e766a8e0 2024-04-22T07:19:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 10 client.0 _ll_register_callbacks cb 0x7fb250009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e766ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e766ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58240/0 (socket says 172.21.15.113:58240) 2024-04-22T07:19:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 -- 172.21.15.113:0/2568089422 learned_addr learned my addr 172.21.15.113:0/2568089422 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb256ffd640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e766e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb257fff640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e76e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 -- 172.21.15.113:0/2568089422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 msgr2=0x5598e766e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e766e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 -- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 msgr2=0x5598e76e98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e76e98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 -- 172.21.15.113:0/2568089422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e76e9e30 con 0x5598e7664fb0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb257fff640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e76e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb2577fe640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e766ddd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb24802c7f0 tx=0x7fb248045410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1147713c0d98c2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb236ffd640 1 -- 172.21.15.113:0/2568089422 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248005040 con 0x5598e7664fb0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb236ffd640 1 -- 172.21.15.113:0/2568089422 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb2480051e0 con 0x5598e7664fb0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.104+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb24804a070 con 0x5598e7664fb0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb236ffd640 1 -- 172.21.15.113:0/2568089422 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb2480053e0 con 0x5598e7664fb0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 10 client.16767 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5598e76eaf70 con 0x5598e7664fb0 2024-04-22T07:19:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb236ffd640 1 -- 172.21.15.113:0/2568089422 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb24804a070 con 0x5598e7664fb0 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb236ffd640 1 -- 172.21.15.113:0/2568089422 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7fb248004850 con 0x5598e7664fb0 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 10 client.16767 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 20 client.16767 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7673e10 0x5598e76ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 4 client.16767 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598e75d6790 con 0x5598e7673e10 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.105+0000 7fb256ffd640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7673e10 0x5598e76ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.106+0000 7fb236ffd640 10 client.16767 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.106+0000 7fb256ffd640 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7673e10 0x5598e76ed780 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a96240f23dbcee9 server_cookie=899ebc01021933ff in_seq=0 out_seq=0 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb236ffd640 1 -- 172.21.15.113:0/2568089422 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598e75d6790 con 0x5598e7673e10 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb236ffd640 10 client.16767 handle_command_reply: tid=0 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 client.16767 shutdown 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 msgr2=0x5598e766ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e766ddd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb24802c7f0 tx=0x7fb248045410 comp rx=0 tx=0).stop 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 shutdown_connections 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5598e7673e10 0x5598e76ed780 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598e766b290 0x5598e76e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598e766a8e0 0x5598e766e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb236ffd640 0 client.16767 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 --2- 172.21.15.113:0/2568089422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5598e7664fb0 0x5598e766ddd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 >> 172.21.15.113:0/2568089422 conn(0x5598e766b9e0 msgr2=0x5598e7663780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 shutdown_connections 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 1 -- 172.21.15.113:0/2568089422 wait complete. 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.107+0000 7fb260d97740 20 client.16767 trim_cache size 0 max 16384 2024-04-22T07:19:45.110 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:19:45.111 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:19:45.111 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "72d640c7-01eb-44c4-b535-60635c1b9920", 2024-04-22T07:19:45.111 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:19:45.111 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2400a3d20 msgr2=0x7fb2400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2400a3d20 0x7fb2400a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb250005fd0 tx=0x7fb250004a40 comp rx=0 tx=0).stop 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 msgr2=0x7fb25819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 0x7fb25819e980 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb24c005fd0 tx=0x7fb24c00dde0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 shutdown_connections 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb2400a3d20 0x7fb2400a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2581985f0 0x7fb258198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb258107270 0x7fb25819e980 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 --2- 172.21.15.113:0/104949203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb25806a840 0x7fb25819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 >> 172.21.15.113:0/104949203 conn(0x7fb258077eb0 msgr2=0x7fb258108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 shutdown_connections 2024-04-22T07:19:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.108+0000 7fb25e0c8640 1 -- 172.21.15.113:0/104949203 wait complete. 2024-04-22T07:19:45.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:19:45.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 -- 172.21.15.113:0/3163596319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 msgr2=0x7f868010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 --2- 172.21.15.113:0/3163596319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868010ee40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ca0 tx=0x7f867802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 -- 172.21.15.113:0/3163596319 shutdown_connections 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 --2- 172.21.15.113:0/3163596319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868010ee40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 --2- 172.21.15.113:0/3163596319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680102b50 0x7f8680106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 --2- 172.21.15.113:0/3163596319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86801021a0 0x7f8680102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 -- 172.21.15.113:0/3163596319 >> 172.21.15.113:0/3163596319 conn(0x7f86800793e0 msgr2=0x7f86800797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 -- 172.21.15.113:0/3163596319 shutdown_connections 2024-04-22T07:19:45.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.219+0000 7f86848a8640 1 -- 172.21.15.113:0/3163596319 wait complete. 2024-04-22T07:19:45.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 Processor -- start 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 -- start start 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86801021a0 0x7f868019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680198630 0x7f8680198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8680112370 con 0x7f8680107270 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86801121f0 con 0x7f8680198630 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f86848a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86801124f0 con 0x7f86801021a0 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58260/0 (socket says 172.21.15.113:58260) 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 -- 172.21.15.113:0/1291405569 learned_addr learned my addr 172.21.15.113:0/1291405569 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f867ed9d640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680198630 0x7f8680198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f867e59c640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86801021a0 0x7f868019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 -- 172.21.15.113:0/1291405569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86801021a0 msgr2=0x7f868019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86801021a0 0x7f868019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 -- 172.21.15.113:0/1291405569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680198630 msgr2=0x7f8680198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680198630 0x7f8680198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f8675d9b640 1 -- 172.21.15.113:0/1291405569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8680199230 con 0x7f8680107270 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.220+0000 7f867ed9d640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680198630 0x7f8680198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.221+0000 7f8675d9b640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868019eab0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f866800c970 tx=0x7f866800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.221+0000 7f8677fff640 1 -- 172.21.15.113:0/1291405569 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668007bb0 con 0x7f8680107270 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.221+0000 7f8677fff640 1 -- 172.21.15.113:0/1291405569 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8668007d50 con 0x7f8680107270 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.221+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86801a5310 con 0x7f8680107270 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.221+0000 7f8677fff640 1 -- 172.21.15.113:0/1291405569 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800f450 con 0x7f8680107270 2024-04-22T07:19:45.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.221+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801a5710 con 0x7f8680107270 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.223+0000 7f8677fff640 1 -- 172.21.15.113:0/1291405569 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f86680046c0 con 0x7f8680107270 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.223+0000 7f8687579740 1 Processor -- start 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.223+0000 7f8687579740 1 -- start start 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8677fff640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f866c0a3d20 0x7f866c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8687579740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17de9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8687579740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17df2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8687579740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17df2a80 0x55cd17df4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8687579740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cd17d4e1e0 con 0x55cd17de9580 2024-04-22T07:19:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8687579740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cd17ceb390 con 0x55cd17de9ea0 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8687579740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd17d4a1a0 con 0x55cd17df2a80 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867ed9d640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f866c0a3d20 0x7f866c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8677fff640 1 -- 172.21.15.113:0/1291405569 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f86680d0a60 con 0x7f8680107270 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17de9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17de9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58270/0 (socket says 172.21.15.113:58270) 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 -- 172.21.15.113:0/3715789954 learned_addr learned my addr 172.21.15.113:0/3715789954 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867ed9d640 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17df2a80 0x55cd17df4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8675d9b640 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17df2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 -- 172.21.15.113:0/3715789954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17df2a80 msgr2=0x55cd17df4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17df2a80 0x55cd17df4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 -- 172.21.15.113:0/3715789954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 msgr2=0x55cd17df2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17df2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 -- 172.21.15.113:0/3715789954 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd17b2a7d0 con 0x55cd17de9580 2024-04-22T07:19:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867ed9d640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f866c0a3d20 0x7f866c0a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8678009430 tx=0x7f86780325a0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f867e59c640 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17de9960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f867000b820 tx=0x7f867000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a20a9cca4832f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8674d99640 1 -- 172.21.15.113:0/3715789954 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867000d8d0 con 0x55cd17de9580 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8674d99640 1 -- 172.21.15.113:0/3715789954 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8670019070 con 0x55cd17de9580 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.224+0000 7f8674d99640 1 -- 172.21.15.113:0/3715789954 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86700123e0 con 0x55cd17de9580 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 -- 172.21.15.113:0/3715789954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 msgr2=0x55cd17de9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17de9960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f867000b820 tx=0x7f867000bce0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 -- 172.21.15.113:0/3715789954 shutdown_connections 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17df2a80 0x55cd17df4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17df2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 --2- 172.21.15.113:0/3715789954 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17de9960 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 -- 172.21.15.113:0/3715789954 >> 172.21.15.113:0/3715789954 conn(0x55cd17de3cb0 msgr2=0x55cd17de4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 -- 172.21.15.113:0/3715789954 shutdown_connections 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.225+0000 7f8687579740 1 -- 172.21.15.113:0/3715789954 wait complete. 2024-04-22T07:19:45.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 -- start start 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 Processor -- start 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17defa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17deffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17df04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cd17df5870 con 0x55cd17df2a80 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cd17d072f0 con 0x55cd17de9ea0 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd17df59f0 con 0x55cd17de9580 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8687579740 10 client.0 _ll_register_callbacks cb 0x55cd17d14580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17df04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17df04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58284/0 (socket says 172.21.15.113:58284) 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 -- 172.21.15.113:0/2339853414 learned_addr learned my addr 172.21.15.113:0/2339853414 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f8675d9b640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17deffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867e59c640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17defa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 -- 172.21.15.113:0/2339853414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 msgr2=0x55cd17defa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17defa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 -- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 msgr2=0x55cd17deffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17deffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 -- 172.21.15.113:0/2339853414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd17a603f0 con 0x55cd17df2a80 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867e59c640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17defa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.226+0000 7f867ed9d640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17df04f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ca0 tx=0x7f867802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eedf7ab56bf369 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8665ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8665ffb640 1 -- 172.21.15.113:0/2339853414 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678052060 con 0x55cd17df2a80 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8665ffb640 1 -- 172.21.15.113:0/2339853414 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8678004450 con 0x55cd17df2a80 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8670019070 con 0x55cd17df2a80 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8665ffb640 1 -- 172.21.15.113:0/2339853414 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8678059050 con 0x55cd17df2a80 2024-04-22T07:19:45.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 10 client.16779 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cd17a200f0 con 0x55cd17df2a80 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8665ffb640 1 -- 172.21.15.113:0/2339853414 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8670019070 con 0x55cd17df2a80 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8665ffb640 1 -- 172.21.15.113:0/2339853414 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f8678034440 con 0x55cd17df2a80 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 10 client.16779 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 20 client.16779 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e02a60 0x55cd17e04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 4 client.16779 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.227+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd17d5bc00 con 0x55cd17e02a60 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.228+0000 7f867e59c640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e02a60 0x55cd17e04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.228+0000 7f867e59c640 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e02a60 0x55cd17e04e90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6518a76dd6cf1e18 server_cookie=d146d9bbbfe9084 in_seq=0 out_seq=0 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.228+0000 7f8665ffb640 10 client.16779 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.228+0000 7f8665ffb640 1 -- 172.21.15.113:0/2339853414 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd17d5bc00 con 0x55cd17e02a60 2024-04-22T07:19:45.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8665ffb640 10 client.16779 handle_command_reply: tid=0 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 client.16779 shutdown 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 msgr2=0x55cd17df04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17df04f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ca0 tx=0x7f867802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 shutdown_connections 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e02a60 0x55cd17e04e90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17df04f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17deffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8665ffb640 0 client.16779 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 --2- 172.21.15.113:0/2339853414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17defa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 >> 172.21.15.113:0/2339853414 conn(0x55cd17de3cb0 msgr2=0x55cd17de5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 shutdown_connections 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 1 -- 172.21.15.113:0/2339853414 wait complete. 2024-04-22T07:19:45.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.229+0000 7f8687579740 20 client.16779 trim_cache size 0 max 16384 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 Processor -- start 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 -- start start 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17df21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9ea0 0x55cd17dfe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17df2a80 0x55cd17de57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86801b08b0 con 0x55cd17de9580 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ae120 con 0x55cd17df2a80 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8687579740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86801abf90 con 0x55cd17de9ea0 2024-04-22T07:19:45.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17df21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17df21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58292/0 (socket says 172.21.15.113:58292) 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 -- 172.21.15.113:0/3189831873 learned_addr learned my addr 172.21.15.113:0/3189831873 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 -- 172.21.15.113:0/3189831873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9ea0 msgr2=0x55cd17dfe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867ed9d640 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17df2a80 0x55cd17de57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8675d9b640 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9ea0 0x55cd17dfe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9ea0 0x55cd17dfe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 -- 172.21.15.113:0/3189831873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17df2a80 msgr2=0x55cd17de57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17df2a80 0x55cd17de57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 -- 172.21.15.113:0/3189831873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd17dea4b0 con 0x55cd17de9580 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867ed9d640 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17df2a80 0x55cd17de57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f867e59c640 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17df21a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f867000b7f0 tx=0x7f8670014a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982a879abfb1eb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.246+0000 7f8667fff640 1 -- 172.21.15.113:0/3189831873 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867000dad0 con 0x55cd17de9580 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8667fff640 1 -- 172.21.15.113:0/3189831873 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8670019050 con 0x55cd17de9580 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8667fff640 1 -- 172.21.15.113:0/3189831873 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867001fc40 con 0x55cd17de9580 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 -- 172.21.15.113:0/3189831873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 msgr2=0x55cd17df21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17df21a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f867000b7f0 tx=0x7f8670014a80 comp rx=0 tx=0).stop 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 -- 172.21.15.113:0/3189831873 shutdown_connections 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17df2a80 0x55cd17de57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9ea0 0x55cd17dfe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 --2- 172.21.15.113:0/3189831873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17de9580 0x55cd17df21a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 -- 172.21.15.113:0/3189831873 >> 172.21.15.113:0/3189831873 conn(0x55cd17de3cb0 msgr2=0x55cd17de50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 -- 172.21.15.113:0/3189831873 shutdown_connections 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 -- 172.21.15.113:0/3189831873 wait complete. 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 -- start start 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.247+0000 7f8687579740 1 Processor -- start 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17dec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17decd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17ded2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ac5e0 con 0x55cd17df2a80 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ada50 con 0x55cd17de9ea0 2024-04-22T07:19:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86801accb0 con 0x55cd17de9580 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 10 client.0 _ll_register_callbacks cb 0x7f8678047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17ded2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17ded2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58302/0 (socket says 172.21.15.113:58302) 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 -- 172.21.15.113:0/3557148769 learned_addr learned my addr 172.21.15.113:0/3557148769 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 -- 172.21.15.113:0/3557148769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 msgr2=0x55cd17dec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17dec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 -- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 msgr2=0x55cd17decd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17decd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 -- 172.21.15.113:0/3557148769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd17e68110 con 0x55cd17df2a80 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f867ed9d640 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17ded2d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f867802c860 tx=0x7f86780346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6021289c415f3bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8665ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8665ffb640 1 -- 172.21.15.113:0/3557148769 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678052040 con 0x55cd17df2a80 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8665ffb640 1 -- 172.21.15.113:0/3557148769 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8678034c90 con 0x55cd17df2a80 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8670019050 con 0x55cd17df2a80 2024-04-22T07:19:45.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8665ffb640 1 -- 172.21.15.113:0/3557148769 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f867806b050 con 0x55cd17df2a80 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 10 client.16785 fetch_fsmap learned FSMap version 47 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.248+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cd17e69370 con 0x55cd17df2a80 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8665ffb640 1 -- 172.21.15.113:0/3557148769 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8670019050 con 0x55cd17df2a80 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8665ffb640 1 -- 172.21.15.113:0/3557148769 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f867804d6b0 con 0x55cd17df2a80 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8687579740 10 client.16785 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8687579740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8687579740 20 client.16785 populate_metadata read hostname 'smithi113' 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8687579740 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e69bd0 0x55cd17e6c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8687579740 4 client.16785 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd17d55a70 con 0x55cd17e69bd0 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f867e59c640 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e69bd0 0x55cd17e6c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f8665ffb640 10 client.16785 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.249+0000 7f867e59c640 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e69bd0 0x55cd17e6c050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf35e78c7b14744 server_cookie=745bd04c1e22d991 in_seq=0 out_seq=0 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8665ffb640 1 -- 172.21.15.113:0/3557148769 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cd17d55a70 con 0x55cd17e69bd0 2024-04-22T07:19:45.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8665ffb640 10 client.16785 handle_command_reply: tid=0 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 client.16785 shutdown 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 msgr2=0x55cd17ded2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17ded2d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f867802c860 tx=0x7f86780346f0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 shutdown_connections 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd17e69bd0 0x55cd17e6c050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd17df2a80 0x55cd17ded2d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8665ffb640 0 client.16785 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd17de9ea0 0x55cd17decd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 --2- 172.21.15.113:0/3557148769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd17de9580 0x55cd17dec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 >> 172.21.15.113:0/3557148769 conn(0x55cd17de3cb0 msgr2=0x55cd17de2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 shutdown_connections 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 1 -- 172.21.15.113:0/3557148769 wait complete. 2024-04-22T07:19:45.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f8687579740 20 client.16785 trim_cache size 0 max 16384 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: "72d640c7-01eb-44c4-b535-60635c1b9920": { 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "72d640c7-01eb-44c4-b535-60635c1b9920", 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:19:45.254 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:19:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f866c0a3d20 msgr2=0x7f866c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f866c0a3d20 0x7f866c0a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8678009430 tx=0x7f86780325a0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 msgr2=0x7f868019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:19:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868019eab0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f866800c970 tx=0x7f866800ce30 comp rx=0 tx=0).stop 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 shutdown_connections 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f866c0a3d20 0x7f866c0a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680198630 0x7f8680198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680107270 0x7f868019eab0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 --2- 172.21.15.113:0/1291405569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86801021a0 0x7f868019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.250+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 >> 172.21.15.113:0/1291405569 conn(0x7f86800793e0 msgr2=0x7f86800ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.251+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 shutdown_connections 2024-04-22T07:19:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:19:45.251+0000 7f86848a8640 1 -- 172.21.15.113:0/1291405569 wait complete. 2024-04-22T07:19:45.259 INFO:tasks.cephfs.filesystem:scrub status for tag:72d640c7-01eb-44c4-b535-60635c1b9920 - {'path': '/', 'tag': '72d640c7-01eb-44c4-b535-60635c1b9920', 'options': 'recursive,force'} 2024-04-22T07:19:46.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:45 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:19:46.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:45 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:46.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:45 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:19:46.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:45 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:46.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:45 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:19:46.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:45 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:19:46.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:45 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:46.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:45 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:19:46.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:45 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:46.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:45 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:19:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:45 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:19:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:45 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:45 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:19:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:45 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:19:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:45 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:19:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:46 smithi133 ceph-mon[31198]: pgmap v155: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:47.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:46 smithi154 ceph-mon[31862]: pgmap v155: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:46 smithi113 ceph-mon[27933]: pgmap v155: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:48 smithi133 ceph-mon[31198]: pgmap v156: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:49.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:48 smithi154 ceph-mon[31862]: pgmap v156: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:49.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:48 smithi113 ceph-mon[27933]: pgmap v156: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:50 smithi133 ceph-mon[31198]: pgmap v157: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:51.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:50 smithi154 ceph-mon[31862]: pgmap v157: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:51.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:50 smithi113 ceph-mon[27933]: pgmap v157: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:53.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:52 smithi133 ceph-mon[31198]: pgmap v158: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:53.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:52 smithi154 ceph-mon[31862]: pgmap v158: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:52 smithi113 ceph-mon[27933]: pgmap v158: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:54 smithi133 ceph-mon[31198]: pgmap v159: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:55.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:54 smithi154 ceph-mon[31862]: pgmap v159: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:54 smithi113 ceph-mon[27933]: pgmap v159: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:19:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:56 smithi133 ceph-mon[31198]: pgmap v160: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:56 smithi154 ceph-mon[31862]: pgmap v160: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:56 smithi113 ceph-mon[27933]: pgmap v160: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:19:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:58 smithi133 ceph-mon[31198]: pgmap v161: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:58 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:19:59.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:58 smithi154 ceph-mon[31862]: pgmap v161: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:59.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:58 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:19:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:58 smithi113 ceph-mon[27933]: pgmap v161: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:19:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:58 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:20:00.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:59 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:20:00.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:59 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:20:00.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:19:59 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:20:00.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:59 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:20:00.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:59 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:20:00.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:19:59 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:20:00.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:59 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:20:00.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:59 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:20:00.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:19:59 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:20:01.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:00 smithi133 ceph-mon[31198]: pgmap v162: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:01.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:00 smithi133 ceph-mon[31198]: overall HEALTH_OK 2024-04-22T07:20:01.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:00 smithi154 ceph-mon[31862]: pgmap v162: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:01.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:00 smithi154 ceph-mon[31862]: overall HEALTH_OK 2024-04-22T07:20:01.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:00 smithi113 ceph-mon[27933]: pgmap v162: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:01.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:00 smithi113 ceph-mon[27933]: overall HEALTH_OK 2024-04-22T07:20:03.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:02 smithi113 ceph-mon[27933]: pgmap v163: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:20:03.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:02 smithi133 ceph-mon[31198]: pgmap v163: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:20:03.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:02 smithi154 ceph-mon[31862]: pgmap v163: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:20:05.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:04 smithi113 ceph-mon[27933]: pgmap v164: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:05.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:04 smithi133 ceph-mon[31198]: pgmap v164: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:05.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:04 smithi154 ceph-mon[31862]: pgmap v164: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:07.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:06 smithi113 ceph-mon[27933]: pgmap v165: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:07.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:06 smithi133 ceph-mon[31198]: pgmap v165: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:07.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:06 smithi154 ceph-mon[31862]: pgmap v165: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:09.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:08 smithi113 ceph-mon[27933]: pgmap v166: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:09.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:08 smithi133 ceph-mon[31198]: pgmap v166: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:09.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:08 smithi154 ceph-mon[31862]: pgmap v166: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:11.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:10 smithi133 ceph-mon[31198]: pgmap v167: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:11.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:10 smithi113 ceph-mon[27933]: pgmap v167: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:11.117 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:10 smithi154 ceph-mon[31862]: pgmap v167: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:13.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:12 smithi113 ceph-mon[27933]: pgmap v168: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:13.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:12 smithi133 ceph-mon[31198]: pgmap v168: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:13.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:12 smithi154 ceph-mon[31862]: pgmap v168: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:15.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:20:15.261 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:20:15.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:14 smithi113 ceph-mon[27933]: pgmap v169: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:15.313 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:14 smithi133 ceph-mon[31198]: pgmap v169: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:15.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:14 smithi154 ceph-mon[31862]: pgmap v169: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-22T07:20:15.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 -- 172.21.15.113:0/3266709218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9478106180 msgr2=0x7f947810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 --2- 172.21.15.113:0/3266709218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9478106180 0x7f947810dd50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9470009900 tx=0x7f947002bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:15.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 -- 172.21.15.113:0/3266709218 shutdown_connections 2024-04-22T07:20:15.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 --2- 172.21.15.113:0/3266709218 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9478106180 0x7f947810dd50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 --2- 172.21.15.113:0/3266709218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478101a60 0x7f9478105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 --2- 172.21.15.113:0/3266709218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94781010b0 0x7f9478101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 -- 172.21.15.113:0/3266709218 >> 172.21.15.113:0/3266709218 conn(0x7f94780771d0 msgr2=0x7f94780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 -- 172.21.15.113:0/3266709218 shutdown_connections 2024-04-22T07:20:15.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 -- 172.21.15.113:0/3266709218 wait complete. 2024-04-22T07:20:15.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.377+0000 7f947d077640 1 Processor -- start 2024-04-22T07:20:15.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 -- start start 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94781010b0 0x7f94781a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9478106180 0x7f94781a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 0x7f947819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9478111220 con 0x7f947819a850 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94781110a0 con 0x7f94781010b0 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947d077640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94781113a0 con 0x7f9478106180 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 0x7f947819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 0x7f947819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55370/0 (socket says 172.21.15.113:55370) 2024-04-22T07:20:15.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 -- 172.21.15.113:0/1301161486 learned_addr learned my addr 172.21.15.113:0/1301161486 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f9476d9d640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94781010b0 0x7f94781a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947659c640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9478106180 0x7f94781a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 -- 172.21.15.113:0/1301161486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9478106180 msgr2=0x7f94781a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9478106180 0x7f94781a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 -- 172.21.15.113:0/1301161486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94781010b0 msgr2=0x7f94781a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94781010b0 0x7f94781a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947759e640 1 -- 172.21.15.113:0/1301161486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947819b3e0 con 0x7f947819a850 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f947659c640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9478106180 0x7f94781a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.378+0000 7f9476d9d640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94781010b0 0x7f94781a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.379+0000 7f947759e640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 0x7f947819ad00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9470005940 tx=0x7f947002cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.379+0000 7f945ffff640 1 -- 172.21.15.113:0/1301161486 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470041070 con 0x7f947819a850 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.379+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947819b670 con 0x7f947819a850 2024-04-22T07:20:15.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.379+0000 7f945ffff640 1 -- 172.21.15.113:0/1301161486 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9470004450 con 0x7f947819a850 2024-04-22T07:20:15.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.379+0000 7f945ffff640 1 -- 172.21.15.113:0/1301161486 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470003bb0 con 0x7f947819a850 2024-04-22T07:20:15.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.379+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a7800 con 0x7f947819a850 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.381+0000 7f945ffff640 1 -- 172.21.15.113:0/1301161486 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9470044070 con 0x7f947819a850 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.381+0000 7f947fc61740 1 Processor -- start 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 -- start start 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f945ffff640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94600a3d20 0x7f94600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365580 0x558276365960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365ea0 0x55827636e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x558276370e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5582762ca1e0 con 0x55827636ea80 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558276267390 con 0x558276365580 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947fc61740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5582762c61a0 con 0x558276365ea0 2024-04-22T07:20:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947659c640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94600a3d20 0x7f94600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x558276370e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x558276370e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55374/0 (socket says 172.21.15.113:55374) 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 -- 172.21.15.113:0/2341050579 learned_addr learned my addr 172.21.15.113:0/2341050579 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f9476d9d640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365580 0x558276365960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f945ffff640 1 -- 172.21.15.113:0/1301161486 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f94700f2270 con 0x7f947819a850 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 -- 172.21.15.113:0/2341050579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365ea0 msgr2=0x55827636e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947659c640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365ea0 0x55827636e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365ea0 0x55827636e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 -- 172.21.15.113:0/2341050579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365580 msgr2=0x558276365960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365580 0x558276365960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 -- 172.21.15.113:0/2341050579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582760a67d0 con 0x55827636ea80 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f9476d9d640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365580 0x558276365960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947659c640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365ea0 0x55827636e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:15.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947659c640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94600a3d20 0x7f94600a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f947810dc60 tx=0x7f946c009680 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.382+0000 7f947759e640 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x558276370e70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f94700efa00 tx=0x7f9470003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688bc3183d3d606b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f945d7fa640 1 -- 172.21.15.113:0/2341050579 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947004b070 con 0x55827636ea80 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f945d7fa640 1 -- 172.21.15.113:0/2341050579 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94700be8a0 con 0x55827636ea80 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f945d7fa640 1 -- 172.21.15.113:0/2341050579 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470003d50 con 0x55827636ea80 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 -- 172.21.15.113:0/2341050579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 msgr2=0x558276370e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x558276370e70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f94700efa00 tx=0x7f9470003680 comp rx=0 tx=0).stop 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 -- 172.21.15.113:0/2341050579 shutdown_connections 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x558276370e70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365ea0 0x55827636e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 --2- 172.21.15.113:0/2341050579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365580 0x558276365960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 -- 172.21.15.113:0/2341050579 >> 172.21.15.113:0/2341050579 conn(0x55827635fcb0 msgr2=0x558276360090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 -- 172.21.15.113:0/2341050579 shutdown_connections 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.383+0000 7f947fc61740 1 -- 172.21.15.113:0/2341050579 wait complete. 2024-04-22T07:20:15.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 -- start start 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 Processor -- start 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x55827636c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x55827636c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5582762ca1e0 con 0x55827636ea80 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582762832f0 con 0x558276365ea0 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558276371630 con 0x558276365580 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947fc61740 10 client.0 _ll_register_callbacks cb 0x558276290580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x55827636c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x55827636c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x55827636c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55384/0 (socket says 172.21.15.113:55384) 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947759e640 1 -- 172.21.15.113:0/1473354724 learned_addr learned my addr 172.21.15.113:0/1473354724 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f9476d9d640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947659c640 1 -- 172.21.15.113:0/1473354724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 msgr2=0x55827636bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947659c640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947659c640 1 -- 172.21.15.113:0/1473354724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 msgr2=0x55827636c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947659c640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x55827636c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947659c640 1 -- 172.21.15.113:0/1473354724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558275fdc3f0 con 0x558276365ea0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.384+0000 7f947759e640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x55827636c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f947659c640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x55827636c020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f947819bb50 tx=0x7f946c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89f511ce4931a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f945aff5640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f945aff5640 1 -- 172.21.15.113:0/1473354724 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c024070 con 0x558276365ea0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94700be8a0 con 0x558276365ea0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f945aff5640 1 -- 172.21.15.113:0/1473354724 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f946c004460 con 0x558276365ea0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f945aff5640 1 -- 172.21.15.113:0/1473354724 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f946c030050 con 0x558276365ea0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f947fc61740 10 client.26114 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558275f9c0f0 con 0x558276365ea0 2024-04-22T07:20:15.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.385+0000 7f945aff5640 1 -- 172.21.15.113:0/1473354724 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f94700be8a0 con 0x558276365ea0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f945aff5640 1 -- 172.21.15.113:0/1473354724 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f946c00f040 con 0x558276365ea0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f947fc61740 10 client.26114 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f947fc61740 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f947fc61740 20 client.26114 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f947fc61740 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x558276380ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f947fc61740 4 client.26114 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582762d7c00 con 0x55827637e6f0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f9476d9d640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x558276380ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f945aff5640 10 client.26114 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.386+0000 7f9476d9d640 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x558276380ae0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8203db138e673de server_cookie=c1d1a5244e1e7fb4 in_seq=0 out_seq=0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f945aff5640 1 -- 172.21.15.113:0/1473354724 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582762d7c00 con 0x55827637e6f0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f945aff5640 10 client.26114 handle_command_reply: tid=0 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 client.26114 shutdown 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 msgr2=0x55827636c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x55827636c020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f947819bb50 tx=0x7f946c004730 comp rx=0 tx=0).stop 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 shutdown_connections 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x558276380ae0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55827636ea80 0x55827636c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x55827636c020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f945aff5640 0 client.26114 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 --2- 172.21.15.113:0/1473354724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 >> 172.21.15.113:0/1473354724 conn(0x55827635fcb0 msgr2=0x558276361a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 shutdown_connections 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 1 -- 172.21.15.113:0/1473354724 wait complete. 2024-04-22T07:20:15.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.387+0000 7f947fc61740 20 client.26114 trim_cache size 0 max 16384 2024-04-22T07:20:15.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 Processor -- start 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 -- start start 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 0x55827637a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55827636ea80 0x55827636ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94781b2a80 con 0x558276365ea0 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94781b02f0 con 0x55827636ea80 2024-04-22T07:20:15.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947fc61740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94781ae140 con 0x558276365580 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 0x55827637a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947659c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 0x55827637a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55398/0 (socket says 172.21.15.113:55398) 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947659c640 1 -- 172.21.15.113:0/3535524739 learned_addr learned my addr 172.21.15.113:0/3535524739 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.405+0000 7f947759e640 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55827636ea80 0x55827636ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f9476d9d640 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f947659c640 1 -- 172.21.15.113:0/3535524739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 msgr2=0x55827636e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f947659c640 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f947659c640 1 -- 172.21.15.113:0/3535524739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55827636ea80 msgr2=0x55827636ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f947659c640 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55827636ea80 0x55827636ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f947659c640 1 -- 172.21.15.113:0/3535524739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558276365cd0 con 0x558276365ea0 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f9476d9d640 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f947659c640 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 0x55827637a370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f946c0095c0 tx=0x7f946c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acdee1b0060feba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f945cff9640 1 -- 172.21.15.113:0/3535524739 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c038070 con 0x558276365ea0 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f945cff9640 1 -- 172.21.15.113:0/3535524739 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f946c020ca0 con 0x558276365ea0 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.406+0000 7f945cff9640 1 -- 172.21.15.113:0/3535524739 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c024040 con 0x558276365ea0 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- 172.21.15.113:0/3535524739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 msgr2=0x55827637a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 0x55827637a370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f946c0095c0 tx=0x7f946c005cf0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- 172.21.15.113:0/3535524739 shutdown_connections 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55827636ea80 0x55827636ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365ea0 0x55827637a370 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- 172.21.15.113:0/3535524739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558276365580 0x55827636e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- 172.21.15.113:0/3535524739 >> 172.21.15.113:0/3535524739 conn(0x55827635fcb0 msgr2=0x558276361100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- 172.21.15.113:0/3535524739 shutdown_connections 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- 172.21.15.113:0/3535524739 wait complete. 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- start start 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 Processor -- start 2024-04-22T07:20:15.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 0x558276376c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x558276368490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55827636ea80 0x5582763689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f94781ae7b0 con 0x558276365580 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94781afc20 con 0x558276365ea0 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94781aee80 con 0x55827636ea80 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f947fc61740 10 client.0 _ll_register_callbacks cb 0x7f946c0098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 0x558276376c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.407+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 0x558276376c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55402/0 (socket says 172.21.15.113:55402) 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 -- 172.21.15.113:0/2162459823 learned_addr learned my addr 172.21.15.113:0/2162459823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f947759e640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55827636ea80 0x5582763689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 -- 172.21.15.113:0/2162459823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55827636ea80 msgr2=0x5582763689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55827636ea80 0x5582763689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 -- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 msgr2=0x558276368490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f947659c640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x558276368490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x558276368490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 -- 172.21.15.113:0/2162459823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558276369200 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f947759e640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55827636ea80 0x5582763689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f9476d9d640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 0x558276376c50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9468005fd0 tx=0x7f9468003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5894e70bcd74711 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f945aff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f945aff5640 1 -- 172.21.15.113:0/2162459823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468024020 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f945aff5640 1 -- 172.21.15.113:0/2162459823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f94680047a0 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946c020ca0 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f945aff5640 1 -- 172.21.15.113:0/2162459823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f946802f050 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f947fc61740 10 client.16803 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5582763e4bd0 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.408+0000 7f945aff5640 1 -- 172.21.15.113:0/2162459823 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f946c020ca0 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f945aff5640 1 -- 172.21.15.113:0/2162459823 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f9468010690 con 0x558276365580 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947fc61740 10 client.16803 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:15.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947fc61740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947fc61740 20 client.16803 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947fc61740 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x5582763e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947fc61740 4 client.16803 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582762d1a70 con 0x55827637e6f0 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947659c640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x5582763e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f945aff5640 10 client.16803 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.409+0000 7f947659c640 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x5582763e7440 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953a76480d6f6d6d server_cookie=e86ee70df04eb49a in_seq=0 out_seq=0 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f945aff5640 1 -- 172.21.15.113:0/2162459823 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582762d1a70 con 0x55827637e6f0 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f945aff5640 10 client.16803 handle_command_reply: tid=0 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 client.16803 shutdown 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 msgr2=0x558276376c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 0x558276376c50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9468005fd0 tx=0x7f9468003d40 comp rx=0 tx=0).stop 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 shutdown_connections 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55827637e6f0 0x5582763e7440 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55827636ea80 0x5582763689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558276365ea0 0x558276368490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f945aff5640 0 client.16803 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 --2- 172.21.15.113:0/2162459823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558276365580 0x558276376c50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 >> 172.21.15.113:0/2162459823 conn(0x55827635fcb0 msgr2=0x558276361070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 shutdown_connections 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 1 -- 172.21.15.113:0/2162459823 wait complete. 2024-04-22T07:20:15.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947fc61740 20 client.16803 trim_cache size 0 max 16384 2024-04-22T07:20:15.416 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:20:15.416 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:20:15.416 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:20:15.416 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94600a3d20 msgr2=0x7f94600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94600a3d20 0x7f94600a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f947810dc60 tx=0x7f946c009680 comp rx=0 tx=0).stop 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 msgr2=0x7f947819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 0x7f947819ad00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9470005940 tx=0x7f947002cba0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 shutdown_connections 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f94600a3d20 0x7f94600a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f947819a850 0x7f947819ad00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9478106180 0x7f94781a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 --2- 172.21.15.113:0/1301161486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94781010b0 0x7f94781a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.410+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 >> 172.21.15.113:0/1301161486 conn(0x7f94780771d0 msgr2=0x7f94780ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.411+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 shutdown_connections 2024-04-22T07:20:15.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.411+0000 7f947d077640 1 -- 172.21.15.113:0/1301161486 wait complete. 2024-04-22T07:20:15.419 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:20:15.419 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:20:15.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 -- 172.21.15.113:0/4094884101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 msgr2=0x7f72ec10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 --2- 172.21.15.113:0/4094884101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec10dc90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009cd0 tx=0x7f72e402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 -- 172.21.15.113:0/4094884101 shutdown_connections 2024-04-22T07:20:15.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 --2- 172.21.15.113:0/4094884101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec10dc90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 --2- 172.21.15.113:0/4094884101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 0x7f72ec101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 --2- 172.21.15.113:0/4094884101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ec105e30 0x7f72ec106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 -- 172.21.15.113:0/4094884101 >> 172.21.15.113:0/4094884101 conn(0x7f72ec077e30 msgr2=0x7f72ec078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 -- 172.21.15.113:0/4094884101 shutdown_connections 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 -- 172.21.15.113:0/4094884101 wait complete. 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 Processor -- start 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.560+0000 7f72f4159640 1 -- start start 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 0x7f72ec1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ec105e30 0x7f72ec19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec110140 con 0x7f72ec101b60 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec10ffc0 con 0x7f72ec1011a0 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1102c0 con 0x7f72ec105e30 2024-04-22T07:20:15.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55418/0 (socket says 172.21.15.113:55418) 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 -- 172.21.15.113:0/3196547292 learned_addr learned my addr 172.21.15.113:0/3196547292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f26f6640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ec105e30 0x7f72ec19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f1ef5640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 0x7f72ec1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 -- 172.21.15.113:0/3196547292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ec105e30 msgr2=0x7f72ec19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ec105e30 0x7f72ec19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 -- 172.21.15.113:0/3196547292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 msgr2=0x7f72ec1965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 0x7f72ec1965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 -- 172.21.15.113:0/3196547292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec19b540 con 0x7f72ec101b60 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f1ef5640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 0x7f72ec1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f16f4640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec196ae0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d950 tx=0x7f72e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8017030 con 0x7f72ec101b60 2024-04-22T07:20:15.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f72e8004020 con 0x7f72ec101b60 2024-04-22T07:20:15.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e80047a0 con 0x7f72ec101b60 2024-04-22T07:20:15.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec19b830 con 0x7f72ec101b60 2024-04-22T07:20:15.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.561+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec1a30c0 con 0x7f72ec101b60 2024-04-22T07:20:15.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.562+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ec068de0 con 0x7f72ec101b60 2024-04-22T07:20:15.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.563+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f72e80041d0 con 0x7f72ec101b60 2024-04-22T07:20:15.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.564+0000 7f72e2ffd640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f72d80a5f30 0x7f72d80a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.564+0000 7f72f1ef5640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f72d80a5f30 0x7f72d80a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.564+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f72e80d03d0 con 0x7f72ec101b60 2024-04-22T07:20:15.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.564+0000 7f72f1ef5640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f72d80a5f30 0x7f72d80a83f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0058c0 tx=0x7f72dc005cb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.569+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72e809c8f0 con 0x7f72ec101b60 2024-04-22T07:20:15.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.782+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72ec1978a0 con 0x7f72ec101b60 2024-04-22T07:20:15.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.782+0000 7f72e2ffd640 1 -- 172.21.15.113:0/3196547292 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11894 (secure 0 0 0) 0x7f72e809c1f0 con 0x7f72ec101b60 2024-04-22T07:20:15.780 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:20:15.780 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:18:21.107438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:20:15.780 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-04-22T07:20:15.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f72d80a5f30 msgr2=0x7f72d80a83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f72d80a5f30 0x7f72d80a83f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0058c0 tx=0x7f72dc005cb0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 msgr2=0x7f72ec196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec196ae0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f72e800d950 tx=0x7f72e800de10 comp rx=0 tx=0).stop 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 shutdown_connections 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f72d80a5f30 0x7f72d80a83f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72ec105e30 0x7f72ec19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ec101b60 0x7f72ec196ae0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 --2- 172.21.15.113:0/3196547292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec1011a0 0x7f72ec1965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 >> 172.21.15.113:0/3196547292 conn(0x7f72ec077e30 msgr2=0x7f72ec0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 shutdown_connections 2024-04-22T07:20:15.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.784+0000 7f72f4159640 1 -- 172.21.15.113:0/3196547292 wait complete. 2024-04-22T07:20:15.793 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 -- 172.21.15.113:0/2480403275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 msgr2=0x7f8c5410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2480403275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c5410e9b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009ca0 tx=0x7f8c4c02c230 comp rx=0 tx=0).stop 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 -- 172.21.15.113:0/2480403275 shutdown_connections 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2480403275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c5410e9b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2480403275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c54103af0 0x7f8c5410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2480403275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c541031d0 0x7f8c541035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 -- 172.21.15.113:0/2480403275 >> 172.21.15.113:0/2480403275 conn(0x7f8c54077f50 msgr2=0x7f8c54078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.896+0000 7f8c590a8640 1 -- 172.21.15.113:0/2480403275 shutdown_connections 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.897+0000 7f8c590a8640 1 -- 172.21.15.113:0/2480403275 wait complete. 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.897+0000 7f8c590a8640 1 Processor -- start 2024-04-22T07:20:15.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.897+0000 7f8c590a8640 1 -- start start 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c541031d0 0x7f8c5419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c54103af0 0x7f8c5419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c54198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54112340 con 0x7f8c5410c510 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541121c0 con 0x7f8c541031d0 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541124c0 con 0x7f8c54103af0 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c54198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c54198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39434/0 (socket says 172.21.15.113:39434) 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 -- 172.21.15.113:0/2628385531 learned_addr learned my addr 172.21.15.113:0/2628385531 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c541031d0 0x7f8c5419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5259c640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c54103af0 0x7f8c5419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 -- 172.21.15.113:0/2628385531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c54103af0 msgr2=0x7f8c5419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c54103af0 0x7f8c5419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 -- 172.21.15.113:0/2628385531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c541031d0 msgr2=0x7f8c5419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c541031d0 0x7f8c5419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 -- 172.21.15.113:0/2628385531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c54198da0 con 0x7f8c5410c510 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c541031d0 0x7f8c5419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c5359e640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c54198620 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02c840 tx=0x7f8c4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c3bfff640 1 -- 172.21.15.113:0/2628385531 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c039040 con 0x7f8c5410c510 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c54199030 con 0x7f8c5410c510 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c3bfff640 1 -- 172.21.15.113:0/2628385531 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8c4c044070 con 0x7f8c5410c510 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c3bfff640 1 -- 172.21.15.113:0/2628385531 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004570 con 0x7f8c5410c510 2024-04-22T07:20:15.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.898+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a52e0 con 0x7f8c5410c510 2024-04-22T07:20:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c3bfff640 1 -- 172.21.15.113:0/2628385531 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8c4c004710 con 0x7f8c5410c510 2024-04-22T07:20:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 Processor -- start 2024-04-22T07:20:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 -- start start 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40ccbe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccc7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e40cc231f0 con 0x55e40ccbee80 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e40cbc04b0 con 0x55e40ccc7a00 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e40cc1f1b0 con 0x55e40ccbe4d0 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccc7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccc7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39440/0 (socket says 172.21.15.113:39440) 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 -- 172.21.15.113:0/598702096 learned_addr learned my addr 172.21.15.113:0/598702096 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5259c640 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5259c640 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40ccbe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 -- 172.21.15.113:0/598702096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 msgr2=0x55e40ccbe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40ccbe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 -- 172.21.15.113:0/598702096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 msgr2=0x55e40ccc9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 -- 172.21.15.113:0/598702096 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40cbb21b0 con 0x55e40ccbee80 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c3bfff640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8c3c0a3d20 0x7f8c3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c5259c640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8c3c0a3d20 0x7f8c3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.901+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccc7430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8c44007850 tx=0x7f8c44007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517f63d3bee9b17a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c3bfff640 1 -- 172.21.15.113:0/2628385531 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8c4c0f3120 con 0x7f8c5410c510 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c397fa640 1 -- 172.21.15.113:0/598702096 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400d8d0 con 0x55e40ccbee80 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c397fa640 1 -- 172.21.15.113:0/598702096 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8c44018070 con 0x55e40ccbee80 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c397fa640 1 -- 172.21.15.113:0/598702096 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44013de0 con 0x55e40ccbee80 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/598702096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 msgr2=0x55e40ccc7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5259c640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8c3c0a3d20 0x7f8c3c0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800f010 tx=0x7f8c480046c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccc7430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8c44007850 tx=0x7f8c44007d10 comp rx=0 tx=0).stop 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/598702096 shutdown_connections 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccc7430 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/598702096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40ccbe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/598702096 >> 172.21.15.113:0/598702096 conn(0x55e40ccb8ba0 msgr2=0x55e40ccb8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/598702096 shutdown_connections 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.902+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/598702096 wait complete. 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 -- start start 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 Processor -- start 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 0x55e40ccc7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbee80 0x55e40ccc45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccc7a00 0x55e40ccc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e40ccca970 con 0x55e40ccbe4d0 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e40cbdc500 con 0x55e40ccbee80 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e40cccaaf0 con 0x55e40ccc7a00 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5bd7b740 10 client.0 _ll_register_callbacks cb 0x55e40cbaab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:15.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 0x55e40ccc7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 0x55e40ccc7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39442/0 (socket says 172.21.15.113:39442) 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 -- 172.21.15.113:0/840345561 learned_addr learned my addr 172.21.15.113:0/840345561 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5359e640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccc7a00 0x55e40ccc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c5259c640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbee80 0x55e40ccc45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 -- 172.21.15.113:0/840345561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccc7a00 msgr2=0x55e40ccc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccc7a00 0x55e40ccc4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 -- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbee80 msgr2=0x55e40ccc45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbee80 0x55e40ccc45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 -- 172.21.15.113:0/840345561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40c9353f0 con 0x55e40ccbe4d0 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.903+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 0x55e40ccc7230 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006f40 tx=0x7f8c44013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a8ffa72bafb162 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:15.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c36ff5640 1 -- 172.21.15.113:0/840345561 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400dad0 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c36ff5640 1 -- 172.21.15.113:0/840345561 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8c4400dc70 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c44018070 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c36ff5640 1 -- 172.21.15.113:0/840345561 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8c44013e50 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 10 client.16821 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e40c9ff7d0 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c36ff5640 1 -- 172.21.15.113:0/840345561 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8c44018070 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c36ff5640 1 -- 172.21.15.113:0/840345561 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f8c4401c3f0 con 0x55e40ccbe4d0 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 10 client.16821 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 20 client.16821 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40ccd7b30 0x55e40ccd9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 4 client.16821 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.904+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40cbc2930 con 0x55e40ccd7b30 2024-04-22T07:20:15.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.905+0000 7f8c5259c640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40ccd7b30 0x55e40ccd9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.905+0000 7f8c36ff5640 10 client.16821 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.905+0000 7f8c5259c640 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40ccd7b30 0x55e40ccd9f20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be7fa2dc6143993 server_cookie=f7d42f3d3785f0a0 in_seq=0 out_seq=0 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c36ff5640 1 -- 172.21.15.113:0/840345561 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e40cbc2930 con 0x55e40ccd7b30 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c36ff5640 10 client.16821 handle_command_reply: tid=0 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 client.16821 shutdown 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 msgr2=0x55e40ccc7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 0x55e40ccc7230 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006f40 tx=0x7f8c44013730 comp rx=0 tx=0).stop 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 shutdown_connections 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40ccd7b30 0x55e40ccd9f20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccc7a00 0x55e40ccc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbee80 0x55e40ccc45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/840345561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbe4d0 0x55e40ccc7230 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c36ff5640 0 client.16821 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 >> 172.21.15.113:0/840345561 conn(0x55e40ccb8ba0 msgr2=0x55e40ccba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 shutdown_connections 2024-04-22T07:20:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/840345561 wait complete. 2024-04-22T07:20:15.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.906+0000 7f8c5bd7b740 20 client.16821 trim_cache size 0 max 16384 2024-04-22T07:20:15.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 Processor -- start 2024-04-22T07:20:15.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 -- start start 2024-04-22T07:20:15.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbe4d0 0x55e40ccd3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbee80 0x55e40ccc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 0x55e40ccc7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b0760 con 0x55e40ccc7a00 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541adfd0 con 0x55e40ccbe4d0 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541abe20 con 0x55e40ccbee80 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 0x55e40ccc7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 0x55e40ccc7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39444/0 (socket says 172.21.15.113:39444) 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 -- 172.21.15.113:0/1232919869 learned_addr learned my addr 172.21.15.113:0/1232919869 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5259c640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbee80 0x55e40ccc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbe4d0 0x55e40ccd3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 -- 172.21.15.113:0/1232919869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbee80 msgr2=0x55e40ccc6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbee80 0x55e40ccc6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 -- 172.21.15.113:0/1232919869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbe4d0 msgr2=0x55e40ccd3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbe4d0 0x55e40ccd3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5359e640 1 -- 172.21.15.113:0/1232919869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40ccd4400 con 0x55e40ccc7a00 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c5259c640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbee80 0x55e40ccc6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:15.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.919+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbe4d0 0x55e40ccd3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5359e640 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 0x55e40ccc7230 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009dc0 tx=0x7f8c4c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c058238c33ae4041 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c38ff9640 1 -- 172.21.15.113:0/1232919869 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0bf050 con 0x55e40ccc7a00 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c38ff9640 1 -- 172.21.15.113:0/1232919869 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8c4c0c3400 con 0x55e40ccc7a00 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c38ff9640 1 -- 172.21.15.113:0/1232919869 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0bf2b0 con 0x55e40ccc7a00 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/1232919869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 msgr2=0x55e40ccc7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 0x55e40ccc7230 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009dc0 tx=0x7f8c4c0095c0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/1232919869 shutdown_connections 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccc7a00 0x55e40ccc7230 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbee80 0x55e40ccc6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/1232919869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccbe4d0 0x55e40ccd3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/1232919869 >> 172.21.15.113:0/1232919869 conn(0x55e40ccb8ba0 msgr2=0x55e40ccb7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/1232919869 shutdown_connections 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/1232919869 wait complete. 2024-04-22T07:20:15.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.920+0000 7f8c5bd7b740 1 -- start start 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 Processor -- start 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40cccfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccd02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541ac490 con 0x55e40ccbee80 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541ad900 con 0x55e40ccc7a00 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541acb60 con 0x55e40ccbe4d0 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 10 client.0 _ll_register_callbacks cb 0x7f8c440074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccd02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccd02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39458/0 (socket says 172.21.15.113:39458) 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 -- 172.21.15.113:0/2792480240 learned_addr learned my addr 172.21.15.113:0/2792480240 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5359e640 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 -- 172.21.15.113:0/2792480240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 msgr2=0x55e40cccfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40cccfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 -- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 msgr2=0x55e40ccc0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 -- 172.21.15.113:0/2792480240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40ccc1230 con 0x55e40ccbee80 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5259c640 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccd02e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55e40ccc3e20 tx=0x7f8c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a4aa02189c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c36ff5640 1 -- 172.21.15.113:0/2792480240 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4802e070 con 0x55e40ccbee80 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c36ff5640 1 -- 172.21.15.113:0/2792480240 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f8c48003c40 con 0x55e40ccbee80 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.921+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c0c3400 con 0x55e40ccbee80 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c36ff5640 1 -- 172.21.15.113:0/2792480240 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8c4803a050 con 0x55e40ccbee80 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 10 client.16827 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:15.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e40cd3da90 con 0x55e40ccbee80 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c36ff5640 1 -- 172.21.15.113:0/2792480240 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8c4c0c3400 con 0x55e40ccbee80 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c36ff5640 1 -- 172.21.15.113:0/2792480240 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f8c48021670 con 0x55e40ccbee80 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 10 client.16827 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 20 client.16827 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40cd3e250 0x55e40cd40670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 4 client.16827 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e40cc30c10 con 0x55e40cd3e250 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.922+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40cd3e250 0x55e40cd40670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c36ff5640 10 client.16827 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c52d9d640 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40cd3e250 0x55e40cd40670 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab26660e1f5566cb server_cookie=a364e0a4da1fd1d7 in_seq=0 out_seq=0 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c36ff5640 1 -- 172.21.15.113:0/2792480240 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e40cc30c10 con 0x55e40cd3e250 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c36ff5640 10 client.16827 handle_command_reply: tid=0 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 client.16827 shutdown 2024-04-22T07:20:15.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 msgr2=0x55e40ccd02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccd02e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x55e40ccc3e20 tx=0x7f8c48004730 comp rx=0 tx=0).stop 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 shutdown_connections 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55e40cd3e250 0x55e40cd40670 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e40ccc7a00 0x55e40ccc0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e40ccbee80 0x55e40ccd02e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c36ff5640 0 client.16827 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 --2- 172.21.15.113:0/2792480240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e40ccbe4d0 0x55e40cccfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 >> 172.21.15.113:0/2792480240 conn(0x55e40ccb8ba0 msgr2=0x55e40ccc3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 shutdown_connections 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 1 -- 172.21.15.113:0/2792480240 wait complete. 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c5bd7b740 20 client.16827 trim_cache size 0 max 16384 2024-04-22T07:20:15.928 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:20:15.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8c3c0a3d20 msgr2=0x7f8c3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8c3c0a3d20 0x7f8c3c0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800f010 tx=0x7f8c480046c0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 msgr2=0x7f8c54198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:15.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.923+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c54198620 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02c840 tx=0x7f8c4c003c20 comp rx=0 tx=0).stop 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 shutdown_connections 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8c3c0a3d20 0x7f8c3c0a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c5410c510 0x7f8c54198620 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c54103af0 0x7f8c5419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 --2- 172.21.15.113:0/2628385531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c541031d0 0x7f8c5419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 >> 172.21.15.113:0/2628385531 conn(0x7f8c54077f50 msgr2=0x7f8c540fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 shutdown_connections 2024-04-22T07:20:15.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:15.924+0000 7f8c590a8640 1 -- 172.21.15.113:0/2628385531 wait complete. 2024-04-22T07:20:15.932 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:20:16.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 -- 172.21.15.113:0/2917481552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406b230 msgr2=0x7efcb4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 --2- 172.21.15.113:0/2917481552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406b230 0x7efcb4106b90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efca4009900 tx=0x7efca402bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:16.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 -- 172.21.15.113:0/2917481552 shutdown_connections 2024-04-22T07:20:16.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 --2- 172.21.15.113:0/2917481552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4107270 0x7efcb410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 --2- 172.21.15.113:0/2917481552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406b230 0x7efcb4106b90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 --2- 172.21.15.113:0/2917481552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb406a880 0x7efcb406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 -- 172.21.15.113:0/2917481552 >> 172.21.15.113:0/2917481552 conn(0x7efcb4077eb0 msgr2=0x7efcb40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 -- 172.21.15.113:0/2917481552 shutdown_connections 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.034+0000 7efcbb396640 1 -- 172.21.15.113:0/2917481552 wait complete. 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 Processor -- start 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 -- start start 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 0x7efcb419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4107270 0x7efcb419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb4198630 0x7efcb4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41122f0 con 0x7efcb406a880 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4112170 con 0x7efcb4198630 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcbb396640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4112470 con 0x7efcb4107270 2024-04-22T07:20:16.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 0x7efcb419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 0x7efcb419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39480/0 (socket says 172.21.15.113:39480) 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 -- 172.21.15.113:0/1113579957 learned_addr learned my addr 172.21.15.113:0/1113579957 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9933640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb4198630 0x7efcb4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb8931640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4107270 0x7efcb419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 -- 172.21.15.113:0/1113579957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4107270 msgr2=0x7efcb419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4107270 0x7efcb419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 -- 172.21.15.113:0/1113579957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb4198630 msgr2=0x7efcb4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb4198630 0x7efcb4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9132640 1 -- 172.21.15.113:0/1113579957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb4199230 con 0x7efcb406a880 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.035+0000 7efcb9933640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb4198630 0x7efcb4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.036+0000 7efcb9132640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 0x7efcb419e570 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efcb000d950 tx=0x7efcb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.036+0000 7efcaa7fc640 1 -- 172.21.15.113:0/1113579957 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0017070 con 0x7efcb406a880 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.036+0000 7efcaa7fc640 1 -- 172.21.15.113:0/1113579957 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efcb0009c30 con 0x7efcb406a880 2024-04-22T07:20:16.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.036+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb41a5310 con 0x7efcb406a880 2024-04-22T07:20:16.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.036+0000 7efcaa7fc640 1 -- 172.21.15.113:0/1113579957 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000f6b0 con 0x7efcb406a880 2024-04-22T07:20:16.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.036+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a56c0 con 0x7efcb406a880 2024-04-22T07:20:16.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.038+0000 7efcaa7fc640 1 -- 172.21.15.113:0/1113579957 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7efcb00109c0 con 0x7efcb406a880 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.038+0000 7efcbdf83740 1 Processor -- start 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.038+0000 7efcbdf83740 1 -- start start 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 0x55bba76d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcbdf83740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bba76351f0 con 0x55bba76d0e80 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcbdf83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bba75d24b0 con 0x55bba76d04d0 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcbdf83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bba76311b0 con 0x55bba76d9a00 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcaa7fc640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7efca00a3d20 0x7efca00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 0x55bba76d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 0x55bba76d9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39484/0 (socket says 172.21.15.113:39484) 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 -- 172.21.15.113:0/1574989132 learned_addr learned my addr 172.21.15.113:0/1574989132 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9933640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7efca00a3d20 0x7efca00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb8931640 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9933640 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcaa7fc640 1 -- 172.21.15.113:0/1113579957 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7efcb00d0620 con 0x7efcb406a880 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 -- 172.21.15.113:0/1574989132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 msgr2=0x55bba76dbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76dbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 -- 172.21.15.113:0/1574989132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 msgr2=0x55bba76d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 -- 172.21.15.113:0/1574989132 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba75c41b0 con 0x55bba76d0e80 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9933640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7efca00a3d20 0x7efca00a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efcac013ab0 tx=0x7efcac00ac90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.039+0000 7efcb9132640 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 0x55bba76d9430 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efcb00988a0 tx=0x7efcb0010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc942260d06e2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efc9f7fe640 1 -- 172.21.15.113:0/1574989132 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0020070 con 0x55bba76d0e80 2024-04-22T07:20:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efc9f7fe640 1 -- 172.21.15.113:0/1574989132 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efcb0009de0 con 0x55bba76d0e80 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efc9f7fe640 1 -- 172.21.15.113:0/1574989132 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0098ab0 con 0x55bba76d0e80 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 -- 172.21.15.113:0/1574989132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 msgr2=0x55bba76d9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 0x55bba76d9430 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efcb00988a0 tx=0x7efcb0010d60 comp rx=0 tx=0).stop 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 -- 172.21.15.113:0/1574989132 shutdown_connections 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76dbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d0e80 0x55bba76d9430 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1574989132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 -- 172.21.15.113:0/1574989132 >> 172.21.15.113:0/1574989132 conn(0x55bba76caba0 msgr2=0x55bba76caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 -- 172.21.15.113:0/1574989132 shutdown_connections 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.040+0000 7efcbdf83740 1 -- 172.21.15.113:0/1574989132 wait complete. 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 -- start start 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 Processor -- start 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d04d0 0x55bba76d9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55bba76dc8b0 con 0x55bba76d9a00 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bba75ee500 con 0x55bba76d0e80 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55bba76dca30 con 0x55bba76d04d0 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcbdf83740 10 client.0 _ll_register_callbacks cb 0x55bba75bcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76d6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39486/0 (socket says 172.21.15.113:39486) 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 -- 172.21.15.113:0/10530809 learned_addr learned my addr 172.21.15.113:0/10530809 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb8931640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9132640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d04d0 0x55bba76d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 -- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d04d0 msgr2=0x55bba76d9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d04d0 0x55bba76d9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 -- 172.21.15.113:0/10530809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 msgr2=0x55bba76d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb9933640 1 -- 172.21.15.113:0/10530809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba73473f0 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.041+0000 7efcb8931640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcb9132640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d04d0 0x55bba76d9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcb9933640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76d6c10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efcac010fc0 tx=0x7efcac005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373ab8808363d298 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efc9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efc9cff9640 1 -- 172.21.15.113:0/10530809 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac026070 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efc9cff9640 1 -- 172.21.15.113:0/10530809 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7efcac004d20 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb0009de0 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efc9cff9640 1 -- 172.21.15.113:0/10530809 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7efcac031050 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 10 client.16839 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bba74117d0 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efc9cff9640 1 -- 172.21.15.113:0/10530809 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efcb0009de0 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efc9cff9640 1 -- 172.21.15.113:0/10530809 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7efcac005020 con 0x55bba76d9a00 2024-04-22T07:20:16.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 10 client.16839 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 10 client.16839 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 20 client.16839 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba76e9bc0 0x55bba76ec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 4 client.16839 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.042+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bba75d4930 con 0x55bba76e9bc0 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.043+0000 7efcb9132640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba76e9bc0 0x55bba76ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.043+0000 7efcb9132640 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba76e9bc0 0x55bba76ec060 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df7e09461931379e server_cookie=6f1daf81cf65fbcd in_seq=0 out_seq=0 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.043+0000 7efc9cff9640 10 client.16839 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efc9cff9640 1 -- 172.21.15.113:0/10530809 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bba75d4930 con 0x55bba76e9bc0 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efc9cff9640 10 client.16839 handle_command_reply: tid=0 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 client.16839 shutdown 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 msgr2=0x55bba76d6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76d6c10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efcac010fc0 tx=0x7efcac005c10 comp rx=0 tx=0).stop 2024-04-22T07:20:16.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 shutdown_connections 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba76e9bc0 0x55bba76ec060 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76d6c10 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efc9cff9640 0 client.16839 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 --2- 172.21.15.113:0/10530809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d04d0 0x55bba76d9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 >> 172.21.15.113:0/10530809 conn(0x55bba76caba0 msgr2=0x55bba76cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 shutdown_connections 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 1 -- 172.21.15.113:0/10530809 wait complete. 2024-04-22T07:20:16.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.044+0000 7efcbdf83740 20 client.16839 trim_cache size 0 max 16384 2024-04-22T07:20:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 Processor -- start 2024-04-22T07:20:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 -- start start 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 0x55bba76e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76d8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41b0840 con 0x55bba76d04d0 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ae0b0 con 0x55bba76d0e80 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcbdf83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41abf20 con 0x55bba76d9a00 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.057+0000 7efcb9132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 0x55bba76e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 0x55bba76e57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39496/0 (socket says 172.21.15.113:39496) 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 -- 172.21.15.113:0/1083513236 learned_addr learned my addr 172.21.15.113:0/1083513236 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 -- 172.21.15.113:0/1083513236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 msgr2=0x55bba76d8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76d8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 -- 172.21.15.113:0/1083513236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 msgr2=0x55bba76d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 -- 172.21.15.113:0/1083513236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba76d2270 con 0x55bba76d04d0 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcb9132640 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 0x55bba76e57a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7efcb00cdca0 tx=0x7efcb0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991617e88ea14b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efc9effd640 1 -- 172.21.15.113:0/1083513236 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0085020 con 0x55bba76d04d0 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efc9effd640 1 -- 172.21.15.113:0/1083513236 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efcb00a50b0 con 0x55bba76d04d0 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efc9effd640 1 -- 172.21.15.113:0/1083513236 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0020040 con 0x55bba76d04d0 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcbdf83740 1 -- 172.21.15.113:0/1083513236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 msgr2=0x55bba76e57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.058+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 0x55bba76e57a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7efcb00cdca0 tx=0x7efcb0005f70 comp rx=0 tx=0).stop 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- 172.21.15.113:0/1083513236 shutdown_connections 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d9a00 0x55bba76d8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d0e80 0x55bba76d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1083513236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d04d0 0x55bba76e57a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- 172.21.15.113:0/1083513236 >> 172.21.15.113:0/1083513236 conn(0x55bba76caba0 msgr2=0x55bba76c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- 172.21.15.113:0/1083513236 shutdown_connections 2024-04-22T07:20:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- 172.21.15.113:0/1083513236 wait complete. 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- start start 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 Processor -- start 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76dd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d0e80 0x55bba76dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ac570 con 0x55bba76d9a00 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ad9e0 con 0x55bba76d04d0 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41acc40 con 0x55bba76d0e80 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcbdf83740 10 client.0 _ll_register_callbacks cb 0x7efcac013360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcb9933640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcb9933640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76de340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39504/0 (socket says 172.21.15.113:39504) 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.059+0000 7efcb9933640 1 -- 172.21.15.113:0/1124308868 learned_addr learned my addr 172.21.15.113:0/1124308868 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcb9933640 1 -- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d0e80 msgr2=0x55bba76dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcb9933640 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d0e80 0x55bba76dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcb9933640 1 -- 172.21.15.113:0/1124308868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 msgr2=0x55bba76dd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcb9933640 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76dd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcb9933640 1 -- 172.21.15.113:0/1124308868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba774f3a0 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcb9933640 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76de340 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7efcac011010 tx=0x7efcac005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01f39a9e38b6afd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efc9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efc9cff9640 1 -- 172.21.15.113:0/1124308868 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac026040 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efc9cff9640 1 -- 172.21.15.113:0/1124308868 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7efcac003970 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb00a50b0 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efc9cff9640 1 -- 172.21.15.113:0/1124308868 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7efcac040050 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcbdf83740 10 client.16845 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bba7750680 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efc9cff9640 1 -- 172.21.15.113:0/1124308868 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efcb00a50b0 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efc9cff9640 1 -- 172.21.15.113:0/1124308868 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7efcac003c00 con 0x55bba76d9a00 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcbdf83740 10 client.16845 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:16.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.060+0000 7efcbdf83740 10 client.16845 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 20 client.16845 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba7750dc0 0x55bba7753200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 4 client.16845 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bba7642c10 con 0x55bba7750dc0 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcb9132640 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba7750dc0 0x55bba7753200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efc9cff9640 10 client.16845 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcb9132640 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba7750dc0 0x55bba7753200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e92d1545f8eeb98d server_cookie=9dc4036ce86c31e1 in_seq=0 out_seq=0 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efc9cff9640 1 -- 172.21.15.113:0/1124308868 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bba7642c10 con 0x55bba7750dc0 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efc9cff9640 10 client.16845 handle_command_reply: tid=0 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 client.16845 shutdown 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 msgr2=0x55bba76de340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76de340 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7efcac011010 tx=0x7efcac005890 comp rx=0 tx=0).stop 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 shutdown_connections 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55bba7750dc0 0x55bba7753200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55bba76d9a00 0x55bba76de340 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bba76d0e80 0x55bba76dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efc9cff9640 0 client.16845 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 --2- 172.21.15.113:0/1124308868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bba76d04d0 0x55bba76dd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.061+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 >> 172.21.15.113:0/1124308868 conn(0x55bba76caba0 msgr2=0x55bba76c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 shutdown_connections 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbdf83740 1 -- 172.21.15.113:0/1124308868 wait complete. 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbdf83740 20 client.16845 trim_cache size 0 max 16384 2024-04-22T07:20:16.066 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:20:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7efca00a3d20 msgr2=0x7efca00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7efca00a3d20 0x7efca00a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efcac013ab0 tx=0x7efcac00ac90 comp rx=0 tx=0).stop 2024-04-22T07:20:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 msgr2=0x7efcb419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 0x7efcb419e570 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efcb000d950 tx=0x7efcb000de10 comp rx=0 tx=0).stop 2024-04-22T07:20:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 shutdown_connections 2024-04-22T07:20:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7efca00a3d20 0x7efca00a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb4198630 0x7efcb4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb4107270 0x7efcb419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 --2- 172.21.15.113:0/1113579957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb406a880 0x7efcb419e570 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 >> 172.21.15.113:0/1113579957 conn(0x7efcb4077eb0 msgr2=0x7efcb4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 shutdown_connections 2024-04-22T07:20:16.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.062+0000 7efcbb396640 1 -- 172.21.15.113:0/1113579957 wait complete. 2024-04-22T07:20:16.070 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:15 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3196547292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:20:16.071 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.171+0000 7faed706d640 1 -- 172.21.15.113:0/2562144093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0101a40 msgr2=0x7faed0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.171+0000 7faed706d640 1 --2- 172.21.15.113:0/2562144093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0101a40 0x7faed0105a80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- 172.21.15.113:0/2562144093 shutdown_connections 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 --2- 172.21.15.113:0/2562144093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed0106160 0x7faed010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 --2- 172.21.15.113:0/2562144093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0101a40 0x7faed0105a80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 --2- 172.21.15.113:0/2562144093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed0101090 0x7faed0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- 172.21.15.113:0/2562144093 >> 172.21.15.113:0/2562144093 conn(0x7faed0077f50 msgr2=0x7faed0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- 172.21.15.113:0/2562144093 shutdown_connections 2024-04-22T07:20:16.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- 172.21.15.113:0/2562144093 wait complete. 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 Processor -- start 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- start start 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed0101090 0x7faed01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 0x7faed0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed019bfa0 0x7faed019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faed01111e0 con 0x7faed0106160 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faed0111060 con 0x7faed0101090 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faed706d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faed0111360 con 0x7faed019bfa0 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faecffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 0x7faed0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faecffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 0x7faed0197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39520/0 (socket says 172.21.15.113:39520) 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.172+0000 7faecffff640 1 -- 172.21.15.113:0/4191556521 learned_addr learned my addr 172.21.15.113:0/4191556521 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faed4e09640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed0101090 0x7faed01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faed560a640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed019bfa0 0x7faed019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecffff640 1 -- 172.21.15.113:0/4191556521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed019bfa0 msgr2=0x7faed019c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecffff640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed019bfa0 0x7faed019c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecffff640 1 -- 172.21.15.113:0/4191556521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed0101090 msgr2=0x7faed01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecffff640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed0101090 0x7faed01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecffff640 1 -- 172.21.15.113:0/4191556521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed019ca90 con 0x7faed0106160 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecffff640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 0x7faed0197c20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc02cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecdffb640 1 -- 172.21.15.113:0/4191556521 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc041070 con 0x7faed0106160 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed019cd20 con 0x7faed0106160 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecdffb640 1 -- 172.21.15.113:0/4191556521 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faebc0044d0 con 0x7faed0106160 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faecdffb640 1 -- 172.21.15.113:0/4191556521 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc003d50 con 0x7faed0106160 2024-04-22T07:20:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.173+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01a4520 con 0x7faed0106160 2024-04-22T07:20:16.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faecdffb640 1 -- 172.21.15.113:0/4191556521 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7faebc044070 con 0x7faed0106160 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 Processor -- start 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 -- start start 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cf9d9281f0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf9d8c54b0 con 0x55cf9d9d0f70 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed9c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cf9d9241b0 con 0x55cf9d9c5cf0 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39526/0 (socket says 172.21.15.113:39526) 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 -- 172.21.15.113:0/1162015547 learned_addr learned my addr 172.21.15.113:0/1162015547 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faecdffb640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faec00a3d20 0x7faec00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faecffff640 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faec00a3d20 0x7faec00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 -- 172.21.15.113:0/1162015547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 msgr2=0x55cf9d9d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed4e09640 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faecdffb640 1 -- 172.21.15.113:0/4191556521 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7faebc0f2400 con 0x7faed0106160 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 -- 172.21.15.113:0/1162015547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 msgr2=0x55cf9d9d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.176+0000 7faed560a640 1 -- 172.21.15.113:0/1162015547 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9d8b71b0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed560a640 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9c57b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7faec8014940 tx=0x7faec8014e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5f3413752ede2b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faebaffd640 1 -- 172.21.15.113:0/1162015547 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8014050 con 0x55cf9d9c53d0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faebaffd640 1 -- 172.21.15.113:0/1162015547 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faec80141f0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faebaffd640 1 -- 172.21.15.113:0/1162015547 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8013660 con 0x55cf9d9c53d0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed560a640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faec00a3d20 0x7faec00a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faec8021000 tx=0x7faec8020000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 -- 172.21.15.113:0/1162015547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 msgr2=0x55cf9d9c57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9c57b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7faec8014940 tx=0x7faec8014e00 comp rx=0 tx=0).stop 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 -- 172.21.15.113:0/1162015547 shutdown_connections 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 --2- 172.21.15.113:0/1162015547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9c57b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.177+0000 7faed9c63740 1 -- 172.21.15.113:0/1162015547 >> 172.21.15.113:0/1162015547 conn(0x55cf9d9bfdb0 msgr2=0x55cf9d9c0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 -- 172.21.15.113:0/1162015547 shutdown_connections 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 -- 172.21.15.113:0/1162015547 wait complete. 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 -- start start 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 Processor -- start 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cf9d9d3d50 con 0x55cf9d9c53d0 2024-04-22T07:20:16.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf9d8e1500 con 0x55cf9d9d0f70 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cf9d9d3ed0 con 0x55cf9d9c5cf0 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed9c63740 10 client.0 _ll_register_callbacks cb 0x55cf9d8afb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed4e09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed4e09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39540/0 (socket says 172.21.15.113:39540) 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.178+0000 7faed4e09640 1 -- 172.21.15.113:0/3215503557 learned_addr learned my addr 172.21.15.113:0/3215503557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed560a640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faecffff640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed4e09640 1 -- 172.21.15.113:0/3215503557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 msgr2=0x55cf9d9cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed4e09640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed4e09640 1 -- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 msgr2=0x55cf9d9ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed4e09640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed4e09640 1 -- 172.21.15.113:0/3215503557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9d63a3f0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed560a640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faecffff640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:16.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed4e09640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9d0920 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7faec4010670 tx=0x7faec4010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94490713bbbdab6e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faeaffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faeaffff640 1 -- 172.21.15.113:0/3215503557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400ead0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faeaffff640 1 -- 172.21.15.113:0/3215503557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7faec400ec70 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec80141f0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faeaffff640 1 -- 172.21.15.113:0/3215503557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7faec4025050 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed9c63740 10 client.16857 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cf9d7047d0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.179+0000 7faeaffff640 1 -- 172.21.15.113:0/3215503557 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faec80141f0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faeaffff640 1 -- 172.21.15.113:0/3215503557 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7faec400f040 con 0x55cf9d9c53d0 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faed9c63740 10 client.16857 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faed9c63740 10 client.16857 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faed9c63740 20 client.16857 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faed9c63740 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9d9e1130 0x55cf9d9e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faed9c63740 4 client.16857 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9d8c7930 con 0x55cf9d9e1130 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faecffff640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9d9e1130 0x55cf9d9e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faecffff640 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9d9e1130 0x55cf9d9e3590 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fbf32d7e44a92d6 server_cookie=d5a1fc2224dbb373 in_seq=0 out_seq=0 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.180+0000 7faeaffff640 10 client.16857 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faeaffff640 1 -- 172.21.15.113:0/3215503557 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf9d8c7930 con 0x55cf9d9e1130 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faeaffff640 10 client.16857 handle_command_reply: tid=0 2024-04-22T07:20:16.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 client.16857 shutdown 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 msgr2=0x55cf9d9d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9d0920 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7faec4010670 tx=0x7faec4010b30 comp rx=0 tx=0).stop 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 shutdown_connections 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9d9e1130 0x55cf9d9e3590 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faeaffff640 0 client.16857 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 --2- 172.21.15.113:0/3215503557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9d0920 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 >> 172.21.15.113:0/3215503557 conn(0x55cf9d9bfdb0 msgr2=0x55cf9d9c1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 shutdown_connections 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.181+0000 7faed9c63740 1 -- 172.21.15.113:0/3215503557 wait complete. 2024-04-22T07:20:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.182+0000 7faed9c63740 20 client.16857 trim_cache size 0 max 16384 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 Processor -- start 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 -- start start 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faed01ae620 con 0x55cf9d9d0f70 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faed01abe90 con 0x55cf9d9c53d0 2024-04-22T07:20:16.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed9c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faed01a9ce0 con 0x55cf9d9c5cf0 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39548/0 (socket says 172.21.15.113:39548) 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 -- 172.21.15.113:0/1199770684 learned_addr learned my addr 172.21.15.113:0/1199770684 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed4e09640 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 -- 172.21.15.113:0/1199770684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 msgr2=0x55cf9d9d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 -- 172.21.15.113:0/1199770684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c53d0 msgr2=0x55cf9d9dcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9dcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 -- 172.21.15.113:0/1199770684 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9d9c4770 con 0x55cf9d9d0f70 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faed560a640 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0920 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7faec8015010 tx=0x7faec8012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ce6271380b0a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faeba7fc640 1 -- 172.21.15.113:0/1199770684 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8012070 con 0x55cf9d9d0f70 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faeba7fc640 1 -- 172.21.15.113:0/1199770684 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faec8013400 con 0x55cf9d9d0f70 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.195+0000 7faeba7fc640 1 -- 172.21.15.113:0/1199770684 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8013d40 con 0x55cf9d9d0f70 2024-04-22T07:20:16.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 -- 172.21.15.113:0/1199770684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 msgr2=0x55cf9d9d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0920 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7faec8015010 tx=0x7faec8012810 comp rx=0 tx=0).stop 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 -- 172.21.15.113:0/1199770684 shutdown_connections 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0920 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 --2- 172.21.15.113:0/1199770684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 -- 172.21.15.113:0/1199770684 >> 172.21.15.113:0/1199770684 conn(0x55cf9d9bfdb0 msgr2=0x55cf9d9c0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 -- 172.21.15.113:0/1199770684 shutdown_connections 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 -- 172.21.15.113:0/1199770684 wait complete. 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 -- start start 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.196+0000 7faed9c63740 1 Processor -- start 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faed01aa350 con 0x55cf9d9c53d0 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faed01ab7c0 con 0x55cf9d9c5cf0 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faed01aaa20 con 0x55cf9d9d0f70 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 10 client.0 _ll_register_callbacks cb 0x7faec400bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39562/0 (socket says 172.21.15.113:39562) 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 -- 172.21.15.113:0/1429152284 learned_addr learned my addr 172.21.15.113:0/1429152284 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed560a640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 -- 172.21.15.113:0/1429152284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9d0f70 msgr2=0x55cf9d9d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 -- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c5cf0 msgr2=0x55cf9d9d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 -- 172.21.15.113:0/1429152284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9da46970 con 0x55cf9d9c53d0 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed560a640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:16.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed4e09640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9cfbb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faec4014020 tx=0x7faec400e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61c77c29586e3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faeaffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faeaffff640 1 -- 172.21.15.113:0/1429152284 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400e8d0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faeaffff640 1 -- 172.21.15.113:0/1429152284 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7faec400ea70 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.197+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec8013400 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faeaffff640 1 -- 172.21.15.113:0/1429152284 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7faec4038050 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 10 client.16863 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cf9da47c70 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faeaffff640 1 -- 172.21.15.113:0/1429152284 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faec8013400 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faeaffff640 1 -- 172.21.15.113:0/1429152284 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7faec401a9a0 con 0x55cf9d9c53d0 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 10 client.16863 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 10 client.16863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:20:16.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 20 client.16863 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9da48470 0x55cf9da4a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 4 client.16863 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf9d935c10 con 0x55cf9da48470 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.198+0000 7faecffff640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9da48470 0x55cf9da4a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faeaffff640 10 client.16863 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faecffff640 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9da48470 0x55cf9da4a890 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=655467f57b9a5da6 server_cookie=ad05fc3a93cf9714 in_seq=0 out_seq=0 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faeaffff640 1 -- 172.21.15.113:0/1429152284 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf9d935c10 con 0x55cf9da48470 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faeaffff640 10 client.16863 handle_command_reply: tid=0 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 client.16863 shutdown 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 msgr2=0x55cf9d9cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9cfbb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faec4014020 tx=0x7faec400e7b0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 shutdown_connections 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55cf9da48470 0x55cf9da4a890 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cf9d9d0f70 0x55cf9d9d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faeaffff640 0 client.16863 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:16.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf9d9c5cf0 0x55cf9d9d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 --2- 172.21.15.113:0/1429152284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cf9d9c53d0 0x55cf9d9cfbb0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 >> 172.21.15.113:0/1429152284 conn(0x55cf9d9bfdb0 msgr2=0x55cf9d9be940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 shutdown_connections 2024-04-22T07:20:16.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 1 -- 172.21.15.113:0/1429152284 wait complete. 2024-04-22T07:20:16.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.199+0000 7faed9c63740 20 client.16863 trim_cache size 0 max 16384 2024-04-22T07:20:16.204 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faec00a3d20 msgr2=0x7faec00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faec00a3d20 0x7faec00a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faec8021000 tx=0x7faec8020000 comp rx=0 tx=0).stop 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 msgr2=0x7faed0197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 0x7faed0197c20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc02cdf0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 shutdown_connections 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faec00a3d20 0x7faec00a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed019bfa0 0x7faed019c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faed0106160 0x7faed0197c20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 --2- 172.21.15.113:0/4191556521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed0101090 0x7faed01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 >> 172.21.15.113:0/4191556521 conn(0x7faed0077f50 msgr2=0x7faed00ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:16.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 shutdown_connections 2024-04-22T07:20:16.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:16.200+0000 7faed706d640 1 -- 172.21.15.113:0/4191556521 wait complete. 2024-04-22T07:20:16.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:15 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3196547292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:20:16.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:15 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3196547292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:20:17.211 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:20:17.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:20:17.211 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:20:17.236 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:16 smithi113 ceph-mon[27933]: pgmap v170: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:17.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:16 smithi133 ceph-mon[31198]: pgmap v170: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:17.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:16 smithi154 ceph-mon[31862]: pgmap v170: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T07:20:17.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/1593059853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0107260 msgr2=0x7f2bf010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/1593059853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0107260 0x7f2bf010ee30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009ca0 tx=0x7f2be802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/1593059853 shutdown_connections 2024-04-22T07:20:17.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/1593059853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0107260 0x7f2bf010ee30 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/1593059853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0102b40 0x7f2bf0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/1593059853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0102190 0x7f2bf0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/1593059853 >> 172.21.15.113:0/1593059853 conn(0x7f2bf00793a0 msgr2=0x7f2bf00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/1593059853 shutdown_connections 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.328+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/1593059853 wait complete. 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 Processor -- start 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 -- start start 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0102190 0x7f2bf019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0107260 0x7f2bf019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 0x7f2bf0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0112300 con 0x7f2bf0198640 2024-04-22T07:20:17.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0112180 con 0x7f2bf0102190 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2bf5ba4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0112480 con 0x7f2bf0107260 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2beffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 0x7f2bf0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2beffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 0x7f2bf0198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39576/0 (socket says 172.21.15.113:39576) 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.329+0000 7f2beffff640 1 -- 172.21.15.113:0/2652617560 learned_addr learned my addr 172.21.15.113:0/2652617560 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0102190 0x7f2bf019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beeffd640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0107260 0x7f2bf019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beffff640 1 -- 172.21.15.113:0/2652617560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0107260 msgr2=0x7f2bf019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beffff640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0107260 0x7f2bf019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beffff640 1 -- 172.21.15.113:0/2652617560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0102190 msgr2=0x7f2bf019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beffff640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0102190 0x7f2bf019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beffff640 1 -- 172.21.15.113:0/2652617560 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf0199240 con 0x7f2bf0198640 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beeffd640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0107260 0x7f2bf019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2beffff640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 0x7f2bf0198ac0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2be802c700 tx=0x7f2be80324e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2becff9640 1 -- 172.21.15.113:0/2652617560 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8039040 con 0x7f2bf0198640 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2becff9640 1 -- 172.21.15.113:0/2652617560 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2be8044070 con 0x7f2bf0198640 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf01994d0 con 0x7f2bf0198640 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2becff9640 1 -- 172.21.15.113:0/2652617560 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8004280 con 0x7f2bf0198640 2024-04-22T07:20:17.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.330+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a5610 con 0x7f2bf0198640 2024-04-22T07:20:17.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.332+0000 7f2becff9640 1 -- 172.21.15.113:0/2652617560 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f2be8003970 con 0x7f2bf0198640 2024-04-22T07:20:17.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 Processor -- start 2024-04-22T07:20:17.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 -- start start 2024-04-22T07:20:17.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557c4200ef00 con 0x557c420aa8e0 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557c41fac590 con 0x557c420b3e10 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bf879a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c4200aec0 con 0x557c420ab290 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2becff9640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bd80a3d20 0x7f2bd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39584/0 (socket says 172.21.15.113:39584) 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 -- 172.21.15.113:0/3395201713 learned_addr learned my addr 172.21.15.113:0/3395201713 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beffff640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bd80a3d20 0x7f2bd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2becff9640 1 -- 172.21.15.113:0/2652617560 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2be80f2d10 con 0x7f2bf0198640 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 -- 172.21.15.113:0/3395201713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 msgr2=0x557c420b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 -- 172.21.15.113:0/3395201713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 msgr2=0x557c420b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.333+0000 7f2beeffd640 1 -- 172.21.15.113:0/3395201713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c41de97e0 con 0x557c420aa8e0 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aacc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009f90 tx=0x7f2be4009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739c468e63490f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2beffff640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bd80a3d20 0x7f2bd80a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009dc0 tx=0x7f2be80916d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2bd5ffb640 1 -- 172.21.15.113:0/3395201713 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40114c0 con 0x557c420aa8e0 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2bd5ffb640 1 -- 172.21.15.113:0/3395201713 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2be4011b30 con 0x557c420aa8e0 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2bd5ffb640 1 -- 172.21.15.113:0/3395201713 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be400fb10 con 0x557c420aa8e0 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2bf879a740 1 -- 172.21.15.113:0/3395201713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 msgr2=0x557c420aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.334+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aacc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009f90 tx=0x7f2be4009740 comp rx=0 tx=0).stop 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- 172.21.15.113:0/3395201713 shutdown_connections 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3395201713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aacc0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- 172.21.15.113:0/3395201713 >> 172.21.15.113:0/3395201713 conn(0x557c420a4fb0 msgr2=0x557c420a5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- 172.21.15.113:0/3395201713 shutdown_connections 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- 172.21.15.113:0/3395201713 wait complete. 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- start start 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 Processor -- start 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420aa8e0 0x557c420b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 0x557c420b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557c420b6d80 con 0x557c420ab290 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557c41fc8580 con 0x557c420b3e10 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c420b6f00 con 0x557c420aa8e0 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 0x557c420b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.335+0000 7f2bf879a740 10 client.0 _ll_register_callbacks cb 0x557c41fce650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:17.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 0x557c420b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39596/0 (socket says 172.21.15.113:39596) 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 -- 172.21.15.113:0/432137849 learned_addr learned my addr 172.21.15.113:0/432137849 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2beeffd640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2beeffd640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420aa8e0 0x557c420b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 -- 172.21.15.113:0/432137849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420aa8e0 msgr2=0x557c420b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420aa8e0 0x557c420b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 -- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 msgr2=0x557c420b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 -- 172.21.15.113:0/432137849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c41e49f70 con 0x557c420ab290 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2beeffd640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 0x557c420b1350 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2be00126e0 tx=0x7f2be0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ab37dfb915d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bceffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bceffd640 1 -- 172.21.15.113:0/432137849 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0012190 con 0x557c420ab290 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bceffd640 1 -- 172.21.15.113:0/432137849 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2be0004020 con 0x557c420ab290 2024-04-22T07:20:17.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be4011b30 con 0x557c420ab290 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bceffd640 1 -- 172.21.15.113:0/432137849 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2be0004220 con 0x557c420ab290 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bf879a740 10 client.16875 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.336+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557c41ce1170 con 0x557c420ab290 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bceffd640 1 -- 172.21.15.113:0/432137849 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2be4011b30 con 0x557c420ab290 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bceffd640 1 -- 172.21.15.113:0/432137849 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f2be00038c0 con 0x557c420ab290 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bf879a740 10 client.16875 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bf879a740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bf879a740 20 client.16875 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bf879a740 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c420c44a0 0x557c420c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bf879a740 4 client.16875 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c4201c920 con 0x557c420c44a0 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2beeffd640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c420c44a0 0x557c420c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2beeffd640 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c420c44a0 0x557c420c6890 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7306bbb7b202e7c6 server_cookie=4b5d442f02db167d in_seq=0 out_seq=0 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.337+0000 7f2bceffd640 10 client.16875 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bceffd640 1 -- 172.21.15.113:0/432137849 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c4201c920 con 0x557c420c44a0 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bceffd640 10 client.16875 handle_command_reply: tid=0 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 client.16875 shutdown 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 msgr2=0x557c420b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 0x557c420b1350 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2be00126e0 tx=0x7f2be0012ba0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 shutdown_connections 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c420c44a0 0x557c420c6890 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420b3e10 0x557c420b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420ab290 0x557c420b1350 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 --2- 172.21.15.113:0/432137849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420aa8e0 0x557c420b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 >> 172.21.15.113:0/432137849 conn(0x557c420a4fb0 msgr2=0x557c420a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bceffd640 0 client.16875 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 shutdown_connections 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.338+0000 7f2bf879a740 1 -- 172.21.15.113:0/432137849 wait complete. 2024-04-22T07:20:17.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.339+0000 7f2bf879a740 20 client.16875 trim_cache size 0 max 16384 2024-04-22T07:20:17.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 Processor -- start 2024-04-22T07:20:17.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 -- start start 2024-04-22T07:20:17.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420aa8e0 0x557c420bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 0x557c420b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01b0860 con 0x557c420b3e10 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ae0d0 con 0x557c420aa8e0 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2bf879a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01abf40 con 0x557c420ab290 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.356+0000 7f2beeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 0x557c420b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 0x557c420b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39600/0 (socket says 172.21.15.113:39600) 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 -- 172.21.15.113:0/1869831836 learned_addr learned my addr 172.21.15.113:0/1869831836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420aa8e0 0x557c420bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 -- 172.21.15.113:0/1869831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 msgr2=0x557c420bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 -- 172.21.15.113:0/1869831836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420aa8e0 msgr2=0x557c420bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420aa8e0 0x557c420bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 -- 172.21.15.113:0/1869831836 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c420ab030 con 0x557c420b3e10 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2beeffd640 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 0x557c420b16b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2be400f760 tx=0x7f2be400f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aaaf49690b038f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2bd57fa640 1 -- 172.21.15.113:0/1869831836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4011230 con 0x557c420b3e10 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2bd57fa640 1 -- 172.21.15.113:0/1869831836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2be4005510 con 0x557c420b3e10 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.357+0000 7f2bd57fa640 1 -- 172.21.15.113:0/1869831836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40114c0 con 0x557c420b3e10 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 -- 172.21.15.113:0/1869831836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 msgr2=0x557c420b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 0x557c420b16b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f2be400f760 tx=0x7f2be400f6f0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 -- 172.21.15.113:0/1869831836 shutdown_connections 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420b3e10 0x557c420b16b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420ab290 0x557c420bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 --2- 172.21.15.113:0/1869831836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420aa8e0 0x557c420bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 -- 172.21.15.113:0/1869831836 >> 172.21.15.113:0/1869831836 conn(0x557c420a4fb0 msgr2=0x557c420a62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 -- 172.21.15.113:0/1869831836 shutdown_connections 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 -- 172.21.15.113:0/1869831836 wait complete. 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 -- start start 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.358+0000 7f2bf879a740 1 Processor -- start 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420ab290 0x557c420cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420b3e10 0x557c420cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ac590 con 0x557c420aa8e0 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ada00 con 0x557c420ab290 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01acc60 con 0x557c420b3e10 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 10 client.0 _ll_register_callbacks cb 0x7f2be0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39602/0 (socket says 172.21.15.113:39602) 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 -- 172.21.15.113:0/3392276812 learned_addr learned my addr 172.21.15.113:0/3392276812 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420b3e10 0x557c420cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420ab290 0x557c420cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 -- 172.21.15.113:0/3392276812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420b3e10 msgr2=0x557c420cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420b3e10 0x557c420cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 -- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420ab290 msgr2=0x557c420cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420ab290 0x557c420cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 -- 172.21.15.113:0/3392276812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c420cc7f0 con 0x557c420aa8e0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420ab290 0x557c420cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2beeffd640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aea60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2be40095f0 tx=0x7f2be4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1812f74e2a09028a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bceffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bceffd640 1 -- 172.21.15.113:0/3392276812 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be400f790 con 0x557c420aa8e0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bceffd640 1 -- 172.21.15.113:0/3392276812 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f2be400f930 con 0x557c420aa8e0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.359+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c42016790 con 0x557c420aa8e0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bceffd640 1 -- 172.21.15.113:0/3392276812 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2be40114c0 con 0x557c420aa8e0 2024-04-22T07:20:17.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 10 client.16881 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557c4212eb60 con 0x557c420aa8e0 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bceffd640 1 -- 172.21.15.113:0/3392276812 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x557c42016790 con 0x557c420aa8e0 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bceffd640 1 -- 172.21.15.113:0/3392276812 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f2be402d030 con 0x557c420aa8e0 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 10 client.16881 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 20 client.16881 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c4212f040 0x557c42131460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 4 client.16881 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557c41f92c50 con 0x557c4212f040 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.360+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c4212f040 0x557c42131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.361+0000 7f2bef7fe640 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c4212f040 0x557c42131460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30226a15863c5f63 server_cookie=e3d8d1c3d00768dc in_seq=0 out_seq=0 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.361+0000 7f2bceffd640 10 client.16881 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bceffd640 1 -- 172.21.15.113:0/3392276812 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557c41f92c50 con 0x557c4212f040 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bceffd640 10 client.16881 handle_command_reply: tid=0 2024-04-22T07:20:17.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 client.16881 shutdown 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 msgr2=0x557c420aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aea60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2be40095f0 tx=0x7f2be4005f10 comp rx=0 tx=0).stop 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 shutdown_connections 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557c4212f040 0x557c42131460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c420b3e10 0x557c420cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557c420ab290 0x557c420cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bceffd640 0 client.16881 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 --2- 172.21.15.113:0/3392276812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557c420aa8e0 0x557c420aea60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 >> 172.21.15.113:0/3392276812 conn(0x557c420a4fb0 msgr2=0x557c420a6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 shutdown_connections 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 1 -- 172.21.15.113:0/3392276812 wait complete. 2024-04-22T07:20:17.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.362+0000 7f2bf879a740 20 client.16881 trim_cache size 0 max 16384 2024-04-22T07:20:17.367 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:20:17.367 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:20:17.367 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "a11868c3-3a4d-40af-83b9-23b1597e1674", 2024-04-22T07:20:17.367 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:20:17.367 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bd80a3d20 msgr2=0x7f2bd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bd80a3d20 0x7f2bd80a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009dc0 tx=0x7f2be80916d0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 msgr2=0x7f2bf0198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 0x7f2bf0198ac0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2be802c700 tx=0x7f2be80324e0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 shutdown_connections 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f2bd80a3d20 0x7f2bd80a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf0198640 0x7f2bf0198ac0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bf0107260 0x7f2bf019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 --2- 172.21.15.113:0/2652617560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0102190 0x7f2bf019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 >> 172.21.15.113:0/2652617560 conn(0x7f2bf00793a0 msgr2=0x7f2bf00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 shutdown_connections 2024-04-22T07:20:17.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.363+0000 7f2bf5ba4640 1 -- 172.21.15.113:0/2652617560 wait complete. 2024-04-22T07:20:17.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:20:17.371 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:20:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.514+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3511974942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101a30 msgr2=0x7f5a54105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.514+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3511974942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101a30 0x7f5a54105a70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009900 tx=0x7f5a4802bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:17.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3511974942 shutdown_connections 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3511974942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a54106150 0x7f5a5410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3511974942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101a30 0x7f5a54105a70 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3511974942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54101080 0x7f5a54101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3511974942 >> 172.21.15.113:0/3511974942 conn(0x7f5a54078000 msgr2=0x7f5a54078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3511974942 shutdown_connections 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3511974942 wait complete. 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 Processor -- start 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- start start 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 0x7f5a5419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54106150 0x7f5a5419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a54198630 0x7f5a54198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54112350 con 0x7f5a54101080 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541121d0 con 0x7f5a54106150 2024-04-22T07:20:17.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a5adf4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541124d0 con 0x7f5a54198630 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a58b90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 0x7f5a5419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.515+0000 7f5a58b90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 0x7f5a5419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39618/0 (socket says 172.21.15.113:39618) 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 -- 172.21.15.113:0/3319903536 learned_addr learned my addr 172.21.15.113:0/3319903536 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a59391640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a54198630 0x7f5a54198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a53fff640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54106150 0x7f5a5419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 -- 172.21.15.113:0/3319903536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a54198630 msgr2=0x7f5a54198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a54198630 0x7f5a54198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 -- 172.21.15.113:0/3319903536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54106150 msgr2=0x7f5a5419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54106150 0x7f5a5419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 -- 172.21.15.113:0/3319903536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a54199230 con 0x7f5a54101080 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a53fff640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54106150 0x7f5a5419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a58b90640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 0x7f5a5419e570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b4a0 tx=0x7f5a4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a51ffb640 1 -- 172.21.15.113:0/3319903536 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44014070 con 0x7f5a54101080 2024-04-22T07:20:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a51ffb640 1 -- 172.21.15.113:0/3319903536 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a44010ae0 con 0x7f5a54101080 2024-04-22T07:20:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a541a5310 con 0x7f5a54101080 2024-04-22T07:20:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a51ffb640 1 -- 172.21.15.113:0/3319903536 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44017ce0 con 0x7f5a54101080 2024-04-22T07:20:17.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.516+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541a5710 con 0x7f5a54101080 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.518+0000 7f5a51ffb640 1 -- 172.21.15.113:0/3319903536 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5a44017070 con 0x7f5a54101080 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 Processor -- start 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 -- start start 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a51ffb640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0bc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0bc910 0x55c47a0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 0x55c47a0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c47a0211e0 con 0x55c47a0c9e90 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c479fbe390 con 0x55c47a0bbf60 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c47a01d1a0 con 0x55c47a0bc910 2024-04-22T07:20:17.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a51ffb640 1 -- 172.21.15.113:0/3319903536 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5a440cff50 con 0x7f5a54101080 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 0x55c47a0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 0x55c47a0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39620/0 (socket says 172.21.15.113:39620) 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 -- 172.21.15.113:0/1819676535 learned_addr learned my addr 172.21.15.113:0/1819676535 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a58b90640 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a53fff640 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0bc910 0x55c47a0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 -- 172.21.15.113:0/1819676535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0bc910 msgr2=0x55c47a0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0bc910 0x55c47a0c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 -- 172.21.15.113:0/1819676535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 msgr2=0x55c47a0bc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0bc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.519+0000 7f5a59391640 1 -- 172.21.15.113:0/1819676535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c479dfd7d0 con 0x55c47a0c9e90 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a59391640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c012c80 tx=0x7f5a4c009e80 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a59391640 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 0x55c47a0cc280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c010bd0 tx=0x7f5a4c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab6b33ab68a70e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a3affd640 1 -- 172.21.15.113:0/1819676535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c024070 con 0x55c47a0c9e90 2024-04-22T07:20:17.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a3affd640 1 -- 172.21.15.113:0/1819676535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a4c00ebf0 con 0x55c47a0c9e90 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a3affd640 1 -- 172.21.15.113:0/1819676535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c005650 con 0x55c47a0c9e90 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1819676535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 msgr2=0x55c47a0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 0x55c47a0cc280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c010bd0 tx=0x7f5a4c01c000 comp rx=0 tx=0).stop 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1819676535 shutdown_connections 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.520+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0c9e90 0x55c47a0cc280 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0bc910 0x55c47a0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1819676535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0bc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1819676535 >> 172.21.15.113:0/1819676535 conn(0x55c47a0b6cb0 msgr2=0x55c47a0b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1819676535 shutdown_connections 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1819676535 wait complete. 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- start start 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 Processor -- start 2024-04-22T07:20:17.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c47a0ccbf0 con 0x55c47a0bc910 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c479fda2f0 con 0x55c47a0bbf60 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c47a0ccd70 con 0x55c47a0c9e90 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a5d9e5740 10 client.0 _ll_register_callbacks cb 0x55c479fe7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39628/0 (socket says 172.21.15.113:39628) 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.521+0000 7f5a53fff640 1 -- 172.21.15.113:0/1731575751 learned_addr learned my addr 172.21.15.113:0/1731575751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a58b90640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a59391640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a53fff640 1 -- 172.21.15.113:0/1731575751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 msgr2=0x55c47a0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a53fff640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a53fff640 1 -- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 msgr2=0x55c47a0c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a53fff640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a53fff640 1 -- 172.21.15.113:0/1731575751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c479d333f0 con 0x55c47a0bc910 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a58b90640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a53fff640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0c7390 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009900 tx=0x7f5a48005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0c7e41ca3ab1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a33fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a33fff640 1 -- 172.21.15.113:0/1731575751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48005030 con 0x55c47a0bc910 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4c00ebf0 con 0x55c47a0bc910 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a33fff640 1 -- 172.21.15.113:0/1731575751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5a480051d0 con 0x55c47a0bc910 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a33fff640 1 -- 172.21.15.113:0/1731575751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5a4804d050 con 0x55c47a0bc910 2024-04-22T07:20:17.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a5d9e5740 10 client.16893 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c479cf30f0 con 0x55c47a0bc910 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a33fff640 1 -- 172.21.15.113:0/1731575751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a4c00ebf0 con 0x55c47a0bc910 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.522+0000 7f5a33fff640 1 -- 172.21.15.113:0/1731575751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f5a480035f0 con 0x55c47a0bc910 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a5d9e5740 10 client.16893 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a5d9e5740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a5d9e5740 20 client.16893 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a0d9f10 0x55c47a0dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a5d9e5740 4 client.16893 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c47a02ec00 con 0x55c47a0d9f10 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a58b90640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a0d9f10 0x55c47a0dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a33fff640 10 client.16893 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.523+0000 7f5a58b90640 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a0d9f10 0x55c47a0dc390 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f822c57186e1eb server_cookie=d295705bd7b4f717 in_seq=0 out_seq=0 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a33fff640 1 -- 172.21.15.113:0/1731575751 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c47a02ec00 con 0x55c47a0d9f10 2024-04-22T07:20:17.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a33fff640 10 client.16893 handle_command_reply: tid=0 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 client.16893 shutdown 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 msgr2=0x55c47a0c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0c7390 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009900 tx=0x7f5a48005dc0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 shutdown_connections 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a0d9f10 0x55c47a0dc390 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a33fff640 0 client.16893 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0c7390 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/1731575751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 >> 172.21.15.113:0/1731575751 conn(0x55c47a0b6cb0 msgr2=0x55c47a0b8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 shutdown_connections 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/1731575751 wait complete. 2024-04-22T07:20:17.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.524+0000 7f5a5d9e5740 20 client.16893 trim_cache size 0 max 16384 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.541+0000 7f5a5d9e5740 1 Processor -- start 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.541+0000 7f5a5d9e5740 1 -- start start 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 0x55c47a0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bc910 0x55c47a0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541ae6c0 con 0x55c47a0bbf60 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541abf30 con 0x55c47a0bc910 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a9d80 con 0x55c47a0c9e90 2024-04-22T07:20:17.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 0x55c47a0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 0x55c47a0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39632/0 (socket says 172.21.15.113:39632) 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 -- 172.21.15.113:0/3504415228 learned_addr learned my addr 172.21.15.113:0/3504415228 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a53fff640 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bc910 0x55c47a0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 -- 172.21.15.113:0/3504415228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 msgr2=0x55c47a0d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 -- 172.21.15.113:0/3504415228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bc910 msgr2=0x55c47a0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bc910 0x55c47a0c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 -- 172.21.15.113:0/3504415228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c47a0bb670 con 0x55c47a0bbf60 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a58b90640 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 0x55c47a0c9580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5a440cd690 tx=0x7f5a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e72b72c00e5de32 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a3a7fc640 1 -- 172.21.15.113:0/3504415228 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4401e040 con 0x55c47a0bbf60 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a3a7fc640 1 -- 172.21.15.113:0/3504415228 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a4409a180 con 0x55c47a0bbf60 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.542+0000 7f5a3a7fc640 1 -- 172.21.15.113:0/3504415228 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440174f0 con 0x55c47a0bbf60 2024-04-22T07:20:17.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/3504415228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 msgr2=0x55c47a0c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 0x55c47a0c9580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5a440cd690 tx=0x7f5a440046c0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/3504415228 shutdown_connections 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a0d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bc910 0x55c47a0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/3504415228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bbf60 0x55c47a0c9580 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/3504415228 >> 172.21.15.113:0/3504415228 conn(0x55c47a0b6cb0 msgr2=0x55c47a0b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/3504415228 shutdown_connections 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/3504415228 wait complete. 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 -- start start 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.543+0000 7f5a5d9e5740 1 Processor -- start 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a13f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541aa3f0 con 0x55c47a0bc910 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541ab860 con 0x55c47a0bbf60 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541aaac0 con 0x55c47a0c9e90 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 10 client.0 _ll_register_callbacks cb 0x7f5a48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39646/0 (socket says 172.21.15.113:39646) 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 -- 172.21.15.113:0/942798488 learned_addr learned my addr 172.21.15.113:0/942798488 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a58b90640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a59391640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a13f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 -- 172.21.15.113:0/942798488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 msgr2=0x55c47a13f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a13f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 -- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 msgr2=0x55c47a0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 -- 172.21.15.113:0/942798488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47a0b6ae0 con 0x55c47a0bc910 2024-04-22T07:20:17.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a59391640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a13f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a53fff640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0ce7e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5a48000c00 tx=0x7f5a48004030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874df9eb837016e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a33fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a33fff640 1 -- 172.21.15.113:0/942798488 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480035f0 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a33fff640 1 -- 172.21.15.113:0/942798488 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5a48003790 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4409a180 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a33fff640 1 -- 172.21.15.113:0/942798488 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5a48005040 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 10 client.16899 fetch_fsmap learned FSMap version 47 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.544+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c47a140b80 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a33fff640 1 -- 172.21.15.113:0/942798488 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a4409a180 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a33fff640 1 -- 172.21.15.113:0/942798488 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 47) ==== 8362+0+0 (secure 0 0 0) 0x7f5a48004cd0 con 0x55c47a0bc910 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a5d9e5740 10 client.16899 fetch_fsmap finished waiting for FSMap version 47 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a5d9e5740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a5d9e5740 20 client.16899 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:17.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a141260 0x55c47a143680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a5d9e5740 4 client.16899 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c47a028a70 con 0x55c47a141260 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a58b90640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a141260 0x55c47a143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a33fff640 10 client.16899 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.545+0000 7f5a58b90640 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a141260 0x55c47a143680 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebd8d422216ad91 server_cookie=b9fa970d7dbe5e5c in_seq=0 out_seq=0 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a33fff640 1 -- 172.21.15.113:0/942798488 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c47a028a70 con 0x55c47a141260 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a33fff640 10 client.16899 handle_command_reply: tid=0 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 client.16899 shutdown 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 msgr2=0x55c47a0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0ce7e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5a48000c00 tx=0x7f5a48004030 comp rx=0 tx=0).stop 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 shutdown_connections 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55c47a141260 0x55c47a143680 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c47a0c9e90 0x55c47a13f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47a0bc910 0x55c47a0ce7e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 --2- 172.21.15.113:0/942798488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c47a0bbf60 0x55c47a0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 >> 172.21.15.113:0/942798488 conn(0x55c47a0b6cb0 msgr2=0x55c47a0c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a33fff640 0 client.16899 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 shutdown_connections 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 1 -- 172.21.15.113:0/942798488 wait complete. 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5d9e5740 20 client.16899 trim_cache size 0 max 16384 2024-04-22T07:20:17.551 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: "a11868c3-3a4d-40af-83b9-23b1597e1674": { 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "a11868c3-3a4d-40af-83b9-23b1597e1674", 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:20:17.552 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a3c0a3d20 msgr2=0x7f5a3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c012c80 tx=0x7f5a4c009e80 comp rx=0 tx=0).stop 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 msgr2=0x7f5a5419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.546+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 0x7f5a5419e570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b4a0 tx=0x7f5a4400b960 comp rx=0 tx=0).stop 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 shutdown_connections 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a3c0a3d20 0x7f5a3c0a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a54198630 0x7f5a54198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a54106150 0x7f5a5419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 --2- 172.21.15.113:0/3319903536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a54101080 0x7f5a5419e570 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:17.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 >> 172.21.15.113:0/3319903536 conn(0x7f5a54078000 msgr2=0x7f5a540fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:17.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 shutdown_connections 2024-04-22T07:20:17.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:17.547+0000 7f5a5adf4640 1 -- 172.21.15.113:0/3319903536 wait complete. 2024-04-22T07:20:17.555 INFO:tasks.cephfs.filesystem:scrub status for tag:a11868c3-3a4d-40af-83b9-23b1597e1674 - {'path': '/', 'tag': 'a11868c3-3a4d-40af-83b9-23b1597e1674', 'options': 'recursive,force'} 2024-04-22T07:20:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:17 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:20:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:17 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:17 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:20:18.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:17 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:18.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:17 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:20:18.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:17 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:18.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:17 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:20:18.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:17 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:17 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:20:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:17 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:17 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:20:18.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:17 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:19.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:18 smithi133 ceph-mon[31198]: pgmap v171: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:20:19.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:18 smithi154 ceph-mon[31862]: pgmap v171: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:20:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:18 smithi113 ceph-mon[27933]: pgmap v171: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:20:21.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:20 smithi133 ceph-mon[31198]: pgmap v172: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:20:21.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:20 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:20:21.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:20 smithi154 ceph-mon[31862]: pgmap v172: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:20:21.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:20 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:20:21.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:20 smithi113 ceph-mon[27933]: pgmap v172: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T07:20:21.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:20 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:20:23.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:22 smithi133 ceph-mon[31198]: pgmap v173: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T07:20:23.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:22 smithi154 ceph-mon[31862]: pgmap v173: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T07:20:23.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:22 smithi113 ceph-mon[27933]: pgmap v173: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T07:20:25.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:24 smithi133 ceph-mon[31198]: pgmap v174: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-22T07:20:25.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:24 smithi154 ceph-mon[31862]: pgmap v174: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-22T07:20:25.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:24 smithi113 ceph-mon[27933]: pgmap v174: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-22T07:20:27.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:26 smithi133 ceph-mon[31198]: pgmap v175: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:20:27.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:26 smithi154 ceph-mon[31862]: pgmap v175: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:20:27.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:26 smithi113 ceph-mon[27933]: pgmap v175: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:20:29.136 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:28 smithi113 ceph-mon[27933]: pgmap v176: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:29.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:28 smithi133 ceph-mon[31198]: pgmap v176: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:29.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:28 smithi154 ceph-mon[31862]: pgmap v176: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:31.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:30 smithi133 ceph-mon[31198]: pgmap v177: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:31.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:30 smithi154 ceph-mon[31862]: pgmap v177: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:31.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:30 smithi113 ceph-mon[27933]: pgmap v177: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:33.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:32 smithi133 ceph-mon[31198]: pgmap v178: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T07:20:33.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:32 smithi154 ceph-mon[31862]: pgmap v178: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T07:20:33.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:32 smithi113 ceph-mon[27933]: pgmap v178: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T07:20:35.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:34 smithi133 ceph-mon[31198]: pgmap v179: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:35.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:34 smithi154 ceph-mon[31862]: pgmap v179: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:35.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:34 smithi113 ceph-mon[27933]: pgmap v179: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:36 smithi133 ceph-mon[31198]: pgmap v180: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T07:20:37.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:36 smithi154 ceph-mon[31862]: pgmap v180: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T07:20:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:36 smithi113 ceph-mon[27933]: pgmap v180: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T07:20:39.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:38 smithi133 ceph-mon[31198]: pgmap v181: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:39.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:38 smithi154 ceph-mon[31862]: pgmap v181: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:39.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:38 smithi113 ceph-mon[27933]: pgmap v181: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:41.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:40 smithi133 ceph-mon[31198]: pgmap v182: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:41.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:40 smithi154 ceph-mon[31862]: pgmap v182: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:40 smithi113 ceph-mon[27933]: pgmap v182: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T07:20:43.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:42 smithi133 ceph-mon[31198]: pgmap v183: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T07:20:43.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:42 smithi154 ceph-mon[31862]: pgmap v183: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T07:20:43.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:42 smithi113 ceph-mon[27933]: pgmap v183: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T07:20:45.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:44 smithi133 ceph-mon[31198]: pgmap v184: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:45.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:44 smithi154 ceph-mon[31862]: pgmap v184: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:45.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:44 smithi113 ceph-mon[27933]: pgmap v184: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:47.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:46 smithi133 ceph-mon[31198]: pgmap v185: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:20:47.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:46 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:20:47.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:46 smithi154 ceph-mon[31862]: pgmap v185: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:20:47.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:46 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:20:47.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:46 smithi113 ceph-mon[27933]: pgmap v185: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:20:47.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:46 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:20:47.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:20:47.556 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:20:47.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2216458986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 msgr2=0x7f5aac10edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2216458986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac10edc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009930 tx=0x7f5a9402bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:47.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2216458986 shutdown_connections 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2216458986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac10edc0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2216458986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac06a5f0 0x7f5aac06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2216458986 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 0x7f5aac1083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2216458986 >> 172.21.15.113:0/2216458986 conn(0x7f5aac077e30 msgr2=0x7f5aac078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2216458986 shutdown_connections 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.673+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2216458986 wait complete. 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 Processor -- start 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 -- start start 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac06a5f0 0x7f5aac1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 0x7f5aac19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1122b0 con 0x7f5aac06afb0 2024-04-22T07:20:47.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112130 con 0x7f5aac06a5f0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab2afb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112430 con 0x7f5aac107fe0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49038/0 (socket says 172.21.15.113:49038) 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 -- 172.21.15.113:0/2012080718 learned_addr learned my addr 172.21.15.113:0/2012080718 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 0x7f5aac19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab0897640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac06a5f0 0x7f5aac1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 -- 172.21.15.113:0/2012080718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 msgr2=0x7f5aac19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 0x7f5aac19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 -- 172.21.15.113:0/2012080718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac06a5f0 msgr2=0x7f5aac1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac06a5f0 0x7f5aac1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5aabfff640 1 -- 172.21.15.113:0/2012080718 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac19d6c0 con 0x7f5aac06afb0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.674+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 0x7f5aac19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.675+0000 7f5aabfff640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac198c10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00eab0 tx=0x7f5a9c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.675+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/2012080718 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00ce40 con 0x7f5aac06afb0 2024-04-22T07:20:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.675+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/2012080718 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a9c011940 con 0x7f5aac06afb0 2024-04-22T07:20:47.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.675+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac19d9b0 con 0x7f5aac06afb0 2024-04-22T07:20:47.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.675+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/2012080718 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c010aa0 con 0x7f5aac06afb0 2024-04-22T07:20:47.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.675+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a5240 con 0x7f5aac06afb0 2024-04-22T07:20:47.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.677+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/2012080718 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5a9c011af0 con 0x7f5aac06afb0 2024-04-22T07:20:47.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.677+0000 7f5ab57c5740 1 Processor -- start 2024-04-22T07:20:47.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.677+0000 7f5ab57c5740 1 -- start start 2024-04-22T07:20:47.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.677+0000 7f5aa9ffb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a980a3da0 0x7f5a980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bdf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0be8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0beae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/2012080718 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5a9c0d0fc0 con 0x7f5aac06afb0 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5653c0b441e0 con 0x5653c0be8a80 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5653c0ae1390 con 0x5653c0bdfea0 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a980a3da0 0x7f5a980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5653c0b401a0 con 0x5653c0bdf580 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab0897640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bdf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0beae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0beae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49048/0 (socket says 172.21.15.113:49048) 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 -- 172.21.15.113:0/2110969885 learned_addr learned my addr 172.21.15.113:0/2110969885 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:47.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab0897640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bdf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:39212/0 (socket says 172.21.15.113:39212) 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5aabfff640 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0be8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 -- 172.21.15.113:0/2110969885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 msgr2=0x5653c0bdf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bdf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 -- 172.21.15.113:0/2110969885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 msgr2=0x5653c0be8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0be8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 -- 172.21.15.113:0/2110969885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c09207d0 con 0x5653c0be8a80 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a980a3da0 0x7f5a980a6260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009230 tx=0x7f5a9402c650 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5ab1098640 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0beae70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5a94035000 tx=0x7f5a94046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4f8e9deefce30c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.678+0000 7f5aa2ffd640 1 -- 172.21.15.113:0/2110969885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94005250 con 0x5653c0be8a80 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5aa2ffd640 1 -- 172.21.15.113:0/2110969885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a94050070 con 0x5653c0be8a80 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5aa2ffd640 1 -- 172.21.15.113:0/2110969885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940035f0 con 0x5653c0be8a80 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2110969885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 msgr2=0x5653c0beae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0beae70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5a94035000 tx=0x7f5a94046000 comp rx=0 tx=0).stop 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2110969885 shutdown_connections 2024-04-22T07:20:47.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0beae70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0be8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2110969885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bdf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2110969885 >> 172.21.15.113:0/2110969885 conn(0x5653c0bd9cb0 msgr2=0x5653c0bda090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2110969885 shutdown_connections 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.679+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2110969885 wait complete. 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 -- start start 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 Processor -- start 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 0x5653c0be5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0be6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5653c0beb810 con 0x5653c0bdf580 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5653c0afd2f0 con 0x5653c0be8a80 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5653c0beb990 con 0x5653c0bdfea0 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab57c5740 10 client.0 _ll_register_callbacks cb 0x5653c0b0a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:47.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab0897640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab0897640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49054/0 (socket says 172.21.15.113:49054) 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab0897640 1 -- 172.21.15.113:0/1643592527 learned_addr learned my addr 172.21.15.113:0/1643592527 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab1098640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0be6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.680+0000 7f5ab0897640 1 -- 172.21.15.113:0/1643592527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 msgr2=0x5653c0be5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab0897640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 0x5653c0be5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab0897640 1 -- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 msgr2=0x5653c0be6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab0897640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0be6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab0897640 1 -- 172.21.15.113:0/1643592527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c08563f0 con 0x5653c0bdf580 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab1098640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0be6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab0897640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be5aa0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400cdf0 tx=0x7f5aa40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0099de2ce958e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5a8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5a8bfff640 1 -- 172.21.15.113:0/1643592527 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4011970 con 0x5653c0bdf580 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5a8bfff640 1 -- 172.21.15.113:0/1643592527 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5aa4011b10 con 0x5653c0bdf580 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a94050070 con 0x5653c0bdf580 2024-04-22T07:20:47.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5a8bfff640 1 -- 172.21.15.113:0/1643592527 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5aa4011d10 con 0x5653c0bdf580 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 10 client.16911 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5653c08160f0 con 0x5653c0bdf580 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5a8bfff640 1 -- 172.21.15.113:0/1643592527 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5a94050070 con 0x5653c0bdf580 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5a8bfff640 1 -- 172.21.15.113:0/1643592527 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f5aa4018070 con 0x5653c0bdf580 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 10 client.16911 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 20 client.16911 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0bf8b70 0x5653c0bfaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 4 client.16911 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.681+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653c0b51c00 con 0x5653c0bf8b70 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.682+0000 7f5aabfff640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0bf8b70 0x5653c0bfaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.682+0000 7f5aabfff640 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0bf8b70 0x5653c0bfaff0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542bd481426b0edf server_cookie=5cf32088c7eee88b in_seq=0 out_seq=0 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.682+0000 7f5a8bfff640 10 client.16911 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5a8bfff640 1 -- 172.21.15.113:0/1643592527 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653c0b51c00 con 0x5653c0bf8b70 2024-04-22T07:20:47.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5a8bfff640 10 client.16911 handle_command_reply: tid=0 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 client.16911 shutdown 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 msgr2=0x5653c0be5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be5aa0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400cdf0 tx=0x7f5aa40039c0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 shutdown_connections 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0bf8b70 0x5653c0bfaff0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0be6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 0x5653c0be5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5a8bfff640 0 client.16911 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/1643592527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be5aa0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 >> 172.21.15.113:0/1643592527 conn(0x5653c0bd9cb0 msgr2=0x5653c0bdba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 shutdown_connections 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 1 -- 172.21.15.113:0/1643592527 wait complete. 2024-04-22T07:20:47.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.683+0000 7f5ab57c5740 20 client.16911 trim_cache size 0 max 16384 2024-04-22T07:20:47.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.700+0000 7f5ab57c5740 1 Processor -- start 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.700+0000 7f5ab57c5740 1 -- start start 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 0x5653c0be49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0bf4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1b06c0 con 0x5653c0bdf580 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1adf30 con 0x5653c0be8a80 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1abd80 con 0x5653c0bdfea0 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49064/0 (socket says 172.21.15.113:49064) 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 -- 172.21.15.113:0/2358612774 learned_addr learned my addr 172.21.15.113:0/2358612774 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 -- 172.21.15.113:0/2358612774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 msgr2=0x5653c0be49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 0x5653c0be49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 -- 172.21.15.113:0/2358612774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 msgr2=0x5653c0bf4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0bf4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 -- 172.21.15.113:0/2358612774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c0be3ff0 con 0x5653c0bdf580 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5ab0897640 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be81d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4019620 tx=0x7f5aa40041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913c3f37cfc64eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5aa27fc640 1 -- 172.21.15.113:0/2358612774 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa402c020 con 0x5653c0bdf580 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5aa27fc640 1 -- 172.21.15.113:0/2358612774 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aa4019c60 con 0x5653c0bdf580 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.701+0000 7f5aa27fc640 1 -- 172.21.15.113:0/2358612774 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4004660 con 0x5653c0bdf580 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2358612774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 msgr2=0x5653c0be81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be81d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4019620 tx=0x7f5aa40041d0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2358612774 shutdown_connections 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0be8a80 0x5653c0bf4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdfea0 0x5653c0be49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/2358612774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0bdf580 0x5653c0be81d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2358612774 >> 172.21.15.113:0/2358612774 conn(0x5653c0bd9cb0 msgr2=0x5653c0bd8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2358612774 shutdown_connections 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 -- 172.21.15.113:0/2358612774 wait complete. 2024-04-22T07:20:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 -- start start 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.702+0000 7f5ab57c5740 1 Processor -- start 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bf0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0bedc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0bee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1ac3f0 con 0x5653c0be8a80 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1ad860 con 0x5653c0bdfea0 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1acac0 con 0x5653c0bdf580 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab57c5740 10 client.0 _ll_register_callbacks cb 0x7f5aa4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0bee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0bee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49080/0 (socket says 172.21.15.113:49080) 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 -- 172.21.15.113:0/3264992152 learned_addr learned my addr 172.21.15.113:0/3264992152 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5aabfff640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0bedc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab0897640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bf0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 -- 172.21.15.113:0/3264992152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 msgr2=0x5653c0bf0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bf0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 -- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 msgr2=0x5653c0bedc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0bedc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 -- 172.21.15.113:0/3264992152 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c0bee7e0 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab0897640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bf0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5ab1098640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0bee170 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5a94036040 tx=0x7f5a94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda5179bccd17d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5a8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.703+0000 7f5a8bfff640 1 -- 172.21.15.113:0/3264992152 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94004ad0 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5a8bfff640 1 -- 172.21.15.113:0/3264992152 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f5a94004c70 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa4019c60 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5a8bfff640 1 -- 172.21.15.113:0/3264992152 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5a94064050 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 10 client.16917 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5653c0c5ffe0 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5a8bfff640 1 -- 172.21.15.113:0/3264992152 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aa4019c60 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5a8bfff640 1 -- 172.21.15.113:0/3264992152 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f5a94056460 con 0x5653c0be8a80 2024-04-22T07:20:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 10 client.16917 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 20 client.16917 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0c603e0 0x5653c0c627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 4 client.16917 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653c0b4ba70 con 0x5653c0c603e0 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.704+0000 7f5ab0897640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0c603e0 0x5653c0c627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5a8bfff640 10 client.16917 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab0897640 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0c603e0 0x5653c0c627d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3ab1e13c01a5a server_cookie=d8cb357ea722efd in_seq=0 out_seq=0 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5a8bfff640 1 -- 172.21.15.113:0/3264992152 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5653c0b4ba70 con 0x5653c0c603e0 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5a8bfff640 10 client.16917 handle_command_reply: tid=0 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 client.16917 shutdown 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 msgr2=0x5653c0bee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0bee170 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5a94036040 tx=0x7f5a94005840 comp rx=0 tx=0).stop 2024-04-22T07:20:47.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 shutdown_connections 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5653c0c603e0 0x5653c0c627d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5653c0be8a80 0x5653c0bee170 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5a8bfff640 0 client.16917 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653c0bdfea0 0x5653c0bedc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 --2- 172.21.15.113:0/3264992152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653c0bdf580 0x5653c0bf0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 >> 172.21.15.113:0/3264992152 conn(0x5653c0bd9cb0 msgr2=0x5653c0bd84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 shutdown_connections 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 1 -- 172.21.15.113:0/3264992152 wait complete. 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.705+0000 7f5ab57c5740 20 client.16917 trim_cache size 0 max 16384 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:20:47.709 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:20:47.710 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:20:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a980a3da0 msgr2=0x7f5a980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a980a3da0 0x7f5a980a6260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009230 tx=0x7f5a9402c650 comp rx=0 tx=0).stop 2024-04-22T07:20:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 msgr2=0x7f5aac198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac198c10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00eab0 tx=0x7f5a9c00ef70 comp rx=0 tx=0).stop 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 shutdown_connections 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5a980a3da0 0x7f5a980a6260 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5aac107fe0 0x7f5aac19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aac06afb0 0x7f5aac198c10 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 --2- 172.21.15.113:0/2012080718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac06a5f0 0x7f5aac1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 >> 172.21.15.113:0/2012080718 conn(0x7f5aac077e30 msgr2=0x7f5aac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 shutdown_connections 2024-04-22T07:20:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.706+0000 7f5ab2afb640 1 -- 172.21.15.113:0/2012080718 wait complete. 2024-04-22T07:20:47.715 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:20:47.715 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:20:47.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 -- 172.21.15.113:0/3453958222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 msgr2=0x7fd0bc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 --2- 172.21.15.113:0/3453958222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc106a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009900 tx=0x7fd0b002bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:47.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 -- 172.21.15.113:0/3453958222 shutdown_connections 2024-04-22T07:20:47.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 --2- 172.21.15.113:0/3453958222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100fc0 0x7fd0bc10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 --2- 172.21.15.113:0/3453958222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0bc100600 0x7fd0bc100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 --2- 172.21.15.113:0/3453958222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc106a80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 -- 172.21.15.113:0/3453958222 >> 172.21.15.113:0/3453958222 conn(0x7fd0bc077480 msgr2=0x7fd0bc0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 -- 172.21.15.113:0/3453958222 shutdown_connections 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 -- 172.21.15.113:0/3453958222 wait complete. 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 Processor -- start 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.815+0000 7fd0c4611640 1 -- start start 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100600 0x7fd0bc197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0bc100fc0 0x7fd0bc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc111250 con 0x7fd0bc1066a0 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1110d0 con 0x7fd0bc100fc0 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1113d0 con 0x7fd0bc100600 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc19bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49102/0 (socket says 172.21.15.113:49102) 2024-04-22T07:20:47.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 -- 172.21.15.113:0/199305699 learned_addr learned my addr 172.21.15.113:0/199305699 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c23ad640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100600 0x7fd0bc197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 -- 172.21.15.113:0/199305699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100600 msgr2=0x7fd0bc197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100600 0x7fd0bc197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 -- 172.21.15.113:0/199305699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0bc100fc0 msgr2=0x7fd0bc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c1bac640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0bc100fc0 0x7fd0bc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0bc100fc0 0x7fd0bc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 -- 172.21.15.113:0/199305699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc19c660 con 0x7fd0bc1066a0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c23ad640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100600 0x7fd0bc197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c2bae640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc19bf30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400cce0 tx=0x7fd0b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400f450 con 0x7fd0bc1066a0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc19c950 con 0x7fd0bc1066a0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd0b400fac0 con 0x7fd0bc1066a0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4018be0 con 0x7fd0bc1066a0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.816+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc1a41e0 con 0x7fd0bc1066a0 2024-04-22T07:20:47.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.817+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0bc069230 con 0x7fd0bc1066a0 2024-04-22T07:20:47.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.818+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd0b400fc70 con 0x7fd0bc1066a0 2024-04-22T07:20:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.819+0000 7fd0af7fe640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd0a80a3d20 0x7fd0a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.819+0000 7fd0c23ad640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd0a80a3d20 0x7fd0a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.819+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fd0b40d10b0 con 0x7fd0bc1066a0 2024-04-22T07:20:47.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.819+0000 7fd0c23ad640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd0a80a3d20 0x7fd0a80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0036040 tx=0x7fd0b0000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:47.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:47.823+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0b4014030 con 0x7fd0bc1066a0 2024-04-22T07:20:48.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.041+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0bc1a4410 con 0x7fd0bc1066a0 2024-04-22T07:20:48.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.041+0000 7fd0af7fe640 1 -- 172.21.15.113:0/199305699 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11892 (secure 0 0 0) 0x7fd0b409d5d0 con 0x7fd0bc1066a0 2024-04-22T07:20:48.039 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:20:48.039 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:20:45.564040+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:20:48.040 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 48 2024-04-22T07:20:48.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd0a80a3d20 msgr2=0x7fd0a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd0a80a3d20 0x7fd0a80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0036040 tx=0x7fd0b0000bc0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 msgr2=0x7fd0bc19bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc19bf30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400cce0 tx=0x7fd0b400a620 comp rx=0 tx=0).stop 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 shutdown_connections 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd0a80a3d20 0x7fd0a80a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0bc1066a0 0x7fd0bc19bf30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0bc100fc0 0x7fd0bc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 --2- 172.21.15.113:0/199305699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0bc100600 0x7fd0bc197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 >> 172.21.15.113:0/199305699 conn(0x7fd0bc077480 msgr2=0x7fd0bc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 shutdown_connections 2024-04-22T07:20:48.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.043+0000 7fd0c4611640 1 -- 172.21.15.113:0/199305699 wait complete. 2024-04-22T07:20:48.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:20:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 -- 172.21.15.113:0/2105451288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006b1f0 msgr2=0x7fb7e0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 --2- 172.21.15.113:0/2105451288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006b1f0 0x7fb7e0106b90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009900 tx=0x7fb7d402bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 -- 172.21.15.113:0/2105451288 shutdown_connections 2024-04-22T07:20:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 --2- 172.21.15.113:0/2105451288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e0107270 0x7fb7e010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 --2- 172.21.15.113:0/2105451288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006b1f0 0x7fb7e0106b90 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 --2- 172.21.15.113:0/2105451288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e006a840 0x7fb7e006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 -- 172.21.15.113:0/2105451288 >> 172.21.15.113:0/2105451288 conn(0x7fb7e0077eb0 msgr2=0x7fb7e00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 -- 172.21.15.113:0/2105451288 shutdown_connections 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.154+0000 7fb7e647a640 1 -- 172.21.15.113:0/2105451288 wait complete. 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 Processor -- start 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 -- start start 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 0x7fb7e019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e0107270 0x7fb7e019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e0198610 0x7fb7e0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01122d0 con 0x7fb7e006a840 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0112150 con 0x7fb7e0107270 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e647a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0112450 con 0x7fb7e0198610 2024-04-22T07:20:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 0x7fb7e019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 0x7fb7e019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49114/0 (socket says 172.21.15.113:49114) 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7dffff640 1 -- 172.21.15.113:0/1172419652 learned_addr learned my addr 172.21.15.113:0/1172419652 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.155+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e0198610 0x7fb7e0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e0107270 0x7fb7e019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dffff640 1 -- 172.21.15.113:0/1172419652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e0198610 msgr2=0x7fb7e0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e0198610 0x7fb7e0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dffff640 1 -- 172.21.15.113:0/1172419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e0107270 msgr2=0x7fb7e019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e0107270 0x7fb7e019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dffff640 1 -- 172.21.15.113:0/1172419652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e0199130 con 0x7fb7e006a840 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 0x7fb7e019e460 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000d950 tx=0x7fb7d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dd7fa640 1 -- 172.21.15.113:0/1172419652 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0017070 con 0x7fb7e006a840 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dd7fa640 1 -- 172.21.15.113:0/1172419652 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb7d0009c30 con 0x7fb7e006a840 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e0199420 con 0x7fb7e006a840 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7dd7fa640 1 -- 172.21.15.113:0/1172419652 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000f6b0 con 0x7fb7e006a840 2024-04-22T07:20:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.156+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e01a5670 con 0x7fb7e006a840 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.158+0000 7fb7dd7fa640 1 -- 172.21.15.113:0/1172419652 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb7d00109c0 con 0x7fb7e006a840 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.158+0000 7fb7e906a740 1 Processor -- start 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 -- start start 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x562310522430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x562310524e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56231047e1f0 con 0x5623105194d0 2024-04-22T07:20:48.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56231041b4b0 con 0x562310519e80 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e906a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56231047a1b0 con 0x562310522a00 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7dd7fa640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb7c80a3d20 0x7fb7c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105198b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49120/0 (socket says 172.21.15.113:49120) 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 -- 172.21.15.113:0/2555827370 learned_addr learned my addr 172.21.15.113:0/2555827370 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x562310524e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb7c80a3d20 0x7fb7c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7dffff640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x562310522430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7dd7fa640 1 -- 172.21.15.113:0/1172419652 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb7d00d05c0 con 0x7fb7e006a840 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 -- 172.21.15.113:0/2555827370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 msgr2=0x562310524e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x562310524e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 -- 172.21.15.113:0/2555827370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 msgr2=0x562310522430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x562310522430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 -- 172.21.15.113:0/2555827370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231040d1b0 con 0x5623105194d0 2024-04-22T07:20:48.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7dffff640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x562310522430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x562310524e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.159+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105198b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8012d40 tx=0x7fb7d8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff52131ecdb008b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb7c80a3d20 0x7fb7c80a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8010c40 tx=0x7fb7d801d000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7c67fc640 1 -- 172.21.15.113:0/2555827370 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80128b0 con 0x5623105194d0 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7c67fc640 1 -- 172.21.15.113:0/2555827370 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb7d8009d40 con 0x5623105194d0 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7c67fc640 1 -- 172.21.15.113:0/2555827370 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8005390 con 0x5623105194d0 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 -- 172.21.15.113:0/2555827370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 msgr2=0x5623105198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105198b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8012d40 tx=0x7fb7d8010740 comp rx=0 tx=0).stop 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 -- 172.21.15.113:0/2555827370 shutdown_connections 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x562310524e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x562310522430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2555827370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105198b0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 -- 172.21.15.113:0/2555827370 >> 172.21.15.113:0/2555827370 conn(0x562310513ba0 msgr2=0x562310513f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 -- 172.21.15.113:0/2555827370 shutdown_connections 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.160+0000 7fb7e906a740 1 -- 172.21.15.113:0/2555827370 wait complete. 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 -- start start 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 Processor -- start 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x56231051f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x56231051fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562310525910 con 0x5623105194d0 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562310437500 con 0x562310519e80 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562310525a90 con 0x562310522a00 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e906a740 10 client.0 _ll_register_callbacks cb 0x562310405b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105222d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49124/0 (socket says 172.21.15.113:49124) 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7dffff640 1 -- 172.21.15.113:0/2350414730 learned_addr learned my addr 172.21.15.113:0/2350414730 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x56231051fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.161+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x56231051f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7dffff640 1 -- 172.21.15.113:0/2350414730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 msgr2=0x56231051fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7dffff640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x56231051fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7dffff640 1 -- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 msgr2=0x56231051f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7dffff640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x56231051f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7dffff640 1 -- 172.21.15.113:0/2350414730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623101903f0 con 0x5623105194d0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7dffff640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105222d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb7d009c370 tx=0x7fb7d0010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3988b49395ec6b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/2350414730 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0020040 con 0x5623105194d0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/2350414730 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb7d009b830 con 0x5623105194d0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d8009d40 con 0x5623105194d0 2024-04-22T07:20:48.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/2350414730 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb7d00d0420 con 0x5623105194d0 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 10 client.16935 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56231025a7d0 con 0x5623105194d0 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/2350414730 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb7d8009d40 con 0x5623105194d0 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/2350414730 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fb7d0010460 con 0x5623105194d0 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 10 client.16935 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 20 client.16935 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310532a00 0x562310534e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 4 client.16935 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.162+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231041d930 con 0x562310532a00 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.163+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310532a00 0x562310534e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.163+0000 7fb7bf7fe640 10 client.16935 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.163+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310532a00 0x562310534e30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d7780aa6b6ae4a server_cookie=7343f3f8177e2c2a in_seq=0 out_seq=0 2024-04-22T07:20:48.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/2350414730 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56231041d930 con 0x562310532a00 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7bf7fe640 10 client.16935 handle_command_reply: tid=0 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 client.16935 shutdown 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 msgr2=0x5623105222d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105222d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb7d009c370 tx=0x7fb7d0010d60 comp rx=0 tx=0).stop 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 shutdown_connections 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310532a00 0x562310534e30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x56231051fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7bf7fe640 0 client.16935 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562310519e80 0x56231051f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 --2- 172.21.15.113:0/2350414730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623105194d0 0x5623105222d0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 >> 172.21.15.113:0/2350414730 conn(0x562310513ba0 msgr2=0x5623105158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 shutdown_connections 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 1 -- 172.21.15.113:0/2350414730 wait complete. 2024-04-22T07:20:48.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.164+0000 7fb7e906a740 20 client.16935 trim_cache size 0 max 16384 2024-04-22T07:20:48.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 Processor -- start 2024-04-22T07:20:48.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 -- start start 2024-04-22T07:20:48.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x56231052e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310519e80 0x562310521d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 0x5623105222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01b07f0 con 0x562310522a00 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ae060 con 0x5623105194d0 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e906a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01abeb0 con 0x562310519e80 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e4a17640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 0x5623105222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e4a17640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 0x5623105222d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49126/0 (socket says 172.21.15.113:49126) 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e4a17640 1 -- 172.21.15.113:0/1712723026 learned_addr learned my addr 172.21.15.113:0/1712723026 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310519e80 0x562310521d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x56231052e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.177+0000 7fb7e4a17640 1 -- 172.21.15.113:0/1712723026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310519e80 msgr2=0x562310521d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310519e80 0x562310521d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e4a17640 1 -- 172.21.15.113:0/1712723026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 msgr2=0x56231052e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x56231052e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e4a17640 1 -- 172.21.15.113:0/1712723026 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231050f470 con 0x562310522a00 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310519e80 0x562310521d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x56231052e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e4a17640 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 0x5623105222d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8012f30 tx=0x7fb7d8009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8581ba2dfcc10e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7c5ffb640 1 -- 172.21.15.113:0/1712723026 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d801b040 con 0x562310522a00 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7c5ffb640 1 -- 172.21.15.113:0/1712723026 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb7d80096a0 con 0x562310522a00 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7c5ffb640 1 -- 172.21.15.113:0/1712723026 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8005390 con 0x562310522a00 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 -- 172.21.15.113:0/1712723026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 msgr2=0x5623105222d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 0x5623105222d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8012f30 tx=0x7fb7d8009440 comp rx=0 tx=0).stop 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 -- 172.21.15.113:0/1712723026 shutdown_connections 2024-04-22T07:20:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310522a00 0x5623105222d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310519e80 0x562310521d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1712723026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x56231052e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 -- 172.21.15.113:0/1712723026 >> 172.21.15.113:0/1712723026 conn(0x562310513ba0 msgr2=0x562310512c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 -- 172.21.15.113:0/1712723026 shutdown_connections 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.178+0000 7fb7e906a740 1 -- 172.21.15.113:0/1712723026 wait complete. 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 -- start start 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 Processor -- start 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x562310526840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 0x562310526d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x5623105272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ac520 con 0x562310519e80 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01ad990 con 0x5623105194d0 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e01acbf0 con 0x562310522a00 2024-04-22T07:20:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7e906a740 10 client.0 _ll_register_callbacks cb 0x7fb7d009c280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 0x562310526d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 0x562310526d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49130/0 (socket says 172.21.15.113:49130) 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 -- 172.21.15.113:0/1341675429 learned_addr learned my addr 172.21.15.113:0/1341675429 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x562310526840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 -- 172.21.15.113:0/1341675429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 msgr2=0x5623105272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x5623105272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 -- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 msgr2=0x562310526840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x562310526840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7df7fe640 1 -- 172.21.15.113:0/1341675429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623105980f0 con 0x562310519e80 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.179+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x562310526840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7df7fe640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 0x562310526d80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009900 tx=0x7fb7d402cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a70e9af2c6c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/1341675429 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d404d060 con 0x562310519e80 2024-04-22T07:20:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/1341675429 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb7d4004040 con 0x562310519e80 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d80096a0 con 0x562310519e80 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/1341675429 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb7d4004240 con 0x562310519e80 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 10 client.16941 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562310599370 con 0x562310519e80 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/1341675429 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb7d80096a0 con 0x562310519e80 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/1341675429 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fb7d4004930 con 0x562310519e80 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 10 client.16941 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 20 client.16941 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310599b00 0x56231059bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 4 client.16941 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.180+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56231048bc10 con 0x562310599b00 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310599b00 0x56231059bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7bf7fe640 10 client.16941 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7dffff640 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310599b00 0x56231059bf20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd56e6ecb37182b server_cookie=4baffb1dee6044e8 in_seq=0 out_seq=0 2024-04-22T07:20:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7bf7fe640 1 -- 172.21.15.113:0/1341675429 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56231048bc10 con 0x562310599b00 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7bf7fe640 10 client.16941 handle_command_reply: tid=0 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 client.16941 shutdown 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 msgr2=0x562310526d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 0x562310526d80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009900 tx=0x7fb7d402cb50 comp rx=0 tx=0).stop 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 shutdown_connections 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x562310599b00 0x56231059bf20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562310522a00 0x5623105272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562310519e80 0x562310526d80 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7bf7fe640 0 client.16941 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 --2- 172.21.15.113:0/1341675429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623105194d0 0x562310526840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 >> 172.21.15.113:0/1341675429 conn(0x562310513ba0 msgr2=0x562310514f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 shutdown_connections 2024-04-22T07:20:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 1 -- 172.21.15.113:0/1341675429 wait complete. 2024-04-22T07:20:48.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.181+0000 7fb7e906a740 20 client.16941 trim_cache size 0 max 16384 2024-04-22T07:20:48.187 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb7c80a3d20 msgr2=0x7fb7c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb7c80a3d20 0x7fb7c80a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8010c40 tx=0x7fb7d801d000 comp rx=0 tx=0).stop 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 msgr2=0x7fb7e019e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 0x7fb7e019e460 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000d950 tx=0x7fb7d000de10 comp rx=0 tx=0).stop 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 shutdown_connections 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb7c80a3d20 0x7fb7c80a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e0198610 0x7fb7e0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e0107270 0x7fb7e019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 --2- 172.21.15.113:0/1172419652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7e006a840 0x7fb7e019e460 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 >> 172.21.15.113:0/1172419652 conn(0x7fb7e0077eb0 msgr2=0x7fb7e0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 shutdown_connections 2024-04-22T07:20:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.182+0000 7fb7e647a640 1 -- 172.21.15.113:0/1172419652 wait complete. 2024-04-22T07:20:48.190 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:20:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 -- 172.21.15.113:0/781921468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 msgr2=0x7f63dc102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 --2- 172.21.15.113:0/781921468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc102570 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009900 tx=0x7f63cc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 -- 172.21.15.113:0/781921468 shutdown_connections 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 --2- 172.21.15.113:0/781921468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 0x7f63dc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 --2- 172.21.15.113:0/781921468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63dc102b40 0x7f63dc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 --2- 172.21.15.113:0/781921468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc102570 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 -- 172.21.15.113:0/781921468 >> 172.21.15.113:0/781921468 conn(0x7f63dc0793a0 msgr2=0x7f63dc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 -- 172.21.15.113:0/781921468 shutdown_connections 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.293+0000 7f63e25d5640 1 -- 172.21.15.113:0/781921468 wait complete. 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 Processor -- start 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 -- start start 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 0x7f63dc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63dc198680 0x7f63dc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc112360 con 0x7f63dc102190 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1121e0 con 0x7f63dc107260 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e25d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1124e0 con 0x7f63dc198680 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49150/0 (socket says 172.21.15.113:49150) 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 -- 172.21.15.113:0/2680366093 learned_addr learned my addr 172.21.15.113:0/2680366093 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63db7fe640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 0x7f63dc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63e0b72640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63dc198680 0x7f63dc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 -- 172.21.15.113:0/2680366093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63dc198680 msgr2=0x7f63dc198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63dc198680 0x7f63dc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 -- 172.21.15.113:0/2680366093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 msgr2=0x7f63dc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 0x7f63dc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63dbfff640 1 -- 172.21.15.113:0/2680366093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc199280 con 0x7f63dc102190 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.294+0000 7f63db7fe640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 0x7f63dc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.295+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc19e5c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f63cc005940 tx=0x7f63cc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.295+0000 7f63d97fa640 1 -- 172.21.15.113:0/2680366093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc041070 con 0x7f63dc102190 2024-04-22T07:20:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.295+0000 7f63d97fa640 1 -- 172.21.15.113:0/2680366093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f63cc0052f0 con 0x7f63dc102190 2024-04-22T07:20:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.295+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc199510 con 0x7f63dc102190 2024-04-22T07:20:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.295+0000 7f63d97fa640 1 -- 172.21.15.113:0/2680366093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc034440 con 0x7f63dc102190 2024-04-22T07:20:48.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.295+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc1a56b0 con 0x7f63dc102190 2024-04-22T07:20:48.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63d97fa640 1 -- 172.21.15.113:0/2680366093 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f63cc044070 con 0x7f63dc102190 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 Processor -- start 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 -- start start 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f64d0 0x5586147f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147ffa00 0x558614801e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55861475b1f0 con 0x5586147f6e80 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586146f84b0 con 0x5586147ffa00 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.297+0000 7f63e51c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5586147571b0 con 0x5586147f64d0 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147ff430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49162/0 (socket says 172.21.15.113:49162) 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 -- 172.21.15.113:0/1859234043 learned_addr learned my addr 172.21.15.113:0/1859234043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63d97fa640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f63c40a3d20 0x7f63c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63db7fe640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147ffa00 0x558614801e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63e0b72640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f64d0 0x5586147f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63e0b72640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f63c40a3d20 0x7f63c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63d97fa640 1 -- 172.21.15.113:0/2680366093 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f63cc0f25b0 con 0x7f63dc102190 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 -- 172.21.15.113:0/1859234043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f64d0 msgr2=0x5586147f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f64d0 0x5586147f68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 -- 172.21.15.113:0/1859234043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147ffa00 msgr2=0x558614801e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147ffa00 0x558614801e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 -- 172.21.15.113:0/1859234043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586146ea1b0 con 0x5586147f6e80 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63db7fe640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147ffa00 0x558614801e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63e0b72640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f64d0 0x5586147f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63dbfff640 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147ff430 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0efba0 tx=0x7f63cc0ba850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bef2b741688206 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63e0b72640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f63c40a3d20 0x7f63c40a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f63d400afd0 tx=0x7f63d4005e90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63c27fc640 1 -- 172.21.15.113:0/1859234043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc04b030 con 0x5586147f6e80 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63c27fc640 1 -- 172.21.15.113:0/1859234043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f63cc0beb80 con 0x5586147f6e80 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.298+0000 7f63c27fc640 1 -- 172.21.15.113:0/1859234043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0c2830 con 0x5586147f6e80 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 -- 172.21.15.113:0/1859234043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 msgr2=0x5586147ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147ff430 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0efba0 tx=0x7f63cc0ba850 comp rx=0 tx=0).stop 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 -- 172.21.15.113:0/1859234043 shutdown_connections 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147ffa00 0x558614801e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147ff430 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1859234043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f64d0 0x5586147f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 -- 172.21.15.113:0/1859234043 >> 172.21.15.113:0/1859234043 conn(0x5586147f0ba0 msgr2=0x5586147f0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 -- 172.21.15.113:0/1859234043 shutdown_connections 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 -- 172.21.15.113:0/1859234043 wait complete. 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.299+0000 7f63e51c9740 1 -- start start 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 Processor -- start 2024-04-22T07:20:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 0x5586147ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f6e80 0x5586147fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558614802730 con 0x5586147f64d0 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558614714500 con 0x5586147f6e80 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5586148028b0 con 0x5586147ffa00 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 0x5586147ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 10 client.0 _ll_register_callbacks cb 0x5586146e2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 0x5586147ff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49176/0 (socket says 172.21.15.113:49176) 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 -- 172.21.15.113:0/1209800656 learned_addr learned my addr 172.21.15.113:0/1209800656 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63db7fe640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f6e80 0x5586147fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e0b72640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 -- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 msgr2=0x5586147fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 -- 172.21.15.113:0/1209800656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f6e80 msgr2=0x5586147fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f6e80 0x5586147fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 -- 172.21.15.113:0/1209800656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55861446d3f0 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e0b72640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63dbfff640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 0x5586147ff2e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f63cc005ec0 tx=0x7f63cc0c7270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e55c4ba689a743 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63bb7fe640 1 -- 172.21.15.113:0/1209800656 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc04b030 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63bb7fe640 1 -- 172.21.15.113:0/1209800656 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f63cc0035f0 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.300+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63cc0beb80 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63bb7fe640 1 -- 172.21.15.113:0/1209800656 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f63cc0bdde0 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 10 client.16953 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5586145377d0 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63bb7fe640 1 -- 172.21.15.113:0/1209800656 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f63cc0beb80 con 0x5586147f64d0 2024-04-22T07:20:48.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63bb7fe640 1 -- 172.21.15.113:0/1209800656 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f63cc0c6c60 con 0x5586147f64d0 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 10 client.16953 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 20 client.16953 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614811b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 4 client.16953 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586146fa930 con 0x55861480f710 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.301+0000 7f63db7fe640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614811b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.302+0000 7f63bb7fe640 10 client.16953 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.302+0000 7f63db7fe640 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614811b00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d4b82599b892250 server_cookie=ca176632e4130387 in_seq=0 out_seq=0 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.302+0000 7f63bb7fe640 1 -- 172.21.15.113:0/1209800656 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586146fa930 con 0x55861480f710 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.302+0000 7f63bb7fe640 10 client.16953 handle_command_reply: tid=0 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.302+0000 7f63e51c9740 1 client.16953 shutdown 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 msgr2=0x5586147ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 0x5586147ff2e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f63cc005ec0 tx=0x7f63cc0c7270 comp rx=0 tx=0).stop 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 shutdown_connections 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614811b00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63bb7fe640 0 client.16953 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f6e80 0x5586147fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 --2- 172.21.15.113:0/1209800656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f64d0 0x5586147ff2e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 >> 172.21.15.113:0/1209800656 conn(0x5586147f0ba0 msgr2=0x5586147f28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 shutdown_connections 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 1 -- 172.21.15.113:0/1209800656 wait complete. 2024-04-22T07:20:48.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.303+0000 7f63e51c9740 20 client.16953 trim_cache size 0 max 16384 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 Processor -- start 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 -- start start 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x55861480b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1b0850 con 0x5586147f6e80 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ae0c0 con 0x5586147f64d0 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63e51c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1abf30 con 0x5586147ffa00 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49188/0 (socket says 172.21.15.113:49188) 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 -- 172.21.15.113:0/2007920030 learned_addr learned my addr 172.21.15.113:0/2007920030 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x55861480b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 -- 172.21.15.113:0/2007920030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 msgr2=0x5586147ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 -- 172.21.15.113:0/2007920030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 msgr2=0x55861480b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x55861480b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.316+0000 7f63db7fe640 1 -- 172.21.15.113:0/2007920030 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586147fc9e0 con 0x5586147f6e80 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63db7fe640 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147feda0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x5586147fbed0 tx=0x7f63d000d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c223f3a6842fdfc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63c1ffb640 1 -- 172.21.15.113:0/2007920030 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0015a50 con 0x5586147f6e80 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63c1ffb640 1 -- 172.21.15.113:0/2007920030 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f63d001f030 con 0x5586147f6e80 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63c1ffb640 1 -- 172.21.15.113:0/2007920030 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00143b0 con 0x5586147f6e80 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 -- 172.21.15.113:0/2007920030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 msgr2=0x5586147feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147feda0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x5586147fbed0 tx=0x7f63d000d7a0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 -- 172.21.15.113:0/2007920030 shutdown_connections 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147ffa00 0x5586147ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147f6e80 0x5586147feda0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2007920030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x55861480b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 -- 172.21.15.113:0/2007920030 >> 172.21.15.113:0/2007920030 conn(0x5586147f0ba0 msgr2=0x5586147efc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 -- 172.21.15.113:0/2007920030 shutdown_connections 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.317+0000 7f63e51c9740 1 -- 172.21.15.113:0/2007920030 wait complete. 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 -- start start 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 Processor -- start 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x558614807830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f6e80 0x558614807d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 0x558614802d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ac580 con 0x5586147ffa00 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1ad9f0 con 0x5586147f64d0 2024-04-22T07:20:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1acc50 con 0x5586147f6e80 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e51c9740 10 client.0 _ll_register_callbacks cb 0x7f63cc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 0x558614802d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 0x558614802d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49200/0 (socket says 172.21.15.113:49200) 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 -- 172.21.15.113:0/2025495447 learned_addr learned my addr 172.21.15.113:0/2025495447 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x558614807830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63db7fe640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f6e80 0x558614807d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 -- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f6e80 msgr2=0x558614807d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f6e80 0x558614807d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 -- 172.21.15.113:0/2025495447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 msgr2=0x558614807830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x558614807830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.318+0000 7f63e0b72640 1 -- 172.21.15.113:0/2025495447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558614803540 con 0x5586147ffa00 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e0b72640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 0x558614802d40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f63d400da70 tx=0x7f63d400a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279adbf93154ace6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63bb7fe640 1 -- 172.21.15.113:0/2025495447 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d400fc70 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d001f030 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63bb7fe640 1 -- 172.21.15.113:0/2025495447 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f63d400fe10 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63bb7fe640 1 -- 172.21.15.113:0/2025495447 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f63d4025430 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 10 client.16959 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558614875760 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63bb7fe640 1 -- 172.21.15.113:0/2025495447 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f63d001f030 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63bb7fe640 1 -- 172.21.15.113:0/2025495447 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f63d4024680 con 0x5586147ffa00 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 10 client.16959 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 10 client.16959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 20 client.16959 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614878000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 4 client.16959 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:20:48.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.319+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558614768c10 con 0x55861480f710 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614878000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63bb7fe640 10 client.16959 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63dbfff640 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614878000 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e766d6e983b75b7e server_cookie=b13ed077e0d86db0 in_seq=0 out_seq=0 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63bb7fe640 1 -- 172.21.15.113:0/2025495447 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558614768c10 con 0x55861480f710 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63bb7fe640 10 client.16959 handle_command_reply: tid=0 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63e51c9740 1 client.16959 shutdown 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 msgr2=0x558614802d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.320+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 0x558614802d40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f63d400da70 tx=0x7f63d400a840 comp rx=0 tx=0).stop 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 shutdown_connections 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55861480f710 0x558614878000 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5586147ffa00 0x558614802d40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586147f6e80 0x558614807d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63bb7fe640 0 client.16959 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 --2- 172.21.15.113:0/2025495447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586147f64d0 0x558614807830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 >> 172.21.15.113:0/2025495447 conn(0x5586147f0ba0 msgr2=0x5586147f7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 shutdown_connections 2024-04-22T07:20:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 1 -- 172.21.15.113:0/2025495447 wait complete. 2024-04-22T07:20:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e51c9740 20 client.16959 trim_cache size 0 max 16384 2024-04-22T07:20:48.325 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f63c40a3d20 msgr2=0x7f63c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f63c40a3d20 0x7f63c40a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f63d400afd0 tx=0x7f63d4005e90 comp rx=0 tx=0).stop 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 msgr2=0x7f63dc19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc19e5c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f63cc005940 tx=0x7f63cc004040 comp rx=0 tx=0).stop 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 shutdown_connections 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f63c40a3d20 0x7f63c40a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63dc198680 0x7f63dc198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc107260 0x7f63dc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 --2- 172.21.15.113:0/2680366093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63dc102190 0x7f63dc19e5c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 >> 172.21.15.113:0/2680366093 conn(0x7f63dc0793a0 msgr2=0x7f63dc0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 shutdown_connections 2024-04-22T07:20:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.321+0000 7f63e25d5640 1 -- 172.21.15.113:0/2680366093 wait complete. 2024-04-22T07:20:48.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 -- 172.21.15.113:0/2759959450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c100af0 msgr2=0x7f705c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 --2- 172.21.15.113:0/2759959450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c100af0 0x7f705c100ed0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 -- 172.21.15.113:0/2759959450 shutdown_connections 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 --2- 172.21.15.113:0/2759959450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c1072b0 0x7f705c10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 --2- 172.21.15.113:0/2759959450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705c1014a0 0x7f705c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 --2- 172.21.15.113:0/2759959450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c100af0 0x7f705c100ed0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 -- 172.21.15.113:0/2759959450 >> 172.21.15.113:0/2759959450 conn(0x7f705c0760e0 msgr2=0x7f705c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 -- 172.21.15.113:0/2759959450 shutdown_connections 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 -- 172.21.15.113:0/2759959450 wait complete. 2024-04-22T07:20:48.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.432+0000 7f70615a4640 1 Processor -- start 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 -- start start 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705c100af0 0x7f705c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c1072b0 0x7f705c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 0x7f705c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112390 con 0x7f705c198670 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112210 con 0x7f705c1072b0 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112510 con 0x7f705c100af0 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 0x7f705c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 0x7f705c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49210/0 (socket says 172.21.15.113:49210) 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 -- 172.21.15.113:0/848713107 learned_addr learned my addr 172.21.15.113:0/848713107 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705a7fc640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c1072b0 0x7f705c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705affd640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705c100af0 0x7f705c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 -- 172.21.15.113:0/848713107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705c100af0 msgr2=0x7f705c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705c100af0 0x7f705c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 -- 172.21.15.113:0/848713107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c1072b0 msgr2=0x7f705c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c1072b0 0x7f705c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 -- 172.21.15.113:0/848713107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c199270 con 0x7f705c198670 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f705b7fe640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 0x7f705c198af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f705400cce0 tx=0x7f705400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f7043fff640 1 -- 172.21.15.113:0/848713107 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705400f450 con 0x7f705c198670 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a5350 con 0x7f705c198670 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f7043fff640 1 -- 172.21.15.113:0/848713107 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f705400fac0 con 0x7f705c198670 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f7043fff640 1 -- 172.21.15.113:0/848713107 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054018be0 con 0x7f705c198670 2024-04-22T07:20:48.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.433+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a5700 con 0x7f705c198670 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.436+0000 7f7064190740 1 Processor -- start 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.436+0000 7f7043fff640 1 -- 172.21.15.113:0/848713107 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f705400fc70 con 0x7f705c198670 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.436+0000 7f7064190740 1 -- start start 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7064190740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0470 0x557e165b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7064190740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7064190740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 0x557e165c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7064190740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557e165151f0 con 0x557e165bde10 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7064190740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557e164b24b0 con 0x557e165b0e20 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7064190740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557e165111b0 con 0x557e165b0470 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 0x557e165c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 0x557e165c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49220/0 (socket says 172.21.15.113:49220) 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 -- 172.21.15.113:0/1014850624 learned_addr learned my addr 172.21.15.113:0/1014850624 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0470 0x557e165b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 -- 172.21.15.113:0/1014850624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0470 msgr2=0x557e165b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0470 0x557e165b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705affd640 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 -- 172.21.15.113:0/1014850624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 msgr2=0x557e165bd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165bd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 -- 172.21.15.113:0/1014850624 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e164a41b0 con 0x557e165bde10 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7043fff640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70440a3d20 0x7f70440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 0x557e165c0220 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7050005880 tx=0x7f7050004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ced849987839241 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f705affd640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70440a3d20 0x7f70440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f7043fff640 1 -- 172.21.15.113:0/848713107 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f70540d0ff0 con 0x7f705c198670 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f70417fa640 1 -- 172.21.15.113:0/1014850624 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050012ce0 con 0x557e165bde10 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f70417fa640 1 -- 172.21.15.113:0/1014850624 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7050005020 con 0x557e165bde10 2024-04-22T07:20:48.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.437+0000 7f70417fa640 1 -- 172.21.15.113:0/1014850624 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050004a00 con 0x557e165bde10 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 -- 172.21.15.113:0/1014850624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 msgr2=0x557e165c0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f705affd640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70440a3d20 0x7f70440a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f704c006010 tx=0x7f704c000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 0x557e165c0220 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7050005880 tx=0x7f7050004730 comp rx=0 tx=0).stop 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 -- 172.21.15.113:0/1014850624 shutdown_connections 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165bde10 0x557e165c0220 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 --2- 172.21.15.113:0/1014850624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0470 0x557e165b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 -- 172.21.15.113:0/1014850624 >> 172.21.15.113:0/1014850624 conn(0x557e165aaba0 msgr2=0x557e165aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 -- 172.21.15.113:0/1014850624 shutdown_connections 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 -- 172.21.15.113:0/1014850624 wait complete. 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 -- start start 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.438+0000 7f7064190740 1 Processor -- start 2024-04-22T07:20:48.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e165bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557e165c0d40 con 0x557e165b0470 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557e164ce500 con 0x557e165b0e20 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557e165c0ec0 con 0x557e165bde10 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 10 client.0 _ll_register_callbacks cb 0x557e1649cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e165bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e165bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49230/0 (socket says 172.21.15.113:49230) 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 -- 172.21.15.113:0/736560095 learned_addr learned my addr 172.21.15.113:0/736560095 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705b7fe640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705a7fc640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 -- 172.21.15.113:0/736560095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 msgr2=0x557e165bb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 -- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 msgr2=0x557e165baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165baad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 -- 172.21.15.113:0/736560095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e162273f0 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705a7fc640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f705affd640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e165bd740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f704c035000 tx=0x7f704c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c994d3876fa6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f703eff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f703eff5640 1 -- 172.21.15.113:0/736560095 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c04c020 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f703eff5640 1 -- 172.21.15.113:0/736560095 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f704c003cf0 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.439+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7050005020 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f703eff5640 1 -- 172.21.15.113:0/736560095 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f704c057050 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 10 client.16971 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557e162f17d0 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f703eff5640 1 -- 172.21.15.113:0/736560095 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7050005020 con 0x557e165b0470 2024-04-22T07:20:48.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f703eff5640 1 -- 172.21.15.113:0/736560095 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f704c004c20 con 0x557e165b0470 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 10 client.16971 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 20 client.16971 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e165ce130 0x557e165d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 4 client.16971 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e164b4930 con 0x557e165ce130 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.440+0000 7f705a7fc640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e165ce130 0x557e165d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f703eff5640 10 client.16971 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f705a7fc640 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e165ce130 0x557e165d05b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86d150a514c81487 server_cookie=72f7145b2ad83179 in_seq=0 out_seq=0 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f703eff5640 1 -- 172.21.15.113:0/736560095 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e164b4930 con 0x557e165ce130 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f703eff5640 10 client.16971 handle_command_reply: tid=0 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f7064190740 1 client.16971 shutdown 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 msgr2=0x557e165bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.441+0000 7f7064190740 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e165bd740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f704c035000 tx=0x7f704c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 shutdown_connections 2024-04-22T07:20:48.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e165ce130 0x557e165d05b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0e20 0x557e165baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f703eff5640 0 client.16971 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 --2- 172.21.15.113:0/736560095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e165bd740 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 >> 172.21.15.113:0/736560095 conn(0x557e165aaba0 msgr2=0x557e165ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 shutdown_connections 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 1 -- 172.21.15.113:0/736560095 wait complete. 2024-04-22T07:20:48.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.442+0000 7f7064190740 20 client.16971 trim_cache size 0 max 16384 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.454+0000 7f7064190740 1 Processor -- start 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 -- start start 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0470 0x557e165c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 0x557e165b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1ae570 con 0x557e165b0e20 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1abde0 con 0x557e165b0470 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7064190740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1a9c50 con 0x557e165bde10 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 0x557e165b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 0x557e165b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49232/0 (socket says 172.21.15.113:49232) 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 -- 172.21.15.113:0/3593862305 learned_addr learned my addr 172.21.15.113:0/3593862305 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705b7fe640 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705affd640 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0470 0x557e165c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 -- 172.21.15.113:0/3593862305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 msgr2=0x557e165bd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 -- 172.21.15.113:0/3593862305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0470 msgr2=0x557e165c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0470 0x557e165c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 -- 172.21.15.113:0/3593862305 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e165b1200 con 0x557e165b0e20 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705affd640 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0470 0x557e165c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f705a7fc640 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 0x557e165b6070 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7050000c00 tx=0x7f705000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ae8285c93d550f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7040ff9640 1 -- 172.21.15.113:0/3593862305 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705002d070 con 0x557e165b0e20 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7040ff9640 1 -- 172.21.15.113:0/3593862305 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7050016040 con 0x557e165b0e20 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.455+0000 7f7040ff9640 1 -- 172.21.15.113:0/3593862305 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050010a70 con 0x557e165b0e20 2024-04-22T07:20:48.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 -- 172.21.15.113:0/3593862305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 msgr2=0x557e165b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 0x557e165b6070 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7050000c00 tx=0x7f705000bbf0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 -- 172.21.15.113:0/3593862305 shutdown_connections 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165bde10 0x557e165bd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0e20 0x557e165b6070 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 --2- 172.21.15.113:0/3593862305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165b0470 0x557e165c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 -- 172.21.15.113:0/3593862305 >> 172.21.15.113:0/3593862305 conn(0x557e165aaba0 msgr2=0x557e165a9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 -- 172.21.15.113:0/3593862305 shutdown_connections 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 -- 172.21.15.113:0/3593862305 wait complete. 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 -- start start 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 Processor -- start 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e1662cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.456+0000 7f7064190740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0e20 0x557e1662d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165bde10 0x557e165c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1aa2a0 con 0x557e165b0470 2024-04-22T07:20:48.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1ab710 con 0x557e165bde10 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1aa970 con 0x557e165b0e20 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 10 client.0 _ll_register_callbacks cb 0x7f704c02ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e1662cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e1662cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49244/0 (socket says 172.21.15.113:49244) 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 -- 172.21.15.113:0/1202655154 learned_addr learned my addr 172.21.15.113:0/1202655154 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0e20 0x557e1662d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 -- 172.21.15.113:0/1202655154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0e20 msgr2=0x557e1662d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0e20 0x557e1662d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 -- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165bde10 msgr2=0x557e165c1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705b7fe640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165bde10 0x557e165c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165bde10 0x557e165c1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 -- 172.21.15.113:0/1202655154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e165c1d50 con 0x557e165b0470 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705b7fe640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165bde10 0x557e165c1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f705affd640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e1662cb60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f704c009a20 tx=0x7f704c02c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60819767b5f165a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f703eff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f703eff5640 1 -- 172.21.15.113:0/1202655154 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c060060 con 0x557e165b0470 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f703eff5640 1 -- 172.21.15.113:0/1202655154 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7f704c0039c0 con 0x557e165b0470 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7050016040 con 0x557e165b0470 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f703eff5640 1 -- 172.21.15.113:0/1202655154 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f704c067050 con 0x557e165b0470 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 10 client.16977 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557e166341d0 con 0x557e165b0470 2024-04-22T07:20:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.457+0000 7f703eff5640 1 -- 172.21.15.113:0/1202655154 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7050016040 con 0x557e165b0470 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f703eff5640 1 -- 172.21.15.113:0/1202655154 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7f704c005020 con 0x557e165b0470 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 10 client.16977 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 20 client.16977 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e166348f0 0x557e16636d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 4 client.16977 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e16522c10 con 0x557e166348f0 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e166348f0 0x557e16636d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f703eff5640 10 client.16977 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f705a7fc640 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e166348f0 0x557e16636d10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76dadd9bdaaca541 server_cookie=aad3dd014b5821b4 in_seq=0 out_seq=0 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f703eff5640 1 -- 172.21.15.113:0/1202655154 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e16522c10 con 0x557e166348f0 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f703eff5640 10 client.16977 handle_command_reply: tid=0 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.458+0000 7f7064190740 1 client.16977 shutdown 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 msgr2=0x557e1662cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e1662cb60 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f704c009a20 tx=0x7f704c02c6f0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 shutdown_connections 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557e166348f0 0x557e16636d10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e165bde10 0x557e165c1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557e165b0e20 0x557e1662d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f703eff5640 0 client.16977 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 --2- 172.21.15.113:0/1202655154 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557e165b0470 0x557e1662cb60 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 >> 172.21.15.113:0/1202655154 conn(0x557e165aaba0 msgr2=0x557e165b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 shutdown_connections 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 1 -- 172.21.15.113:0/1202655154 wait complete. 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f7064190740 20 client.16977 trim_cache size 0 max 16384 2024-04-22T07:20:48.464 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:20:48.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70440a3d20 msgr2=0x7f70440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70440a3d20 0x7f70440a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f704c006010 tx=0x7f704c000bc0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 msgr2=0x7f705c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:48.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 0x7f705c198af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f705400cce0 tx=0x7f705400a620 comp rx=0 tx=0).stop 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 shutdown_connections 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70440a3d20 0x7f70440a61e0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f704c006010 tx=0x7f704c000bc0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f705c198670 0x7f705c198af0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f705c1072b0 0x7f705c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 --2- 172.21.15.113:0/848713107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705c100af0 0x7f705c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 >> 172.21.15.113:0/848713107 conn(0x7f705c0760e0 msgr2=0x7f705c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 shutdown_connections 2024-04-22T07:20:48.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:48.459+0000 7f70615a4640 1 -- 172.21.15.113:0/848713107 wait complete. 2024-04-22T07:20:49.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:48 smithi133 ceph-mon[31198]: pgmap v186: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:49.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:48 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/199305699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:20:49.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:48 smithi154 ceph-mon[31862]: pgmap v186: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:49.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:48 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/199305699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:20:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:48 smithi113 ceph-mon[27933]: pgmap v186: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:48 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/199305699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:20:49.468 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:20:49.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:20:49.469 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:20:49.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 -- 172.21.15.113:0/2806106907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c100450 msgr2=0x7faf3c1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 --2- 172.21.15.113:0/2806106907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c100450 0x7faf3c1008d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009930 tx=0x7faf2c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:49.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 -- 172.21.15.113:0/2806106907 shutdown_connections 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 --2- 172.21.15.113:0/2806106907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf3c100e10 0x7faf3c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 --2- 172.21.15.113:0/2806106907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c100450 0x7faf3c1008d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 --2- 172.21.15.113:0/2806106907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3c106f30 0x7faf3c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.584+0000 7faf43225640 1 -- 172.21.15.113:0/2806106907 >> 172.21.15.113:0/2806106907 conn(0x7faf3c077410 msgr2=0x7faf3c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 -- 172.21.15.113:0/2806106907 shutdown_connections 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 -- 172.21.15.113:0/2806106907 wait complete. 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 Processor -- start 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 -- start start 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf3c100450 0x7faf3c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3c100e10 0x7faf3c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 0x7faf3c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c112430 con 0x7faf3c106f30 2024-04-22T07:20:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1122b0 con 0x7faf3c100e10 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf43225640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1125b0 con 0x7faf3c100450 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf417c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 0x7faf3c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf417c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 0x7faf3c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49262/0 (socket says 172.21.15.113:49262) 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.585+0000 7faf417c2640 1 -- 172.21.15.113:0/713802114 learned_addr learned my addr 172.21.15.113:0/713802114 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf40fc1640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf3c100450 0x7faf3c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf33fff640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3c100e10 0x7faf3c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf417c2640 1 -- 172.21.15.113:0/713802114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf3c100450 msgr2=0x7faf3c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf417c2640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf3c100450 0x7faf3c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf417c2640 1 -- 172.21.15.113:0/713802114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3c100e10 msgr2=0x7faf3c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf417c2640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3c100e10 0x7faf3c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf417c2640 1 -- 172.21.15.113:0/713802114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c199010 con 0x7faf3c106f30 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf417c2640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 0x7faf3c198750 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7faf3400b970 tx=0x7faf3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf31ffb640 1 -- 172.21.15.113:0/713802114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34019070 con 0x7faf3c106f30 2024-04-22T07:20:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf31ffb640 1 -- 172.21.15.113:0/713802114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faf3400cb40 con 0x7faf3c106f30 2024-04-22T07:20:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c199300 con 0x7faf3c106f30 2024-04-22T07:20:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf31ffb640 1 -- 172.21.15.113:0/713802114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34007650 con 0x7faf3c106f30 2024-04-22T07:20:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.586+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c1a5610 con 0x7faf3c106f30 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.588+0000 7faf31ffb640 1 -- 172.21.15.113:0/713802114 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7faf3401c070 con 0x7faf3c106f30 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 Processor -- start 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf31ffb640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faf240a3d20 0x7faf240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 -- start start 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf33fff640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faf240a3d20 0x7faf240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 0x5572721e0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557272134f00 con 0x5572721de220 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5572720d2590 con 0x5572721d1010 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf45e11740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557272130ec0 con 0x5572721d0660 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf31ffb640 1 -- 172.21.15.113:0/713802114 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7faf340cfe80 con 0x7faf3c106f30 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 0x5572721e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 0x5572721e0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49274/0 (socket says 172.21.15.113:49274) 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 -- 172.21.15.113:0/3211435250 learned_addr learned my addr 172.21.15.113:0/3211435250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf40fc1640 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf33fff640 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 -- 172.21.15.113:0/3211435250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 msgr2=0x5572721d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 -- 172.21.15.113:0/3211435250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 msgr2=0x5572721ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf417c2640 1 -- 172.21.15.113:0/3211435250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557271f0f7e0 con 0x5572721de220 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf33fff640 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.589+0000 7faf33fff640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faf240a3d20 0x7faf240a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006010 tx=0x7faf2c02c6b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf417c2640 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 0x5572721e0630 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7faf34098210 tx=0x7faf3409bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ec53697b017d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf22ffd640 1 -- 172.21.15.113:0/3211435250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf340b5070 con 0x5572721de220 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf22ffd640 1 -- 172.21.15.113:0/3211435250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faf340a0160 con 0x5572721de220 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf22ffd640 1 -- 172.21.15.113:0/3211435250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf340a4590 con 0x5572721de220 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 -- 172.21.15.113:0/3211435250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 msgr2=0x5572721e0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 0x5572721e0630 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7faf34098210 tx=0x7faf3409bcb0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 -- 172.21.15.113:0/3211435250 shutdown_connections 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721de220 0x5572721e0630 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 --2- 172.21.15.113:0/3211435250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.590+0000 7faf45e11740 1 -- 172.21.15.113:0/3211435250 >> 172.21.15.113:0/3211435250 conn(0x5572721cafb0 msgr2=0x5572721cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 -- 172.21.15.113:0/3211435250 shutdown_connections 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 -- 172.21.15.113:0/3211435250 wait complete. 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 -- start start 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 Processor -- start 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721db250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721de220 0x5572721dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5572721e1150 con 0x5572721d1010 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5572720ee580 con 0x5572721de220 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5572721e12d0 con 0x5572721d0660 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf45e11740 10 client.0 _ll_register_callbacks cb 0x5572720f4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf33fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf33fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721db790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49280/0 (socket says 172.21.15.113:49280) 2024-04-22T07:20:49.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.591+0000 7faf33fff640 1 -- 172.21.15.113:0/491094856 learned_addr learned my addr 172.21.15.113:0/491094856 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf40fc1640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721db250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf33fff640 1 -- 172.21.15.113:0/491094856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 msgr2=0x5572721db250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf417c2640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721de220 0x5572721dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf33fff640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721db250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf33fff640 1 -- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721de220 msgr2=0x5572721dbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf33fff640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721de220 0x5572721dbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf33fff640 1 -- 172.21.15.113:0/491094856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557271f6ff70 con 0x5572721d1010 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf417c2640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721de220 0x5572721dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf33fff640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721db790 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7faf2c035000 tx=0x7faf2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14da649bc8d5693 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf1bfff640 1 -- 172.21.15.113:0/491094856 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c050060 con 0x5572721d1010 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf1bfff640 1 -- 172.21.15.113:0/491094856 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7faf2c005450 con 0x5572721d1010 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf340a0160 con 0x5572721d1010 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf1bfff640 1 -- 172.21.15.113:0/491094856 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7faf2c057050 con 0x5572721d1010 2024-04-22T07:20:49.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf45e11740 10 client.16989 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557271e07170 con 0x5572721d1010 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.592+0000 7faf1bfff640 1 -- 172.21.15.113:0/491094856 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faf340a0160 con 0x5572721d1010 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf1bfff640 1 -- 172.21.15.113:0/491094856 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7faf2c003a30 con 0x5572721d1010 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf45e11740 10 client.16989 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf45e11740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf45e11740 20 client.16989 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf45e11740 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5572721ee770 0x5572721f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf45e11740 4 client.16989 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557272142920 con 0x5572721ee770 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf40fc1640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5572721ee770 0x5572721f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf1bfff640 10 client.16989 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.593+0000 7faf40fc1640 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5572721ee770 0x5572721f0bd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3679f3b427db8b server_cookie=61f5f6432e1f2518 in_seq=0 out_seq=0 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf1bfff640 1 -- 172.21.15.113:0/491094856 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557272142920 con 0x5572721ee770 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf1bfff640 10 client.16989 handle_command_reply: tid=0 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 client.16989 shutdown 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 msgr2=0x5572721db790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721db790 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7faf2c035000 tx=0x7faf2c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 shutdown_connections 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5572721ee770 0x5572721f0bd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721de220 0x5572721dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721db790 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf1bfff640 0 client.16989 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 --2- 172.21.15.113:0/491094856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721d0660 0x5572721db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.594+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 >> 172.21.15.113:0/491094856 conn(0x5572721cafb0 msgr2=0x5572721ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.595+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 shutdown_connections 2024-04-22T07:20:49.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.595+0000 7faf45e11740 1 -- 172.21.15.113:0/491094856 wait complete. 2024-04-22T07:20:49.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.595+0000 7faf45e11740 20 client.16989 trim_cache size 0 max 16384 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 Processor -- start 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 -- start start 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 0x5572721dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x5572721c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1ae680 con 0x5572721d0660 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1abef0 con 0x5572721d1010 2024-04-22T07:20:49.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.612+0000 7faf45e11740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1a9d40 con 0x5572721de220 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 0x5572721dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 0x5572721dd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49288/0 (socket says 172.21.15.113:49288) 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 -- 172.21.15.113:0/2353117132 learned_addr learned my addr 172.21.15.113:0/2353117132 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 -- 172.21.15.113:0/2353117132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 msgr2=0x5572721c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x5572721c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 -- 172.21.15.113:0/2353117132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 msgr2=0x5572721ea050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf33fff640 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ea050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 -- 172.21.15.113:0/2353117132 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572721d0db0 con 0x5572721d0660 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf40fc1640 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 0x5572721dd980 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x5572721f1410 tx=0x7faf380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e6f222a4a9fa83 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf227fc640 1 -- 172.21.15.113:0/2353117132 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380040f0 con 0x5572721d0660 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf227fc640 1 -- 172.21.15.113:0/2353117132 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faf3801d8f0 con 0x5572721d0660 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.613+0000 7faf227fc640 1 -- 172.21.15.113:0/2353117132 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38005650 con 0x5572721d0660 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- 172.21.15.113:0/2353117132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 msgr2=0x5572721dd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 0x5572721dd980 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x5572721f1410 tx=0x7faf380046c0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- 172.21.15.113:0/2353117132 shutdown_connections 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x5572721c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d1010 0x5572721ea050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- 172.21.15.113:0/2353117132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d0660 0x5572721dd980 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- 172.21.15.113:0/2353117132 >> 172.21.15.113:0/2353117132 conn(0x5572721cafb0 msgr2=0x5572721c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- 172.21.15.113:0/2353117132 shutdown_connections 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- 172.21.15.113:0/2353117132 wait complete. 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- start start 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 Processor -- start 2024-04-22T07:20:49.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d0660 0x5572721d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721d88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x557272253d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1aa3b0 con 0x5572721d1010 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1ab820 con 0x5572721d0660 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.614+0000 7faf45e11740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1aaa80 con 0x5572721de220 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf45e11740 10 client.0 _ll_register_callbacks cb 0x7faf2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721d88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721d88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49296/0 (socket says 172.21.15.113:49296) 2024-04-22T07:20:49.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf40fc1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d0660 0x5572721d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 -- 172.21.15.113:0/2381641022 learned_addr learned my addr 172.21.15.113:0/2381641022 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf417c2640 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x557272253d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 -- 172.21.15.113:0/2381641022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 msgr2=0x557272253d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x557272253d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 -- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d0660 msgr2=0x5572721d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d0660 0x5572721d83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 -- 172.21.15.113:0/2381641022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557272254250 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf33fff640 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721d88e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02c3a0 tx=0x7faf2c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a57a607175cab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf1bfff640 1 -- 172.21.15.113:0/2381641022 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c050040 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf1bfff640 1 -- 172.21.15.113:0/2381641022 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7faf2c005660 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf3801d8f0 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf1bfff640 1 -- 172.21.15.113:0/2381641022 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7faf2c068050 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf45e11740 10 client.16995 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557272255480 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.615+0000 7faf1bfff640 1 -- 172.21.15.113:0/2381641022 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faf3801d8f0 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf1bfff640 1 -- 172.21.15.113:0/2381641022 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7faf2c063030 con 0x5572721d1010 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf45e11740 10 client.16995 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf45e11740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:49.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf45e11740 20 client.16995 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf45e11740 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557272255c80 0x5572722580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf45e11740 4 client.16995 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55727213c790 con 0x557272255c80 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf40fc1640 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557272255c80 0x5572722580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf1bfff640 10 client.16995 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.616+0000 7faf40fc1640 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557272255c80 0x5572722580a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87cb770c5ebde44d server_cookie=9f70f534e4236f69 in_seq=0 out_seq=0 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf1bfff640 1 -- 172.21.15.113:0/2381641022 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55727213c790 con 0x557272255c80 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf1bfff640 10 client.16995 handle_command_reply: tid=0 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 client.16995 shutdown 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 msgr2=0x5572721d88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721d88e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02c3a0 tx=0x7faf2c0098a0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 shutdown_connections 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557272255c80 0x5572722580a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5572721de220 0x557272253d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572721d1010 0x5572721d88e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf1bfff640 0 client.16995 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 --2- 172.21.15.113:0/2381641022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5572721d0660 0x5572721d83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.617+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 >> 172.21.15.113:0/2381641022 conn(0x5572721cafb0 msgr2=0x5572721ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 shutdown_connections 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf45e11740 1 -- 172.21.15.113:0/2381641022 wait complete. 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf45e11740 20 client.16995 trim_cache size 0 max 16384 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:20:49.622 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "53316b90-2d85-4191-90c9-02afeb9684b5", 2024-04-22T07:20:49.623 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:20:49.623 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faf240a3d20 msgr2=0x7faf240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faf240a3d20 0x7faf240a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006010 tx=0x7faf2c02c6b0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 msgr2=0x7faf3c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 0x7faf3c198750 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7faf3400b970 tx=0x7faf3400be30 comp rx=0 tx=0).stop 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 shutdown_connections 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faf240a3d20 0x7faf240a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf3c106f30 0x7faf3c198750 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3c100e10 0x7faf3c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 --2- 172.21.15.113:0/713802114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf3c100450 0x7faf3c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 >> 172.21.15.113:0/713802114 conn(0x7faf3c077410 msgr2=0x7faf3c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 shutdown_connections 2024-04-22T07:20:49.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.618+0000 7faf43225640 1 -- 172.21.15.113:0/713802114 wait complete. 2024-04-22T07:20:49.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:20:49.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:20:49.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/3511110025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 msgr2=0x7fb6ac10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/3511110025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac10dd20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009ca0 tx=0x7fb6a402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/3511110025 shutdown_connections 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/3511110025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac10dd20 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/3511110025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6ac101a30 0x7fb6ac105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/3511110025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac101080 0x7fb6ac101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/3511110025 >> 172.21.15.113:0/3511110025 conn(0x7fb6ac078000 msgr2=0x7fb6ac078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/3511110025 shutdown_connections 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/3511110025 wait complete. 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 Processor -- start 2024-04-22T07:20:49.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.730+0000 7fb6b2e3e640 1 -- start start 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6ac101080 0x7fb6ac1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac19a0c0 0x7fb6ac1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1111f0 con 0x7fb6ac106150 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac111070 con 0x7fb6ac19a0c0 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac111370 con 0x7fb6ac101080 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49308/0 (socket says 172.21.15.113:49308) 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 -- 172.21.15.113:0/1315658037 learned_addr learned my addr 172.21.15.113:0/1315658037 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6ac101080 0x7fb6ac1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 -- 172.21.15.113:0/1315658037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6ac101080 msgr2=0x7fb6ac1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6ac101080 0x7fb6ac1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 -- 172.21.15.113:0/1315658037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac19a0c0 msgr2=0x7fb6ac1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b13db640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac19a0c0 0x7fb6ac1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac19a0c0 0x7fb6ac1a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 -- 172.21.15.113:0/1315658037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac1a4e40 con 0x7fb6ac106150 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6abfff640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac199b30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e9f0 tx=0x7fb6a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6a9ffb640 1 -- 172.21.15.113:0/1315658037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000ccc0 con 0x7fb6ac106150 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6a9ffb640 1 -- 172.21.15.113:0/1315658037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a00054f0 con 0x7fb6ac106150 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac1a5130 con 0x7fb6ac106150 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.731+0000 7fb6a9ffb640 1 -- 172.21.15.113:0/1315658037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0010690 con 0x7fb6ac106150 2024-04-22T07:20:49.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.732+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1a5630 con 0x7fb6ac106150 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6a9ffb640 1 -- 172.21.15.113:0/1315658037 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fb6a00046c0 con 0x7fb6ac106150 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 Processor -- start 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 -- start start 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425de0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6a9ffb640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb6940a3d20 0x7fb6940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425debe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558425d451e0 con 0x558425de0ea0 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558425ce2390 con 0x558425de9a80 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558425d411a0 con 0x558425de0580 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49312/0 (socket says 172.21.15.113:49312) 2024-04-22T07:20:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb6940a3d20 0x7fb6940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.734+0000 7fb6abfff640 1 -- 172.21.15.113:0/2272179136 learned_addr learned my addr 172.21.15.113:0/2272179136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6a9ffb640 1 -- 172.21.15.113:0/1315658037 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb6a00d0dc0 con 0x7fb6ac106150 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6b13db640 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425de0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6b13db640 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425debe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6abfff640 1 -- 172.21.15.113:0/2272179136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 msgr2=0x558425de0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6abfff640 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425de0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6abfff640 1 -- 172.21.15.113:0/2272179136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 msgr2=0x558425debe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6abfff640 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425debe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6abfff640 1 -- 172.21.15.113:0/2272179136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558425b217d0 con 0x558425de0ea0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6b13db640 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425de0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6abfff640 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de9540 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb6a006f6a0 tx=0x7fb6a00c7cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b69d977b6e1b2df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb6940a3d20 0x7fb6940a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005d00 tx=0x7fb69c00b410 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb692ffd640 1 -- 172.21.15.113:0/2272179136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a009d2e0 con 0x558425de0ea0 2024-04-22T07:20:49.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb692ffd640 1 -- 172.21.15.113:0/2272179136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a00676f0 con 0x558425de0ea0 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.735+0000 7fb692ffd640 1 -- 172.21.15.113:0/2272179136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00bd0b0 con 0x558425de0ea0 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2272179136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 msgr2=0x558425de9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de9540 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb6a006f6a0 tx=0x7fb6a00c7cf0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2272179136 shutdown_connections 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425debe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de9540 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2272179136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425de0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2272179136 >> 172.21.15.113:0/2272179136 conn(0x558425ddacb0 msgr2=0x558425ddb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2272179136 shutdown_connections 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2272179136 wait complete. 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 -- start start 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.736+0000 7fb6b5a2a740 1 Processor -- start 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425de75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558425dec810 con 0x558425de0ea0 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558425cfe2f0 con 0x558425de0580 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558425dec990 con 0x558425de9a80 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 10 client.0 _ll_register_callbacks cb 0x558425d0b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49328/0 (socket says 172.21.15.113:49328) 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 -- 172.21.15.113:0/2578657512 learned_addr learned my addr 172.21.15.113:0/2578657512 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b13db640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425de75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 -- 172.21.15.113:0/2578657512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 msgr2=0x558425de75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425de75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 -- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 msgr2=0x558425de6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 -- 172.21.15.113:0/2578657512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558425a573f0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6abfff640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de7060 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb6a009cb70 tx=0x7fb6a00bd2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de36fdd9dfcf6371 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb68bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb68bfff640 1 -- 172.21.15.113:0/2578657512 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a009cbe0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb68bfff640 1 -- 172.21.15.113:0/2578657512 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb6a00a53c0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a00676f0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb68bfff640 1 -- 172.21.15.113:0/2578657512 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb6a00a0f90 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 10 client.17007 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558425a170f0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.737+0000 7fb68bfff640 1 -- 172.21.15.113:0/2578657512 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a00676f0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb68bfff640 1 -- 172.21.15.113:0/2578657512 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fb6a00ceae0 con 0x558425de0ea0 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b5a2a740 10 client.17007 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b5a2a740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b5a2a740 20 client.17007 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425df9900 0x558425dfbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b5a2a740 4 client.17007 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558425d52c00 con 0x558425df9900 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425df9900 0x558425dfbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb68bfff640 10 client.17007 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.738+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425df9900 0x558425dfbcf0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=153c3ad175828c7e server_cookie=3f5ed59ad8b8d41b in_seq=0 out_seq=0 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb68bfff640 1 -- 172.21.15.113:0/2578657512 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558425d52c00 con 0x558425df9900 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb68bfff640 10 client.17007 handle_command_reply: tid=0 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 client.17007 shutdown 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 msgr2=0x558425de7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de7060 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb6a009cb70 tx=0x7fb6a00bd2e0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 shutdown_connections 2024-04-22T07:20:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425df9900 0x558425dfbcf0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425de75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de7060 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/2578657512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb68bfff640 0 client.17007 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 >> 172.21.15.113:0/2578657512 conn(0x558425ddacb0 msgr2=0x558425ddca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 shutdown_connections 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/2578657512 wait complete. 2024-04-22T07:20:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.739+0000 7fb6b5a2a740 20 client.17007 trim_cache size 0 max 16384 2024-04-22T07:20:49.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 Processor -- start 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 -- start start 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425ddc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1b07f0 con 0x558425de0ea0 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1ae060 con 0x558425de0580 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.756+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1abeb0 con 0x558425de9a80 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49336/0 (socket says 172.21.15.113:49336) 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 -- 172.21.15.113:0/3547150851 learned_addr learned my addr 172.21.15.113:0/3547150851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 -- 172.21.15.113:0/3547150851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 msgr2=0x558425ddc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6b13db640 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425ddc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425ddc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 -- 172.21.15.113:0/3547150851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 msgr2=0x558425de9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 -- 172.21.15.113:0/3547150851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558425de5ad0 con 0x558425de0ea0 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6abfff640 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de12e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb6a009c8e0 tx=0x7fb6a009d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7082a1e95ed97d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6927fc640 1 -- 172.21.15.113:0/3547150851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a009c540 con 0x558425de0ea0 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6927fc640 1 -- 172.21.15.113:0/3547150851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a009cbe0 con 0x558425de0ea0 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.757+0000 7fb6927fc640 1 -- 172.21.15.113:0/3547150851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a009c9b0 con 0x558425de0ea0 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/3547150851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 msgr2=0x558425de12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de12e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb6a009c8e0 tx=0x7fb6a009d3a0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/3547150851 shutdown_connections 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de9a80 0x558425ddc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425de12e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/3547150851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de0580 0x558425de9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/3547150851 >> 172.21.15.113:0/3547150851 conn(0x558425ddacb0 msgr2=0x558425ddc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/3547150851 shutdown_connections 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/3547150851 wait complete. 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- start start 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 Processor -- start 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425dff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425dfc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425dfcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1ac520 con 0x558425de0ea0 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1ad990 con 0x558425de9a80 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1acbf0 con 0x558425de0580 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.758+0000 7fb6b5a2a740 10 client.0 _ll_register_callbacks cb 0x7fb6a0004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425dfc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425dfc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49338/0 (socket says 172.21.15.113:49338) 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 -- 172.21.15.113:0/1526980024 learned_addr learned my addr 172.21.15.113:0/1526980024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425dff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6b13db640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425dfcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 -- 172.21.15.113:0/1526980024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 msgr2=0x558425dff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425dff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 -- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 msgr2=0x558425dfcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425dfcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 -- 172.21.15.113:0/1526980024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558425e5eae0 con 0x558425de0ea0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6abfff640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425dfc5c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0099560 tx=0x7fb6a009c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53297ef8462e4e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb68bfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb68bfff640 1 -- 172.21.15.113:0/1526980024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0065020 con 0x558425de0ea0 2024-04-22T07:20:49.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb68bfff640 1 -- 172.21.15.113:0/1526980024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8899+0+0 (secure 0 0 0) 0x7fb6a00ce610 con 0x558425de0ea0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a009cbe0 con 0x558425de0ea0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb68bfff640 1 -- 172.21.15.113:0/1526980024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb6a00a53c0 con 0x558425de0ea0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6b5a2a740 10 client.17013 fetch_fsmap learned FSMap version 48 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558425e5eae0 con 0x558425de0ea0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.759+0000 7fb68bfff640 1 -- 172.21.15.113:0/1526980024 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a009cbe0 con 0x558425de0ea0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb68bfff640 1 -- 172.21.15.113:0/1526980024 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 48) ==== 8358+0+0 (secure 0 0 0) 0x7fb6a0098ac0 con 0x558425de0ea0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b5a2a740 10 client.17013 fetch_fsmap finished waiting for FSMap version 48 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b5a2a740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b5a2a740 20 client.17013 populate_metadata read hostname 'smithi113' 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425e5ff00 0x558425e62320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b5a2a740 4 client.17013 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558425d4ca70 con 0x558425e5ff00 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425e5ff00 0x558425e62320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb6b0bda640 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425e5ff00 0x558425e62320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb71804d23b57db server_cookie=6d79fbd79123cb55 in_seq=0 out_seq=0 2024-04-22T07:20:49.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb68bfff640 10 client.17013 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb68bfff640 1 -- 172.21.15.113:0/1526980024 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558425d4ca70 con 0x558425e5ff00 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.760+0000 7fb68bfff640 10 client.17013 handle_command_reply: tid=0 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 client.17013 shutdown 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 msgr2=0x558425dfc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425dfc5c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0099560 tx=0x7fb6a009c540 comp rx=0 tx=0).stop 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 shutdown_connections 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x558425e5ff00 0x558425e62320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558425de9a80 0x558425dfcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558425de0ea0 0x558425dfc5c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb68bfff640 0 client.17013 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 --2- 172.21.15.113:0/1526980024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558425de0580 0x558425dff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 >> 172.21.15.113:0/1526980024 conn(0x558425ddacb0 msgr2=0x558425df5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 shutdown_connections 2024-04-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 1 -- 172.21.15.113:0/1526980024 wait complete. 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b5a2a740 20 client.17013 trim_cache size 0 max 16384 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: "53316b90-2d85-4191-90c9-02afeb9684b5": { 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "53316b90-2d85-4191-90c9-02afeb9684b5", 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:20:49.766 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:20:49.767 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:20:49.767 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:20:49.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb6940a3d20 msgr2=0x7fb6940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb6940a3d20 0x7fb6940a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005d00 tx=0x7fb69c00b410 comp rx=0 tx=0).stop 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 msgr2=0x7fb6ac199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac199b30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e9f0 tx=0x7fb6a000eeb0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 shutdown_connections 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fb6940a3d20 0x7fb6940a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac19a0c0 0x7fb6ac1a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6ac106150 0x7fb6ac199b30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 --2- 172.21.15.113:0/1315658037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6ac101080 0x7fb6ac1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 >> 172.21.15.113:0/1315658037 conn(0x7fb6ac078000 msgr2=0x7fb6ac0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 shutdown_connections 2024-04-22T07:20:49.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:20:49.761+0000 7fb6b2e3e640 1 -- 172.21.15.113:0/1315658037 wait complete. 2024-04-22T07:20:49.770 INFO:tasks.cephfs.filesystem:scrub status for tag:53316b90-2d85-4191-90c9-02afeb9684b5 - {'path': '/', 'tag': '53316b90-2d85-4191-90c9-02afeb9684b5', 'options': 'recursive,force'} 2024-04-22T07:20:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:50 smithi133 ceph-mon[31198]: pgmap v187: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:50 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:20:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:50 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:50 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:20:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:50 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:51.315 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:50 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:20:51.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:50 smithi154 ceph-mon[31862]: pgmap v187: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:51.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:50 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:20:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:50 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:50 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:20:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:50 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:51.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:50 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:20:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:50 smithi113 ceph-mon[27933]: pgmap v187: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:20:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:50 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:20:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:50 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:50 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:20:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:50 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:20:51.335 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:50 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:20:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:52 smithi133 ceph-mon[31198]: pgmap v188: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:20:53.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:52 smithi154 ceph-mon[31862]: pgmap v188: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:20:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:52 smithi113 ceph-mon[27933]: pgmap v188: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:20:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:54 smithi133 ceph-mon[31198]: pgmap v189: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:55.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:54 smithi154 ceph-mon[31862]: pgmap v189: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:55.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:54 smithi113 ceph-mon[27933]: pgmap v189: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:56 smithi133 ceph-mon[31198]: pgmap v190: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:20:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:56 smithi154 ceph-mon[31862]: pgmap v190: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:20:57.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:56 smithi113 ceph-mon[27933]: pgmap v190: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:20:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:58 smithi133 ceph-mon[31198]: pgmap v191: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:59.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:58 smithi154 ceph-mon[31862]: pgmap v191: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:20:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:58 smithi113 ceph-mon[27933]: pgmap v191: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:21:00.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:20:59 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:21:00.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:20:59 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:21:00.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:20:59 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:21:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:00 smithi133 ceph-mon[31198]: pgmap v192: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:21:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:00 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:21:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:00 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:21:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:00 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:21:01.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:00 smithi154 ceph-mon[31862]: pgmap v192: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:21:01.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:00 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:21:01.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:00 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:21:01.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:00 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:21:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:00 smithi113 ceph-mon[27933]: pgmap v192: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:21:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:00 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:21:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:00 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:21:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:00 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:21:03.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:02 smithi133 ceph-mon[31198]: pgmap v193: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:21:03.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:02 smithi154 ceph-mon[31862]: pgmap v193: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:21:03.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:02 smithi113 ceph-mon[27933]: pgmap v193: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:21:04.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:04 smithi133 ceph-mon[31198]: pgmap v194: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:04.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:04 smithi154 ceph-mon[31862]: pgmap v194: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:04.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:04 smithi113 ceph-mon[27933]: pgmap v194: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:06.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:06 smithi133 ceph-mon[31198]: pgmap v195: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-22T07:21:06.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:06 smithi154 ceph-mon[31862]: pgmap v195: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-22T07:21:06.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:06 smithi113 ceph-mon[27933]: pgmap v195: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-22T07:21:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:07 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:07.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:07 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:07.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:07 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:08.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:08 smithi133 ceph-mon[31198]: pgmap v196: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-22T07:21:08.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:08 smithi154 ceph-mon[31862]: pgmap v196: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-22T07:21:08.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:08 smithi113 ceph-mon[27933]: pgmap v196: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-22T07:21:10.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:10 smithi133 ceph-mon[31198]: pgmap v197: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-22T07:21:10.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:10 smithi154 ceph-mon[31862]: pgmap v197: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-22T07:21:10.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:10 smithi113 ceph-mon[27933]: pgmap v197: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-22T07:21:12.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:12 smithi133 ceph-mon[31198]: pgmap v198: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-22T07:21:12.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:12 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:12.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:12 smithi133 ceph-mon[31198]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T07:21:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:12 smithi154 ceph-mon[31862]: pgmap v198: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-22T07:21:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:12 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:12.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:12 smithi154 ceph-mon[31862]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T07:21:12.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:12 smithi113 ceph-mon[27933]: pgmap v198: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-22T07:21:12.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:12 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:12.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:12 smithi113 ceph-mon[27933]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T07:21:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:14 smithi133 ceph-mon[31198]: pgmap v200: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-22T07:21:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:14 smithi154 ceph-mon[31862]: pgmap v200: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-22T07:21:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:14 smithi113 ceph-mon[27933]: pgmap v200: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-22T07:21:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:16 smithi133 ceph-mon[31198]: pgmap v201: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:16.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:16 smithi154 ceph-mon[31862]: pgmap v201: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:16 smithi113 ceph-mon[27933]: pgmap v201: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:17.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:17 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:17.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:17 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:17.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:17 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:18 smithi133 ceph-mon[31198]: pgmap v202: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:18.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:18 smithi133 ceph-mon[31198]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T07:21:18.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:18 smithi154 ceph-mon[31862]: pgmap v202: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:18.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:18 smithi154 ceph-mon[31862]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T07:21:18.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:18 smithi113 ceph-mon[27933]: pgmap v202: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:18.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:18 smithi113 ceph-mon[27933]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T07:21:19.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:21:19.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:21:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 -- 172.21.15.113:0/1334997768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4100610 msgr2=0x7f82c4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/1334997768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4100610 0x7f82c4100a90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009900 tx=0x7f82c002bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 -- 172.21.15.113:0/1334997768 shutdown_connections 2024-04-22T07:21:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/1334997768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100fd0 0x7f82c410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/1334997768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4100610 0x7f82c4100a90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/1334997768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82c4106680 0x7f82c4106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 -- 172.21.15.113:0/1334997768 >> 172.21.15.113:0/1334997768 conn(0x7f82c4077360 msgr2=0x7f82c40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 -- 172.21.15.113:0/1334997768 shutdown_connections 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 -- 172.21.15.113:0/1334997768 wait complete. 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.886+0000 7f82cc0c8640 1 Processor -- start 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 -- start start 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82c4100610 0x7f82c41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100fd0 0x7f82c4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 0x7f82c41a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111210 con 0x7f82c4106680 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111090 con 0x7f82c4100610 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82cc0c8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111390 con 0x7f82c4100fd0 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 0x7f82c41a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 0x7f82c41a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57762/0 (socket says 172.21.15.113:57762) 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 -- 172.21.15.113:0/2759968291 learned_addr learned my addr 172.21.15.113:0/2759968291 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82c9e64640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82c4100610 0x7f82c41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82c9663640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100fd0 0x7f82c4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 -- 172.21.15.113:0/2759968291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100fd0 msgr2=0x7f82c4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100fd0 0x7f82c4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 -- 172.21.15.113:0/2759968291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82c4100610 msgr2=0x7f82c41995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82c4100610 0x7f82c41995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 -- 172.21.15.113:0/2759968291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a4bb0 con 0x7f82c4106680 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82ca665640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 0x7f82c41a4670 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00eaa0 tx=0x7f82bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.887+0000 7f82b6ffd640 1 -- 172.21.15.113:0/2759968291 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00cdb0 con 0x7f82c4106680 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.888+0000 7f82b6ffd640 1 -- 172.21.15.113:0/2759968291 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f82bc005020 con 0x7f82c4106680 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.888+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a4d80 con 0x7f82c4106680 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.888+0000 7f82b6ffd640 1 -- 172.21.15.113:0/2759968291 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0116d0 con 0x7f82c4106680 2024-04-22T07:21:19.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.888+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a52a0 con 0x7f82c4106680 2024-04-22T07:21:19.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82b6ffd640 1 -- 172.21.15.113:0/2759968291 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f82bc0051d0 con 0x7f82c4106680 2024-04-22T07:21:19.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 Processor -- start 2024-04-22T07:21:19.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 -- start start 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 0x5555c933a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9347950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c9347e90 0x5555c934a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5555c929f1e0 con 0x5555c933a230 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5555c923c390 con 0x5555c933ab50 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82ced96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5555c929b1a0 con 0x5555c9347e90 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.890+0000 7f82c9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 0x5555c933a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82b6ffd640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f82b00a3d20 0x7f82b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 0x5555c933a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57766/0 (socket says 172.21.15.113:57766) 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 -- 172.21.15.113:0/4100060079 learned_addr learned my addr 172.21.15.113:0/4100060079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9e64640 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9347950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82ca665640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f82b00a3d20 0x7f82b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c9347e90 0x5555c934a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 -- 172.21.15.113:0/4100060079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c9347e90 msgr2=0x5555c934a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c9347e90 0x5555c934a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82b6ffd640 1 -- 172.21.15.113:0/2759968291 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f82bc0d0ba0 con 0x7f82c4106680 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 -- 172.21.15.113:0/4100060079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 msgr2=0x5555c9347950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9347950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 -- 172.21.15.113:0/4100060079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c907b7d0 con 0x5555c933a230 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9e64640 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9347950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82c9663640 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 0x5555c933a610 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f82c002efd0 tx=0x7f82c00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93be6006a02d7878 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82affff640 1 -- 172.21.15.113:0/4100060079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004020 con 0x5555c933a230 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82ca665640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f82b00a3d20 0x7f82b00a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009270 tx=0x7f82bc0bd040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82affff640 1 -- 172.21.15.113:0/4100060079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f82c0005510 con 0x5555c933a230 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.891+0000 7f82affff640 1 -- 172.21.15.113:0/4100060079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004ae0 con 0x5555c933a230 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 -- 172.21.15.113:0/4100060079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 msgr2=0x5555c933a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 0x5555c933a610 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f82c002efd0 tx=0x7f82c00058b0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 -- 172.21.15.113:0/4100060079 shutdown_connections 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c9347e90 0x5555c934a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9347950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 --2- 172.21.15.113:0/4100060079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933a230 0x5555c933a610 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 -- 172.21.15.113:0/4100060079 >> 172.21.15.113:0/4100060079 conn(0x5555c9334cb0 msgr2=0x5555c9335090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 -- 172.21.15.113:0/4100060079 shutdown_connections 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 -- 172.21.15.113:0/4100060079 wait complete. 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 -- start start 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.892+0000 7f82ced96740 1 Processor -- start 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c9344e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9345370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c93458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5555c934ac10 con 0x5555c9347e90 2024-04-22T07:21:19.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5555c92582f0 con 0x5555c933ab50 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5555c934ad90 con 0x5555c933a230 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82ced96740 10 client.0 _ll_register_callbacks cb 0x5555c9265580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c93458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c93458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57768/0 (socket says 172.21.15.113:57768) 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 -- 172.21.15.113:0/3548310169 learned_addr learned my addr 172.21.15.113:0/3548310169 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9e64640 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9345370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 -- 172.21.15.113:0/3548310169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 msgr2=0x5555c9344e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c9344e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 -- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 msgr2=0x5555c9345370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9345370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.893+0000 7f82c9663640 1 -- 172.21.15.113:0/3548310169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c8fb13f0 con 0x5555c9347e90 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82c9663640 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c93458b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f82c002c7e0 tx=0x7f82c0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b181ea8e4a977bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ad7fa640 1 -- 172.21.15.113:0/3548310169 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0051060 con 0x5555c9347e90 2024-04-22T07:21:19.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ad7fa640 1 -- 172.21.15.113:0/3548310169 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f82c0004030 con 0x5555c9347e90 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c0005510 con 0x5555c9347e90 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ad7fa640 1 -- 172.21.15.113:0/3548310169 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f82c0004230 con 0x5555c9347e90 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 10 client.17025 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5555c8f710f0 con 0x5555c9347e90 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ad7fa640 1 -- 172.21.15.113:0/3548310169 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f82c0005510 con 0x5555c9347e90 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ad7fa640 1 -- 172.21.15.113:0/3548310169 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7f82c0003b00 con 0x5555c9347e90 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 10 client.17025 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 20 client.17025 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c9357cd0 0x5555c935a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 4 client.17025 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c92acc00 con 0x5555c9357cd0 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.894+0000 7f82c9e64640 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c9357cd0 0x5555c935a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.895+0000 7f82ad7fa640 10 client.17025 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.895+0000 7f82c9e64640 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c9357cd0 0x5555c935a0c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82902b9603fcacd4 server_cookie=dd2b88e2b5a52df9 in_seq=0 out_seq=0 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ad7fa640 1 -- 172.21.15.113:0/3548310169 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555c92acc00 con 0x5555c9357cd0 2024-04-22T07:21:19.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ad7fa640 10 client.17025 handle_command_reply: tid=0 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 client.17025 shutdown 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 msgr2=0x5555c93458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c93458b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f82c002c7e0 tx=0x7f82c0005270 comp rx=0 tx=0).stop 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 shutdown_connections 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c9357cd0 0x5555c935a0c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c93458b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ad7fa640 0 client.17025 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933ab50 0x5555c9345370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 --2- 172.21.15.113:0/3548310169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c9344e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 >> 172.21.15.113:0/3548310169 conn(0x5555c9334cb0 msgr2=0x5555c9336a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 shutdown_connections 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 1 -- 172.21.15.113:0/3548310169 wait complete. 2024-04-22T07:21:19.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.896+0000 7f82ced96740 20 client.17025 trim_cache size 0 max 16384 2024-04-22T07:21:19.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 Processor -- start 2024-04-22T07:21:19.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 -- start start 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933a230 0x5555c9347560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933ab50 0x5555c9343e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c9342090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ae610 con 0x5555c9347e90 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41abeb0 con 0x5555c933a230 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.913+0000 7f82ced96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41a9d20 con 0x5555c933ab50 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c9342090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c9342090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57776/0 (socket says 172.21.15.113:57776) 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 -- 172.21.15.113:0/2589694667 learned_addr learned my addr 172.21.15.113:0/2589694667 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933a230 0x5555c9347560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9e64640 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933ab50 0x5555c9343e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 -- 172.21.15.113:0/2589694667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933ab50 msgr2=0x5555c9343e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933ab50 0x5555c9343e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 -- 172.21.15.113:0/2589694667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933a230 msgr2=0x5555c9347560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933a230 0x5555c9347560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 -- 172.21.15.113:0/2589694667 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c934a280 con 0x5555c9347e90 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9e64640 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933ab50 0x5555c9343e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82c9663640 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c9342090 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f82c002c8a0 tx=0x7f82c002cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8cf03a55af3493 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82af7fe640 1 -- 172.21.15.113:0/2589694667 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0051040 con 0x5555c9347e90 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82af7fe640 1 -- 172.21.15.113:0/2589694667 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f82c0009780 con 0x5555c9347e90 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.914+0000 7f82af7fe640 1 -- 172.21.15.113:0/2589694667 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004990 con 0x5555c9347e90 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- 172.21.15.113:0/2589694667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 msgr2=0x5555c9342090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c9342090 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f82c002c8a0 tx=0x7f82c002cf90 comp rx=0 tx=0).stop 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- 172.21.15.113:0/2589694667 shutdown_connections 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c9347e90 0x5555c9342090 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933ab50 0x5555c9343e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- 172.21.15.113:0/2589694667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c933a230 0x5555c9347560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- 172.21.15.113:0/2589694667 >> 172.21.15.113:0/2589694667 conn(0x5555c9334cb0 msgr2=0x5555c9343700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- 172.21.15.113:0/2589694667 shutdown_connections 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- 172.21.15.113:0/2589694667 wait complete. 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- start start 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 Processor -- start 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c935d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 0x5555c935a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c9347e90 0x5555c935ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41aa370 con 0x5555c933ab50 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41ab7e0 con 0x5555c9347e90 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.915+0000 7f82ced96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41aaa40 con 0x5555c933a230 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ced96740 10 client.0 _ll_register_callbacks cb 0x7f82c002c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:19.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 0x5555c935a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 0x5555c935a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57782/0 (socket says 172.21.15.113:57782) 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 -- 172.21.15.113:0/4104301780 learned_addr learned my addr 172.21.15.113:0/4104301780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9663640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c935d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9663640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c9347e90 0x5555c935ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 -- 172.21.15.113:0/4104301780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 msgr2=0x5555c935d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c935d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 -- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c9347e90 msgr2=0x5555c935ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c9347e90 0x5555c935ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 -- 172.21.15.113:0/4104301780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c935b100 con 0x5555c933ab50 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9663640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c9347e90 0x5555c935ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82c9e64640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 0x5555c935a600 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x5555c929f2f0 tx=0x7f82b800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f24e5fee267e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ad7fa640 1 -- 172.21.15.113:0/4104301780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8021020 con 0x5555c933ab50 2024-04-22T07:21:19.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ad7fa640 1 -- 172.21.15.113:0/4104301780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f82b801d420 con 0x5555c933ab50 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c0009780 con 0x5555c933ab50 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ad7fa640 1 -- 172.21.15.113:0/4104301780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f82b802c050 con 0x5555c933ab50 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ced96740 10 client.17031 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5555c93bdc70 con 0x5555c933ab50 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.916+0000 7f82ad7fa640 1 -- 172.21.15.113:0/4104301780 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f82c0009780 con 0x5555c933ab50 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ad7fa640 1 -- 172.21.15.113:0/4104301780 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7f82b8011be0 con 0x5555c933ab50 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 10 client.17031 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 20 client.17031 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c93be420 0x5555c93c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 4 client.17031 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555c92a6a70 con 0x5555c93be420 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82c9663640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c93be420 0x5555c93c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:19.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82c9663640 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c93be420 0x5555c93c0840 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4d6eae6208be75 server_cookie=bed8f238aeac006e in_seq=0 out_seq=0 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ad7fa640 10 client.17031 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ad7fa640 1 -- 172.21.15.113:0/4104301780 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5555c92a6a70 con 0x5555c93be420 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ad7fa640 10 client.17031 handle_command_reply: tid=0 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.917+0000 7f82ced96740 1 client.17031 shutdown 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 msgr2=0x5555c935a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 0x5555c935a600 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x5555c929f2f0 tx=0x7f82b800bc80 comp rx=0 tx=0).stop 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 shutdown_connections 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5555c93be420 0x5555c93c0840 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555c9347e90 0x5555c935ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555c933ab50 0x5555c935a600 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ad7fa640 0 client.17031 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 --2- 172.21.15.113:0/4104301780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5555c933a230 0x5555c935d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 >> 172.21.15.113:0/4104301780 conn(0x5555c9334cb0 msgr2=0x5555c9336100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 shutdown_connections 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 1 -- 172.21.15.113:0/4104301780 wait complete. 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82ced96740 20 client.17031 trim_cache size 0 max 16384 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:21:19.923 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f82b00a3d20 msgr2=0x7f82b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f82b00a3d20 0x7f82b00a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009270 tx=0x7f82bc0bd040 comp rx=0 tx=0).stop 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 msgr2=0x7f82c41a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 0x7f82c41a4670 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00eaa0 tx=0x7f82bc00ef60 comp rx=0 tx=0).stop 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 shutdown_connections 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f82b00a3d20 0x7f82b00a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c4106680 0x7f82c41a4670 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82c4100fd0 0x7f82c4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 --2- 172.21.15.113:0/2759968291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82c4100610 0x7f82c41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 >> 172.21.15.113:0/2759968291 conn(0x7f82c4077360 msgr2=0x7f82c40fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 shutdown_connections 2024-04-22T07:21:19.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:19.918+0000 7f82cc0c8640 1 -- 172.21.15.113:0/2759968291 wait complete. 2024-04-22T07:21:19.927 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:21:19.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 -- 172.21.15.113:0/921410938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 msgr2=0x7f0f64101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/921410938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f64101620 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009930 tx=0x7f0f4c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 -- 172.21.15.113:0/921410938 shutdown_connections 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/921410938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64101b60 0x7f0f6410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/921410938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f64101620 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/921410938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f64105e30 0x7f0f64106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 -- 172.21.15.113:0/921410938 >> 172.21.15.113:0/921410938 conn(0x7f0f64077e30 msgr2=0x7f0f64078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 -- 172.21.15.113:0/921410938 shutdown_connections 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.028+0000 7f0f6a08c640 1 -- 172.21.15.113:0/921410938 wait complete. 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 Processor -- start 2024-04-22T07:21:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 -- start start 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f6410e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f64101b60 0x7f0f6410ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64105e30 0x7f0f6410f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64112390 con 0x7f0f641011a0 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64112210 con 0x7f0f64105e30 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f6a08c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64112510 con 0x7f0f64101b60 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f6410e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f6410e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57808/0 (socket says 172.21.15.113:57808) 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 -- 172.21.15.113:0/4039566824 learned_addr learned my addr 172.21.15.113:0/4039566824 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f63fff640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64105e30 0x7f0f6410f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 -- 172.21.15.113:0/4039566824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f64101b60 msgr2=0x7f0f6410ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f5bfff640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f64101b60 0x7f0f6410ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f64101b60 0x7f0f6410ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 -- 172.21.15.113:0/4039566824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64105e30 msgr2=0x7f0f6410f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64105e30 0x7f0f6410f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 -- 172.21.15.113:0/4039566824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f641a6cf0 con 0x7f0f641011a0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f63fff640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64105e30 0x7f0f6410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.029+0000 7f0f637fe640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f6410e720 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c970 tx=0x7f0f5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.030+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54007bb0 con 0x7f0f641011a0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.030+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0f54007d50 con 0x7f0f641011a0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.030+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f641a6fe0 con 0x7f0f641011a0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.030+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400f450 con 0x7f0f641011a0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.030+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641a74a0 con 0x7f0f641011a0 2024-04-22T07:21:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.031+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f64068de0 con 0x7f0f641011a0 2024-04-22T07:21:20.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.032+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0f540046c0 con 0x7f0f641011a0 2024-04-22T07:21:20.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.032+0000 7f0f617fa640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0f500a3dd0 0x7f0f500a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.032+0000 7f0f5bfff640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0f500a3dd0 0x7f0f500a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.032+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f0f540d06a0 con 0x7f0f641011a0 2024-04-22T07:21:20.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.033+0000 7f0f5bfff640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0f500a3dd0 0x7f0f500a6290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c036040 tx=0x7f0f4c005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.037+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f54014030 con 0x7f0f641011a0 2024-04-22T07:21:20.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.252+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f64108da0 con 0x7f0f641011a0 2024-04-22T07:21:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.252+0000 7f0f617fa640 1 -- 172.21.15.113:0/4039566824 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11886 (secure 0 0 0) 0x7f0f5409cba0 con 0x7f0f641011a0 2024-04-22T07:21:20.251 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:21:20.251 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:16.111029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,24446,25258]},"id":1}]} 2024-04-22T07:21:20.253 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 51 2024-04-22T07:21:20.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0f500a3dd0 msgr2=0x7f0f500a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0f500a3dd0 0x7f0f500a6290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c036040 tx=0x7f0f4c005fd0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 msgr2=0x7f0f6410e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f6410e720 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c970 tx=0x7f0f5400ce30 comp rx=0 tx=0).stop 2024-04-22T07:21:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 shutdown_connections 2024-04-22T07:21:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0f500a3dd0 0x7f0f500a6290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f64105e30 0x7f0f6410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f64101b60 0x7f0f6410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 --2- 172.21.15.113:0/4039566824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f641011a0 0x7f0f6410e720 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 >> 172.21.15.113:0/4039566824 conn(0x7f0f64077e30 msgr2=0x7f0f640fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 shutdown_connections 2024-04-22T07:21:20.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.254+0000 7f0f6a08c640 1 -- 172.21.15.113:0/4039566824 wait complete. 2024-04-22T07:21:20.263 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:21:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 -- 172.21.15.113:0/2993879748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c41014a0 msgr2=0x7f61c4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/2993879748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c41014a0 0x7f61c4106bd0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009900 tx=0x7f61b402bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 -- 172.21.15.113:0/2993879748 shutdown_connections 2024-04-22T07:21:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/2993879748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c41072b0 0x7f61c410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/2993879748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c41014a0 0x7f61c4106bd0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/2993879748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c4100af0 0x7f61c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 -- 172.21.15.113:0/2993879748 >> 172.21.15.113:0/2993879748 conn(0x7f61c40760e0 msgr2=0x7f61c40764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 -- 172.21.15.113:0/2993879748 shutdown_connections 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.366+0000 7f61cc2fd640 1 -- 172.21.15.113:0/2993879748 wait complete. 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 Processor -- start 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 -- start start 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c4100af0 0x7f61c419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c41072b0 0x7f61c419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 0x7f61c4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41123d0 con 0x7f61c4198730 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4112250 con 0x7f61c4100af0 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61cc2fd640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61c4112550 con 0x7f61c41072b0 2024-04-22T07:21:20.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 0x7f61c4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 0x7f61c4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57828/0 (socket says 172.21.15.113:57828) 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 -- 172.21.15.113:0/3368000292 learned_addr learned my addr 172.21.15.113:0/3368000292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca099640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c4100af0 0x7f61c419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 -- 172.21.15.113:0/3368000292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c41072b0 msgr2=0x7f61c419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c41072b0 0x7f61c419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 -- 172.21.15.113:0/3368000292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c4100af0 msgr2=0x7f61c419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c4100af0 0x7f61c419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 -- 172.21.15.113:0/3368000292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c4199330 con 0x7f61c4198730 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca099640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c4100af0 0x7f61c419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.367+0000 7f61ca89a640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 0x7f61c4198bb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009fd0 tx=0x7f61bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.368+0000 7f61bb7fe640 1 -- 172.21.15.113:0/3368000292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc017070 con 0x7f61c4198730 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.368+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c41a5410 con 0x7f61c4198730 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.368+0000 7f61bb7fe640 1 -- 172.21.15.113:0/3368000292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61bc003e40 con 0x7f61c4198730 2024-04-22T07:21:20.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.368+0000 7f61bb7fe640 1 -- 172.21.15.113:0/3368000292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc004820 con 0x7f61c4198730 2024-04-22T07:21:20.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.368+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c41a5810 con 0x7f61c4198730 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.370+0000 7f61bb7fe640 1 -- 172.21.15.113:0/3368000292 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f61bc003970 con 0x7f61c4198730 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.370+0000 7f61cefc9740 1 Processor -- start 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.370+0000 7f61cefc9740 1 -- start start 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556984644d0 0x5556984648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 0x55569846d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569846da00 0x55569846fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61cefc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5556983c91f0 con 0x555698464e80 2024-04-22T07:21:20.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61cefc9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5556983664b0 con 0x55569846da00 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61cefc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5556983c51b0 con 0x5556984644d0 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61bb7fe640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61b00a3d20 0x7f61b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 0x55569846d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 0x55569846d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57830/0 (socket says 172.21.15.113:57830) 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 -- 172.21.15.113:0/2450840971 learned_addr learned my addr 172.21.15.113:0/2450840971 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca89a640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61b00a3d20 0x7f61b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61c9898640 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556984644d0 0x5556984648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61c9898640 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569846da00 0x55569846fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61bb7fe640 1 -- 172.21.15.113:0/3368000292 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f61bc0d08a0 con 0x7f61c4198730 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 -- 172.21.15.113:0/2450840971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556984644d0 msgr2=0x5556984648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556984644d0 0x5556984648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 -- 172.21.15.113:0/2450840971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569846da00 msgr2=0x55569846fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569846da00 0x55569846fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 -- 172.21.15.113:0/2450840971 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556983581b0 con 0x555698464e80 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca89a640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61b00a3d20 0x7f61b00a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61bc099070 tx=0x7f61bc06e040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61ca099640 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 0x55569846d430 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f61c000aa10 tx=0x7f61c000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92b989995b4181f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61b8bfe640 1 -- 172.21.15.113:0/2450840971 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0015070 con 0x555698464e80 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.371+0000 7f61b8bfe640 1 -- 172.21.15.113:0/2450840971 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61c000cc60 con 0x555698464e80 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61b8bfe640 1 -- 172.21.15.113:0/2450840971 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00186b0 con 0x555698464e80 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 -- 172.21.15.113:0/2450840971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 msgr2=0x55569846d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 0x55569846d430 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f61c000aa10 tx=0x7f61c000aed0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 -- 172.21.15.113:0/2450840971 shutdown_connections 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569846da00 0x55569846fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555698464e80 0x55569846d430 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 --2- 172.21.15.113:0/2450840971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5556984644d0 0x5556984648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 -- 172.21.15.113:0/2450840971 >> 172.21.15.113:0/2450840971 conn(0x55569845eba0 msgr2=0x55569845ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 -- 172.21.15.113:0/2450840971 shutdown_connections 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.372+0000 7f61cefc9740 1 -- 172.21.15.113:0/2450840971 wait complete. 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 -- start start 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 Processor -- start 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556984644d0 0x55569846d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 0x55569846ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555698470970 con 0x55569846da00 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555698382500 con 0x5556984644d0 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555698470af0 con 0x555698464e80 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61cefc9740 10 client.0 _ll_register_callbacks cb 0x555698350b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 0x55569846ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 0x55569846ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57844/0 (socket says 172.21.15.113:57844) 2024-04-22T07:21:20.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 -- 172.21.15.113:0/16344828 learned_addr learned my addr 172.21.15.113:0/16344828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556984644d0 0x55569846d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61ca099640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 -- 172.21.15.113:0/16344828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 msgr2=0x55569846a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 -- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556984644d0 msgr2=0x55569846d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556984644d0 0x55569846d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.373+0000 7f61c9898640 1 -- 172.21.15.113:0/16344828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556980db3f0 con 0x55569846da00 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61c9898640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 0x55569846ab30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f61c4074670 tx=0x7f61b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0d83a8c449e822 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61adffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61adffb640 1 -- 172.21.15.113:0/16344828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4004230 con 0x55569846da00 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61adffb640 1 -- 172.21.15.113:0/16344828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f61b40043d0 con 0x55569846da00 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c000cc60 con 0x55569846da00 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61adffb640 1 -- 172.21.15.113:0/16344828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f61b4005660 con 0x55569846da00 2024-04-22T07:21:20.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 10 client.17049 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5556981a57d0 con 0x55569846da00 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61adffb640 1 -- 172.21.15.113:0/16344828 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61c000cc60 con 0x55569846da00 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61adffb640 1 -- 172.21.15.113:0/16344828 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7f61b404c030 con 0x55569846da00 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 10 client.17049 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 20 client.17049 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569847dbd0 0x555698480000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 4 client.17049 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.374+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555698368930 con 0x55569847dbd0 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.375+0000 7f61ca099640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569847dbd0 0x555698480000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.375+0000 7f61adffb640 10 client.17049 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.375+0000 7f61ca099640 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569847dbd0 0x555698480000 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d5633aa9794ffd server_cookie=f6aeff4c5b3dc0c9 in_seq=0 out_seq=0 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61adffb640 1 -- 172.21.15.113:0/16344828 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555698368930 con 0x55569847dbd0 2024-04-22T07:21:20.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61adffb640 10 client.17049 handle_command_reply: tid=0 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 client.17049 shutdown 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 msgr2=0x55569846ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 0x55569846ab30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f61c4074670 tx=0x7f61b4000bc0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 shutdown_connections 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569847dbd0 0x555698480000 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55569846da00 0x55569846ab30 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61adffb640 0 client.17049 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 --2- 172.21.15.113:0/16344828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556984644d0 0x55569846d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 >> 172.21.15.113:0/16344828 conn(0x55569845eba0 msgr2=0x5556984608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 shutdown_connections 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 1 -- 172.21.15.113:0/16344828 wait complete. 2024-04-22T07:21:20.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.376+0000 7f61cefc9740 20 client.17049 trim_cache size 0 max 16384 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 Processor -- start 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 -- start start 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569845eba0 0x555698479890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x55569845da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41b0900 con 0x5556984644d0 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41ae1a0 con 0x55569845eba0 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61cefc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41ac010 con 0x555698464e80 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61ca099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x55569845da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61ca099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x55569845da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57854/0 (socket says 172.21.15.113:57854) 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.389+0000 7f61ca099640 1 -- 172.21.15.113:0/1107997974 learned_addr learned my addr 172.21.15.113:0/1107997974 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61ca099640 1 -- 172.21.15.113:0/1107997974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 msgr2=0x55569846cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61ca099640 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61ca099640 1 -- 172.21.15.113:0/1107997974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569845eba0 msgr2=0x555698479890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61c9898640 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569845eba0 0x555698479890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61ca099640 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569845eba0 0x555698479890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61ca099640 1 -- 172.21.15.113:0/1107997974 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555698464c20 con 0x5556984644d0 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61ca099640 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x55569845da80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f61c000a930 tx=0x7f61c000a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be56e0ef7a3a6429 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61affff640 1 -- 172.21.15.113:0/1107997974 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0015040 con 0x5556984644d0 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61affff640 1 -- 172.21.15.113:0/1107997974 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61c000cdb0 con 0x5556984644d0 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61affff640 1 -- 172.21.15.113:0/1107997974 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00275a0 con 0x5556984644d0 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61cefc9740 1 -- 172.21.15.113:0/1107997974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 msgr2=0x55569845da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.390+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x55569845da80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f61c000a930 tx=0x7f61c000a180 comp rx=0 tx=0).stop 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- 172.21.15.113:0/1107997974 shutdown_connections 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555698464e80 0x55569846cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x55569845da80 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1107997974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55569845eba0 0x555698479890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- 172.21.15.113:0/1107997974 >> 172.21.15.113:0/1107997974 conn(0x55569846ac90 msgr2=0x55569845d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- 172.21.15.113:0/1107997974 shutdown_connections 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- 172.21.15.113:0/1107997974 wait complete. 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- start start 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 Processor -- start 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55569845eba0 0x5556984761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x555698471510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555698464e80 0x555698471a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41ac660 con 0x5556984644d0 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41adad0 con 0x555698464e80 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41acd30 con 0x55569845eba0 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61cefc9740 10 client.0 _ll_register_callbacks cb 0x7f61b4005520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:20.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x555698471510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x555698471510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57866/0 (socket says 172.21.15.113:57866) 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 -- 172.21.15.113:0/1721496382 learned_addr learned my addr 172.21.15.113:0/1721496382 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61c9898640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555698464e80 0x555698471a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 -- 172.21.15.113:0/1721496382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55569845eba0 msgr2=0x5556984761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55569845eba0 0x5556984761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 -- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555698464e80 msgr2=0x555698471a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555698464e80 0x555698471a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.391+0000 7f61ca099640 1 -- 172.21.15.113:0/1721496382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555698472280 con 0x5556984644d0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61c9898640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555698464e80 0x555698471a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61ca099640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x555698471510 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f61c000a3d0 tx=0x7f61c000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0243680725f4b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61adffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61adffb640 1 -- 172.21.15.113:0/1721496382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0015070 con 0x5556984644d0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61adffb640 1 -- 172.21.15.113:0/1721496382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f61c0011800 con 0x5556984644d0 2024-04-22T07:21:20.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c000cdb0 con 0x5556984644d0 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61adffb640 1 -- 172.21.15.113:0/1721496382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f61c002b070 con 0x5556984644d0 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 10 client.17055 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5556984e41a0 con 0x5556984644d0 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61adffb640 1 -- 172.21.15.113:0/1721496382 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61c000cdb0 con 0x5556984644d0 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61adffb640 1 -- 172.21.15.113:0/1721496382 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7f61c001a6c0 con 0x5556984644d0 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 10 client.17055 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 20 client.17055 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569846da00 0x5556984e6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 4 client.17055 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.392+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556983d6c10 con 0x55569846da00 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61c9898640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569846da00 0x5556984e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61adffb640 10 client.17055 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:20.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61c9898640 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569846da00 0x5556984e6530 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3536c72c69f9b80 server_cookie=6be4bf4c3e04e6e2 in_seq=0 out_seq=0 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61adffb640 1 -- 172.21.15.113:0/1721496382 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556983d6c10 con 0x55569846da00 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61adffb640 10 client.17055 handle_command_reply: tid=0 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 client.17055 shutdown 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 msgr2=0x555698471510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x555698471510 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f61c000a3d0 tx=0x7f61c000cbb0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 shutdown_connections 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55569846da00 0x5556984e6530 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555698464e80 0x555698471a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5556984644d0 0x555698471510 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 --2- 172.21.15.113:0/1721496382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55569845eba0 0x5556984761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61adffb640 0 client.17055 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 >> 172.21.15.113:0/1721496382 conn(0x55569846ac90 msgr2=0x555698460830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 shutdown_connections 2024-04-22T07:21:20.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 1 -- 172.21.15.113:0/1721496382 wait complete. 2024-04-22T07:21:20.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.393+0000 7f61cefc9740 20 client.17055 trim_cache size 0 max 16384 2024-04-22T07:21:20.398 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61b00a3d20 msgr2=0x7f61b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61b00a3d20 0x7f61b00a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f61bc099070 tx=0x7f61bc06e040 comp rx=0 tx=0).stop 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 msgr2=0x7f61c4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 0x7f61c4198bb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009fd0 tx=0x7f61bc00b260 comp rx=0 tx=0).stop 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 shutdown_connections 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61b00a3d20 0x7f61b00a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c4198730 0x7f61c4198bb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c41072b0 0x7f61c419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 --2- 172.21.15.113:0/3368000292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c4100af0 0x7f61c419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 >> 172.21.15.113:0/3368000292 conn(0x7f61c40760e0 msgr2=0x7f61c40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 shutdown_connections 2024-04-22T07:21:20.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.394+0000 7f61cc2fd640 1 -- 172.21.15.113:0/3368000292 wait complete. 2024-04-22T07:21:20.402 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/1884468016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c6806b230 msgr2=0x7f0c68106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/1884468016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c6806b230 0x7f0c68106b90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009900 tx=0x7f0c5002bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/1884468016 shutdown_connections 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/1884468016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c68107270 0x7f0c6810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/1884468016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c6806b230 0x7f0c68106b90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/1884468016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c6806a880 0x7f0c6806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/1884468016 >> 172.21.15.113:0/1884468016 conn(0x7f0c68077eb0 msgr2=0x7f0c680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/1884468016 shutdown_connections 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/1884468016 wait complete. 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.504+0000 7f0c6d4ba640 1 Processor -- start 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 -- start start 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c6806a880 0x7f0c6819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c68107270 0x7f0c6819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 0x7f0c68198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681122d0 con 0x7f0c68198630 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68112150 con 0x7f0c68107270 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c6d4ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68112450 con 0x7f0c6806a880 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 0x7f0c68198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 0x7f0c68198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57876/0 (socket says 172.21.15.113:57876) 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 -- 172.21.15.113:0/3999222506 learned_addr learned my addr 172.21.15.113:0/3999222506 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c6806a880 0x7f0c6819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c68107270 0x7f0c6819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 -- 172.21.15.113:0/3999222506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c6806a880 msgr2=0x7f0c6819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c6806a880 0x7f0c6819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 -- 172.21.15.113:0/3999222506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c68107270 msgr2=0x7f0c6819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c68107270 0x7f0c6819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 -- 172.21.15.113:0/3999222506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c68199230 con 0x7f0c68198630 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.505+0000 7f0c677fe640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 0x7f0c68198ab0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000eaa0 tx=0x7f0c6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.506+0000 7f0c64ff9640 1 -- 172.21.15.113:0/3999222506 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000cdb0 con 0x7f0c68198630 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.506+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c681a5310 con 0x7f0c68198630 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.506+0000 7f0c64ff9640 1 -- 172.21.15.113:0/3999222506 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c60005020 con 0x7f0c68198630 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.506+0000 7f0c64ff9640 1 -- 172.21.15.113:0/3999222506 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c600116d0 con 0x7f0c68198630 2024-04-22T07:21:20.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.506+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c681a56c0 con 0x7f0c68198630 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c64ff9640 1 -- 172.21.15.113:0/3999222506 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0c600051d0 con 0x7f0c68198630 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 Processor -- start 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 -- start start 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409a6e80 0x5643409af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409afa00 0x5643409b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56434090b1f0 con 0x5643409a64d0 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5643408a84b0 con 0x5643409a6e80 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c70184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5643409071b0 con 0x5643409afa00 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c64ff9640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c540a3d20 0x7f0c540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57890/0 (socket says 172.21.15.113:57890) 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.508+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1819840385 learned_addr learned my addr 172.21.15.113:0/1819840385 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c677fe640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c540a3d20 0x7f0c540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c64ff9640 1 -- 172.21.15.113:0/3999222506 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f0c600d0b20 con 0x7f0c68198630 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409a6e80 0x5643409af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1819840385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409afa00 msgr2=0x5643409b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409afa00 0x5643409b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1819840385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409a6e80 msgr2=0x5643409af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409a6e80 0x5643409af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1819840385 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434089a1b0 con 0x5643409a64d0 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409a6e80 0x5643409af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409a68b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f0c6806a460 tx=0x7f0c5002c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaca00f7dd1a8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c677fe640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c540a3d20 0x7f0c540a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000efd0 tx=0x7f0c600054c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5d7fa640 1 -- 172.21.15.113:0/1819840385 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5002cb50 con 0x5643409a64d0 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5d7fa640 1 -- 172.21.15.113:0/1819840385 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c50005040 con 0x5643409a64d0 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.509+0000 7f0c5d7fa640 1 -- 172.21.15.113:0/1819840385 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004a40 con 0x5643409a64d0 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- 172.21.15.113:0/1819840385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 msgr2=0x5643409a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409a68b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f0c6806a460 tx=0x7f0c5002c750 comp rx=0 tx=0).stop 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- 172.21.15.113:0/1819840385 shutdown_connections 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409afa00 0x5643409b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409a6e80 0x5643409af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- 172.21.15.113:0/1819840385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409a68b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- 172.21.15.113:0/1819840385 >> 172.21.15.113:0/1819840385 conn(0x5643409a0ba0 msgr2=0x5643409a0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- 172.21.15.113:0/1819840385 shutdown_connections 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- 172.21.15.113:0/1819840385 wait complete. 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- start start 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 Processor -- start 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x5643409ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5643409b2910 con 0x5643409a64d0 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5643408c4500 con 0x5643409afa00 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5643409b2a90 con 0x5643409a6e80 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c70184740 10 client.0 _ll_register_callbacks cb 0x564340892b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.510+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57906/0 (socket says 172.21.15.113:57906) 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1823743463 learned_addr learned my addr 172.21.15.113:0/1823743463 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 msgr2=0x5643409ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1823743463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 msgr2=0x5643409ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x5643409ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 -- 172.21.15.113:0/1823743463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434061d3f0 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409af3c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0c5002cd60 tx=0x7f0c50005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea3426e86307294 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/1823743463 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004cd0 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/1823743463 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f0c50034440 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c50005040 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/1823743463 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0c50004e70 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c70184740 10 client.17067 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5643406e77d0 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.511+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/1823743463 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c50005040 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/1823743463 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7f0c500457c0 con 0x5643409a64d0 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c70184740 10 client.17067 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c70184740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c70184740 20 client.17067 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:20.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c70184740 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5643409bfa50 0x5643409c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c70184740 4 client.17067 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643408aa930 con 0x5643409bfa50 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5643409bfa50 0x5643409c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5643409bfa50 0x5643409c1e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e29949fa712e0ad5 server_cookie=5f18b9df24448140 in_seq=0 out_seq=0 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.512+0000 7f0c4e7fc640 10 client.17067 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/1823743463 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643408aa930 con 0x5643409bfa50 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c4e7fc640 10 client.17067 handle_command_reply: tid=0 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 client.17067 shutdown 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 msgr2=0x5643409af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409af3c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0c5002cd60 tx=0x7f0c50005640 comp rx=0 tx=0).stop 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 shutdown_connections 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5643409bfa50 0x5643409c1e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x5643409ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c4e7fc640 0 client.17067 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:20.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 --2- 172.21.15.113:0/1823743463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409af3c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 >> 172.21.15.113:0/1823743463 conn(0x5643409a0ba0 msgr2=0x5643409a28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 shutdown_connections 2024-04-22T07:21:20.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 1 -- 172.21.15.113:0/1823743463 wait complete. 2024-04-22T07:21:20.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.513+0000 7f0c70184740 20 client.17067 trim_cache size 0 max 16384 2024-04-22T07:21:20.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.526+0000 7f0c70184740 1 Processor -- start 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.526+0000 7f0c70184740 1 -- start start 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c70184740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c70184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c70184740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x5643409af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c70184740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681b07f0 con 0x5643409a64d0 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c70184740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681ae090 con 0x5643409afa00 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c70184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681abf00 con 0x5643409a6e80 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57912/0 (socket says 172.21.15.113:57912) 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 -- 172.21.15.113:0/3590514079 learned_addr learned my addr 172.21.15.113:0/3590514079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 -- 172.21.15.113:0/3590514079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 msgr2=0x5643409aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 -- 172.21.15.113:0/3590514079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 msgr2=0x5643409af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x5643409af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 -- 172.21.15.113:0/3590514079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643409ad200 con 0x5643409a64d0 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409c5250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0c5002dfd0 tx=0x7f0c500056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f210002429e937 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5cff9640 1 -- 172.21.15.113:0/3590514079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50005010 con 0x5643409a64d0 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5cff9640 1 -- 172.21.15.113:0/3590514079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c500458f0 con 0x5643409a64d0 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.527+0000 7f0c5cff9640 1 -- 172.21.15.113:0/3590514079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004cd0 con 0x5643409a64d0 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 -- 172.21.15.113:0/3590514079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 msgr2=0x5643409c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409c5250 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0c5002dfd0 tx=0x7f0c500056f0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 -- 172.21.15.113:0/3590514079 shutdown_connections 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x5643409af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a6e80 0x5643409aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 --2- 172.21.15.113:0/3590514079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a64d0 0x5643409c5250 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 -- 172.21.15.113:0/3590514079 >> 172.21.15.113:0/3590514079 conn(0x5643409a0ba0 msgr2=0x5643409bb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 -- 172.21.15.113:0/3590514079 shutdown_connections 2024-04-22T07:21:20.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 -- 172.21.15.113:0/3590514079 wait complete. 2024-04-22T07:21:20.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 -- start start 2024-04-22T07:21:20.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.528+0000 7f0c70184740 1 Processor -- start 2024-04-22T07:21:20.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a64d0 0x564340a1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 0x564340a1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x564340a28c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681ac550 con 0x5643409a6e80 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681ad9c0 con 0x5643409afa00 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681acc20 con 0x5643409a64d0 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 10 client.0 _ll_register_callbacks cb 0x7f0c500052c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 0x564340a1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 0x564340a1e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57914/0 (socket says 172.21.15.113:57914) 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 -- 172.21.15.113:0/2570407889 learned_addr learned my addr 172.21.15.113:0/2570407889 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a64d0 0x564340a1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 -- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a64d0 msgr2=0x564340a1ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a64d0 0x564340a1ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 -- 172.21.15.113:0/2570407889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 msgr2=0x564340a28c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x564340a28c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 -- 172.21.15.113:0/2570407889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564340a291a0 con 0x5643409a6e80 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c66ffd640 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 0x564340a1e310 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800ac60 tx=0x7f0c58014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3e1ea61da62a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:20.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/2570407889 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58004220 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/2570407889 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f0c580043c0 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c500458f0 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/2570407889 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0c5802c050 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 10 client.17073 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x564340a2a4b0 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.529+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/2570407889 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0c500458f0 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/2570407889 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7f0c58004ca0 con 0x5643409a6e80 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 10 client.17073 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 20 client.17073 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564340a2ac60 0x564340a2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 4 client.17073 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564340918c10 con 0x564340a2ac60 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564340a2ac60 0x564340a2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c4e7fc640 10 client.17073 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c5ffff640 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564340a2ac60 0x564340a2d080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1271a7797d8dea31 server_cookie=f0e6934c200a4135 in_seq=0 out_seq=0 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c4e7fc640 1 -- 172.21.15.113:0/2570407889 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564340918c10 con 0x564340a2ac60 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c4e7fc640 10 client.17073 handle_command_reply: tid=0 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 client.17073 shutdown 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 msgr2=0x564340a1e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 0x564340a1e310 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800ac60 tx=0x7f0c58014900 comp rx=0 tx=0).stop 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 shutdown_connections 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564340a2ac60 0x564340a2d080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643409afa00 0x564340a28c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c70184740 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643409a6e80 0x564340a1e310 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.530+0000 7f0c4e7fc640 0 client.17073 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c70184740 1 --2- 172.21.15.113:0/2570407889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643409a64d0 0x564340a1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 >> 172.21.15.113:0/2570407889 conn(0x5643409a0ba0 msgr2=0x5643409a1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 shutdown_connections 2024-04-22T07:21:20.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c70184740 1 -- 172.21.15.113:0/2570407889 wait complete. 2024-04-22T07:21:20.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c70184740 20 client.17073 trim_cache size 0 max 16384 2024-04-22T07:21:20.535 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c540a3d20 msgr2=0x7f0c540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c540a3d20 0x7f0c540a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000efd0 tx=0x7f0c600054c0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 msgr2=0x7f0c68198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 0x7f0c68198ab0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000eaa0 tx=0x7f0c6000ef60 comp rx=0 tx=0).stop 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 shutdown_connections 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0c540a3d20 0x7f0c540a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c68198630 0x7f0c68198ab0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c68107270 0x7f0c6819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 --2- 172.21.15.113:0/3999222506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c6806a880 0x7f0c6819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 >> 172.21.15.113:0/3999222506 conn(0x7f0c68077eb0 msgr2=0x7f0c68100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 shutdown_connections 2024-04-22T07:21:20.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.531+0000 7f0c6d4ba640 1 -- 172.21.15.113:0/3999222506 wait complete. 2024-04-22T07:21:20.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:21:20.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 -- 172.21.15.113:0/3929623324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194101b40 msgr2=0x7fc19410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 --2- 172.21.15.113:0/3929623324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194101b40 0x7fc19410dc80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009cd0 tx=0x7fc18c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 -- 172.21.15.113:0/3929623324 shutdown_connections 2024-04-22T07:21:20.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 --2- 172.21.15.113:0/3929623324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc194101b40 0x7fc19410dc80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 --2- 172.21.15.113:0/3929623324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1941011a0 0x7fc194101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 --2- 172.21.15.113:0/3929623324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 0x7fc194107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 -- 172.21.15.113:0/3929623324 >> 172.21.15.113:0/3929623324 conn(0x7fc194076c50 msgr2=0x7fc194077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 -- 172.21.15.113:0/3929623324 shutdown_connections 2024-04-22T07:21:20.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.642+0000 7fc19a30e640 1 -- 172.21.15.113:0/3929623324 wait complete. 2024-04-22T07:21:20.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 Processor -- start 2024-04-22T07:21:20.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 -- start start 2024-04-22T07:21:20.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 0x7fc19419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc194101b40 0x7fc19419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 0x7fc194198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc194111260 con 0x7fc1941011a0 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941110e0 con 0x7fc194101b40 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc19a30e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941113e0 con 0x7fc194107030 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc193fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 0x7fc19419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc193fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 0x7fc19419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57928/0 (socket says 172.21.15.113:57928) 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.643+0000 7fc193fff640 1 -- 172.21.15.113:0/1126309872 learned_addr learned my addr 172.21.15.113:0/1126309872 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1988ab640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 0x7fc194198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1937fe640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc194101b40 0x7fc19419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc193fff640 1 -- 172.21.15.113:0/1126309872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 msgr2=0x7fc194198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc193fff640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 0x7fc194198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc193fff640 1 -- 172.21.15.113:0/1126309872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc194101b40 msgr2=0x7fc19419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc193fff640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc194101b40 0x7fc19419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc193fff640 1 -- 172.21.15.113:0/1126309872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc194198dc0 con 0x7fc1941011a0 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1988ab640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 0x7fc194198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1937fe640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc194101b40 0x7fc19419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc193fff640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 0x7fc19419e440 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d770 tx=0x7fc18400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1917fa640 1 -- 172.21.15.113:0/1126309872 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184017070 con 0x7fc1941011a0 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941990b0 con 0x7fc1941011a0 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1917fa640 1 -- 172.21.15.113:0/1126309872 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc1840054f0 con 0x7fc1941011a0 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc1917fa640 1 -- 172.21.15.113:0/1126309872 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184004de0 con 0x7fc1941011a0 2024-04-22T07:21:20.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.644+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a5300 con 0x7fc1941011a0 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.646+0000 7fc1917fa640 1 -- 172.21.15.113:0/1126309872 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fc184004f80 con 0x7fc1941011a0 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 Processor -- start 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 -- start start 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c50fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5107430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5109e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a8c50631f0 con 0x55a8c5107a00 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a8c50004b0 con 0x55a8c50fe4d0 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc19cfda740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a8c505f1b0 con 0x55a8c50fee80 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc1937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc1917fa640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc17c0a3d20 0x7fc17c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc1937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5109e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57940/0 (socket says 172.21.15.113:57940) 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc1937fe640 1 -- 172.21.15.113:0/814366823 learned_addr learned my addr 172.21.15.113:0/814366823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc193fff640 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.647+0000 7fc1988ab640 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c50fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1988ab640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc17c0a3d20 0x7fc17c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1937fe640 1 -- 172.21.15.113:0/814366823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 msgr2=0x55a8c5107430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1937fe640 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5107430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1917fa640 1 -- 172.21.15.113:0/1126309872 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fc1840d05c0 con 0x7fc1941011a0 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1937fe640 1 -- 172.21.15.113:0/814366823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 msgr2=0x55a8c50fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1937fe640 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c50fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1937fe640 1 -- 172.21.15.113:0/814366823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c4ff21b0 con 0x55a8c5107a00 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1937fe640 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5109e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc188009ca0 tx=0x7fc188010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85de3b926abb84b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc1988ab640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc17c0a3d20 0x7fc17c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc18c006010 tx=0x7fc18c0095c0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc17a7fc640 1 -- 172.21.15.113:0/814366823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188004030 con 0x55a8c5107a00 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc17a7fc640 1 -- 172.21.15.113:0/814366823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc1880041d0 con 0x55a8c5107a00 2024-04-22T07:21:20.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.648+0000 7fc17a7fc640 1 -- 172.21.15.113:0/814366823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188012740 con 0x55a8c5107a00 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 -- 172.21.15.113:0/814366823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 msgr2=0x55a8c5109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5109e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc188009ca0 tx=0x7fc188010740 comp rx=0 tx=0).stop 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 -- 172.21.15.113:0/814366823 shutdown_connections 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5109e50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 --2- 172.21.15.113:0/814366823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c50fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 -- 172.21.15.113:0/814366823 >> 172.21.15.113:0/814366823 conn(0x55a8c50f8ba0 msgr2=0x55a8c50f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 -- 172.21.15.113:0/814366823 shutdown_connections 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 -- 172.21.15.113:0/814366823 wait complete. 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 -- start start 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.649+0000 7fc19cfda740 1 Processor -- start 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c51072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5104670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5104bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a8c510a970 con 0x55a8c5107a00 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a8c501c500 con 0x55a8c50fe4d0 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a8c510aaf0 con 0x55a8c50fee80 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 10 client.0 _ll_register_callbacks cb 0x55a8c4feab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5104bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5104bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57948/0 (socket says 172.21.15.113:57948) 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1937fe640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5104670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 -- 172.21.15.113:0/3030195281 learned_addr learned my addr 172.21.15.113:0/3030195281 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc193fff640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c51072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 -- 172.21.15.113:0/3030195281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 msgr2=0x55a8c5104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5104670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 -- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 msgr2=0x55a8c51072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c51072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 -- 172.21.15.113:0/3030195281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c4d753f0 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1988ab640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5104bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005fe0 tx=0x7fc18c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a60b1671b7c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1737fe640 1 -- 172.21.15.113:0/3030195281 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c054060 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc1737fe640 1 -- 172.21.15.113:0/3030195281 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fc18c004040 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.650+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1880041d0 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc1737fe640 1 -- 172.21.15.113:0/3030195281 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc18c039070 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 10 client.17085 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a8c4e3f7d0 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc1737fe640 1 -- 172.21.15.113:0/3030195281 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc1880041d0 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc1737fe640 1 -- 172.21.15.113:0/3030195281 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7fc18c034850 con 0x55a8c5107a00 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 10 client.17085 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 20 client.17085 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c511a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 4 client.17085 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c5002930 con 0x55a8c5117bd0 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.651+0000 7fc193fff640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c511a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc193fff640 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c511a000 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd3d4b43d13f0115 server_cookie=ef8cb6f1404dfc84 in_seq=0 out_seq=0 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc1737fe640 10 client.17085 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc1737fe640 1 -- 172.21.15.113:0/3030195281 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc184081ae0 con 0x55a8c5117bd0 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc1737fe640 10 client.17085 handle_command_reply: tid=0 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 client.17085 shutdown 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 msgr2=0x55a8c5104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5104bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005fe0 tx=0x7fc18c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 shutdown_connections 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c511a000 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc1737fe640 0 client.17085 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5104bd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c5104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3030195281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c51072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.652+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 >> 172.21.15.113:0/3030195281 conn(0x55a8c50f8ba0 msgr2=0x55a8c50fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.653+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 shutdown_connections 2024-04-22T07:21:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.653+0000 7fc19cfda740 1 -- 172.21.15.113:0/3030195281 wait complete. 2024-04-22T07:21:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.653+0000 7fc19cfda740 20 client.17085 trim_cache size 0 max 16384 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 Processor -- start 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 -- start start 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c5113830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c50fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5106da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941b0890 con 0x55a8c5107a00 2024-04-22T07:21:20.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ae130 con 0x55a8c50fe4d0 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc19cfda740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941abfa0 con 0x55a8c50fee80 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57958/0 (socket says 172.21.15.113:57958) 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 -- 172.21.15.113:0/2748337589 learned_addr learned my addr 172.21.15.113:0/2748337589 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 -- 172.21.15.113:0/2748337589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 msgr2=0x55a8c50fa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c50fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc193fff640 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c5113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 -- 172.21.15.113:0/2748337589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 msgr2=0x55a8c5113830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c5113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 -- 172.21.15.113:0/2748337589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c5102bc0 con 0x55a8c5107a00 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.666+0000 7fc1988ab640 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5106da0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034c70 tx=0x7fc18c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faae50c9a9d457d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc179ffb640 1 -- 172.21.15.113:0/2748337589 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c054040 con 0x55a8c5107a00 2024-04-22T07:21:20.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc179ffb640 1 -- 172.21.15.113:0/2748337589 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc18c032660 con 0x55a8c5107a00 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc179ffb640 1 -- 172.21.15.113:0/2748337589 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004480 con 0x55a8c5107a00 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 -- 172.21.15.113:0/2748337589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 msgr2=0x55a8c5106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5106da0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034c70 tx=0x7fc18c009820 comp rx=0 tx=0).stop 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 -- 172.21.15.113:0/2748337589 shutdown_connections 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c5106da0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fee80 0x55a8c50fa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 --2- 172.21.15.113:0/2748337589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fe4d0 0x55a8c5113830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 -- 172.21.15.113:0/2748337589 >> 172.21.15.113:0/2748337589 conn(0x55a8c50f8ba0 msgr2=0x55a8c50f9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 -- 172.21.15.113:0/2748337589 shutdown_connections 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 -- 172.21.15.113:0/2748337589 wait complete. 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.667+0000 7fc19cfda740 1 -- start start 2024-04-22T07:21:20.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 Processor -- start 2024-04-22T07:21:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fe4d0 0x55a8c5110280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fee80 0x55a8c510b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c510bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ac5f0 con 0x55a8c5107a00 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941ada60 con 0x55a8c50fee80 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941accc0 con 0x55a8c50fe4d0 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc19cfda740 10 client.0 _ll_register_callbacks cb 0x7fc18c005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c510bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c510bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57964/0 (socket says 172.21.15.113:57964) 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 -- 172.21.15.113:0/3991294430 learned_addr learned my addr 172.21.15.113:0/3991294430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1937fe640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fee80 0x55a8c510b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc193fff640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fe4d0 0x55a8c5110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 -- 172.21.15.113:0/3991294430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fe4d0 msgr2=0x55a8c5110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fe4d0 0x55a8c5110280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 -- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fee80 msgr2=0x55a8c510b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fee80 0x55a8c510b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 -- 172.21.15.113:0/3991294430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c517d140 con 0x55a8c5107a00 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1988ab640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c510bd30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034a70 tx=0x7fc18c034aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8202989fe8dc0d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:20.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.668+0000 7fc1737fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc1737fe640 1 -- 172.21.15.113:0/3991294430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c054070 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc1737fe640 1 -- 172.21.15.113:0/3991294430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fc18c0043d0 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18c032660 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc1737fe640 1 -- 172.21.15.113:0/3991294430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc18c0045d0 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 10 client.17091 fetch_fsmap learned FSMap version 51 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a8c517e2e0 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc1737fe640 1 -- 172.21.15.113:0/3991294430 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc18c032660 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc1737fe640 1 -- 172.21.15.113:0/3991294430 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 51) ==== 8342+0+0 (secure 0 0 0) 0x7fc18c04fbe0 con 0x55a8c5107a00 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 10 client.17091 fetch_fsmap finished waiting for FSMap version 51 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 10 client.17091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 20 client.17091 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c51808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 4 client.17091 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8c5070c10 con 0x55a8c5117bd0 2024-04-22T07:21:20.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.669+0000 7fc193fff640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c51808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc193fff640 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c51808a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe44dbb6bc5bca7 server_cookie=886b7cd0eff35ee4 in_seq=0 out_seq=0 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc1737fe640 10 client.17091 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc1737fe640 1 -- 172.21.15.113:0/3991294430 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8c5070c10 con 0x55a8c5117bd0 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc1737fe640 10 client.17091 handle_command_reply: tid=0 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 client.17091 shutdown 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 msgr2=0x55a8c510bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c510bd30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034a70 tx=0x7fc18c034aa0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 shutdown_connections 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55a8c5117bd0 0x55a8c51808a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a8c5107a00 0x55a8c510bd30 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c50fee80 0x55a8c510b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc1737fe640 0 client.17091 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:20.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 --2- 172.21.15.113:0/3991294430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a8c50fe4d0 0x55a8c5110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 >> 172.21.15.113:0/3991294430 conn(0x55a8c50f8ba0 msgr2=0x55a8c50f7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 shutdown_connections 2024-04-22T07:21:20.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 1 -- 172.21.15.113:0/3991294430 wait complete. 2024-04-22T07:21:20.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.670+0000 7fc19cfda740 20 client.17091 trim_cache size 0 max 16384 2024-04-22T07:21:20.675 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc17c0a3d20 msgr2=0x7fc17c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc17c0a3d20 0x7fc17c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc18c006010 tx=0x7fc18c0095c0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 msgr2=0x7fc19419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 0x7fc19419e440 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d770 tx=0x7fc18400dc30 comp rx=0 tx=0).stop 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 shutdown_connections 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc17c0a3d20 0x7fc17c0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc194107030 0x7fc194198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc194101b40 0x7fc19419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 --2- 172.21.15.113:0/1126309872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1941011a0 0x7fc19419e440 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:20.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 >> 172.21.15.113:0/1126309872 conn(0x7fc194076c50 msgr2=0x7fc1940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:20.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 shutdown_connections 2024-04-22T07:21:20.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:20.671+0000 7fc19a30e640 1 -- 172.21.15.113:0/1126309872 wait complete. 2024-04-22T07:21:20.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:20 smithi133 ceph-mon[31198]: pgmap v204: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:21:20.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:20 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4039566824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:21:20.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:20 smithi154 ceph-mon[31862]: pgmap v204: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:21:20.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:20 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4039566824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:21:20.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:20 smithi113 ceph-mon[27933]: pgmap v204: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:21:20.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:20 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4039566824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:21:21.679 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:21:21.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:21:21.680 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/3614828280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e41072b0 msgr2=0x7fd3e410eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/3614828280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e41072b0 0x7fd3e410eec0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009ca0 tx=0x7fd3dc02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/3614828280 shutdown_connections 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/3614828280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e41072b0 0x7fd3e410eec0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/3614828280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3e41014a0 0x7fd3e4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/3614828280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4100af0 0x7fd3e4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/3614828280 >> 172.21.15.113:0/3614828280 conn(0x7fd3e40760e0 msgr2=0x7fd3e40764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/3614828280 shutdown_connections 2024-04-22T07:21:21.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.797+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/3614828280 wait complete. 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 Processor -- start 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 -- start start 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 0x7fd3e419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e41072b0 0x7fd3e419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3e4198690 0x7fd3e4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4112390 con 0x7fd3e4100af0 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4112210 con 0x7fd3e41072b0 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e8f0e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4112510 con 0x7fd3e4198690 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 0x7fd3e419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 0x7fd3e419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57986/0 (socket says 172.21.15.113:57986) 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3e259c640 1 -- 172.21.15.113:0/2505461653 learned_addr learned my addr 172.21.15.113:0/2505461653 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.798+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e41072b0 0x7fd3e419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e259c640 1 -- 172.21.15.113:0/2505461653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3e4198690 msgr2=0x7fd3e4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3e4198690 0x7fd3e4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e259c640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3e4198690 0x7fd3e4198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e259c640 1 -- 172.21.15.113:0/2505461653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e41072b0 msgr2=0x7fd3e419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e259c640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e41072b0 0x7fd3e419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e259c640 1 -- 172.21.15.113:0/2505461653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e4199290 con 0x7fd3e4100af0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e259c640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 0x7fd3e419e5d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400ea60 tx=0x7fd3d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3db7fe640 1 -- 172.21.15.113:0/2505461653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400cde0 con 0x7fd3e4100af0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e41a5370 con 0x7fd3e4100af0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3db7fe640 1 -- 172.21.15.113:0/2505461653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd3d40035f0 con 0x7fd3e4100af0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3db7fe640 1 -- 172.21.15.113:0/2505461653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4010690 con 0x7fd3e4100af0 2024-04-22T07:21:21.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.799+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e41a5770 con 0x7fd3e4100af0 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.801+0000 7fd3db7fe640 1 -- 172.21.15.113:0/2505461653 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd3d40037a0 con 0x7fd3e4100af0 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 Processor -- start 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 -- start start 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3db7fe640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd3d00a3d20 0x7fd3d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c54366cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c54372260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559c542caf00 con 0x559c543668e0 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559c54268590 con 0x559c54367290 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559c542c6ec0 con 0x559c5436fe10 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd3d00a3d20 0x7fd3d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3db7fe640 1 -- 172.21.15.113:0/2505461653 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fd3d40d0d20 con 0x7fd3e4100af0 2024-04-22T07:21:21.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c54366cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c54366cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58002/0 (socket says 172.21.15.113:58002) 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 -- 172.21.15.113:0/3579966722 learned_addr learned my addr 172.21.15.113:0/3579966722 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 -- 172.21.15.113:0/3579966722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 msgr2=0x559c54372260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c54372260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 -- 172.21.15.113:0/3579966722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 msgr2=0x559c5436f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.802+0000 7fd3e259c640 1 -- 172.21.15.113:0/3579966722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c540a57e0 con 0x559c543668e0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd3d00a3d20 0x7fd3d00a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc032540 tx=0x7fd3dc009800 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3e259c640 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c54366cc0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd3d409cab0 tx=0x7fd3d4098e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bda13dbe53518e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3d8ff9640 1 -- 172.21.15.113:0/3579966722 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d409d220 con 0x559c543668e0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3d8ff9640 1 -- 172.21.15.113:0/3579966722 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd3d409de50 con 0x559c543668e0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3d8ff9640 1 -- 172.21.15.113:0/3579966722 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40a5790 con 0x559c543668e0 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3579966722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 msgr2=0x559c54366cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c54366cc0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd3d409cab0 tx=0x7fd3d4098e80 comp rx=0 tx=0).stop 2024-04-22T07:21:21.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3579966722 shutdown_connections 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c54372260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3579966722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c54366cc0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.803+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3579966722 >> 172.21.15.113:0/3579966722 conn(0x559c54360fb0 msgr2=0x559c54361390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3579966722 shutdown_connections 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3579966722 wait complete. 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 -- start start 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 Processor -- start 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c5436d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559c54372d80 con 0x559c543668e0 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559c54284580 con 0x559c54367290 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559c54372f00 con 0x559c5436fe10 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3ebbdb740 10 client.0 _ll_register_callbacks cb 0x559c5428a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:21.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58006/0 (socket says 172.21.15.113:58006) 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3e259c640 1 -- 172.21.15.113:0/2252061566 learned_addr learned my addr 172.21.15.113:0/2252061566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3e259c640 1 -- 172.21.15.113:0/2252061566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 msgr2=0x559c5436d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c5436d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.804+0000 7fd3e259c640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c5436d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3e259c640 1 -- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 msgr2=0x559c5436d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3e259c640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3e259c640 1 -- 172.21.15.113:0/2252061566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c54105f70 con 0x559c543668e0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c5436d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3e259c640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436ce70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4003f60 tx=0x7fd3d40a0ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa0550fafe33c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/2252061566 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d409cb20 con 0x559c543668e0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/2252061566 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fd3d40ce550 con 0x559c543668e0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d409de50 con 0x559c543668e0 2024-04-22T07:21:21.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/2252061566 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd3d409d220 con 0x559c543668e0 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 10 client.17103 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559c53f9d170 con 0x559c543668e0 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/2252061566 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd3d409de50 con 0x559c543668e0 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/2252061566 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fd3d40bbce0 con 0x559c543668e0 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 10 client.17103 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 20 client.17103 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c54380290 0x559c54382680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 4 client.17103 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.805+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c542d8920 con 0x559c54380290 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.806+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c54380290 0x559c54382680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.806+0000 7fd3c9ffb640 10 client.17103 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.806+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c54380290 0x559c54382680 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fde92b63b6b459a server_cookie=2812c9c0b2ac7f8c in_seq=0 out_seq=0 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.806+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/2252061566 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c542d8920 con 0x559c54380290 2024-04-22T07:21:21.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.806+0000 7fd3c9ffb640 10 client.17103 handle_command_reply: tid=0 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 client.17103 shutdown 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 msgr2=0x559c5436ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436ce70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4003f60 tx=0x7fd3d40a0ed0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 shutdown_connections 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c54380290 0x559c54382680 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c5436d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c5436d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/2252061566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436ce70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 >> 172.21.15.113:0/2252061566 conn(0x559c54360fb0 msgr2=0x559c54362d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3c9ffb640 0 client.17103 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 shutdown_connections 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/2252061566 wait complete. 2024-04-22T07:21:21.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.807+0000 7fd3ebbdb740 20 client.17103 trim_cache size 0 max 16384 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 Processor -- start 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 -- start start 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c54367290 0x559c5436c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c5436fe10 0x559c5437b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41b08f0 con 0x559c543668e0 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.824+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41ae1c0 con 0x559c5436fe10 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41ac030 con 0x559c54367290 2024-04-22T07:21:21.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58018/0 (socket says 172.21.15.113:58018) 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 -- 172.21.15.113:0/1340948642 learned_addr learned my addr 172.21.15.113:0/1340948642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c5436fe10 0x559c5437b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c54367290 0x559c5436c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 -- 172.21.15.113:0/1340948642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c54367290 msgr2=0x559c5436c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c54367290 0x559c5436c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 -- 172.21.15.113:0/1340948642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c5436fe10 msgr2=0x559c5437b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c5436fe10 0x559c5437b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 -- 172.21.15.113:0/1340948642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c54365c40 con 0x559c543668e0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3e259c640 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436f5a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005820 tx=0x7fd3d4098e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b66c1bc21b805 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3cbfff640 1 -- 172.21.15.113:0/1340948642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d409c660 con 0x559c543668e0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3cbfff640 1 -- 172.21.15.113:0/1340948642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd3d409de50 con 0x559c543668e0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.825+0000 7fd3cbfff640 1 -- 172.21.15.113:0/1340948642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4004000 con 0x559c543668e0 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/1340948642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 msgr2=0x559c5436f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436f5a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005820 tx=0x7fd3d4098e80 comp rx=0 tx=0).stop 2024-04-22T07:21:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/1340948642 shutdown_connections 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c5436fe10 0x559c5437b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c54367290 0x559c5436c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/1340948642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c5436f5a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/1340948642 >> 172.21.15.113:0/1340948642 conn(0x559c54360fb0 msgr2=0x559c5437b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/1340948642 shutdown_connections 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/1340948642 wait complete. 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- start start 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 Processor -- start 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c543825a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c54382ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c543de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41ac680 con 0x559c543668e0 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41adaf0 con 0x559c54367290 2024-04-22T07:21:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41acd50 con 0x559c5436fe10 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.826+0000 7fd3ebbdb740 10 client.0 _ll_register_callbacks cb 0x7fd3cc00a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c543825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c543825a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58034/0 (socket says 172.21.15.113:58034) 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 -- 172.21.15.113:0/3999759336 learned_addr learned my addr 172.21.15.113:0/3999759336 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c54382ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c543de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 -- 172.21.15.113:0/3999759336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 msgr2=0x559c543de430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c543de430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 -- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 msgr2=0x559c54382ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c54382ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 -- 172.21.15.113:0/3999759336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c543dea20 con 0x559c543668e0 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c54382ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e2d9d640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c543de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3e259c640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c543825a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4004610 tx=0x7fd3d400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a880e48daa39336f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/3999759336 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d407f070 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/3999759336 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fd3d4004000 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d409de50 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/3999759336 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd3d409c660 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3ebbdb740 10 client.17109 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559c543dea20 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.827+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/3999759336 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd3d409de50 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/3999759336 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fd3d4082030 con 0x559c543668e0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3ebbdb740 10 client.17109 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3ebbdb740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3ebbdb740 20 client.17109 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c543e6030 0x559c543e8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3ebbdb740 4 client.17109 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c542d2790 con 0x559c543e6030 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c543e6030 0x559c543e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3dbfff640 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c543e6030 0x559c543e8420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57fa470f00b6982d server_cookie=3f151f4e8b213dd8 in_seq=0 out_seq=0 2024-04-22T07:21:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.828+0000 7fd3c9ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3c9ffb640 1 -- 172.21.15.113:0/3999759336 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c542d2790 con 0x559c543e6030 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3c9ffb640 10 client.17109 handle_command_reply: tid=0 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 client.17109 shutdown 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 msgr2=0x559c543825a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c543825a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4004610 tx=0x7fd3d400ef90 comp rx=0 tx=0).stop 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 shutdown_connections 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x559c543e6030 0x559c543e8420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559c5436fe10 0x559c543de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c54367290 0x559c54382ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 --2- 172.21.15.113:0/3999759336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559c543668e0 0x559c543825a0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3c9ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 >> 172.21.15.113:0/3999759336 conn(0x559c54360fb0 msgr2=0x559c54360010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 shutdown_connections 2024-04-22T07:21:21.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 1 -- 172.21.15.113:0/3999759336 wait complete. 2024-04-22T07:21:21.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.829+0000 7fd3ebbdb740 20 client.17109 trim_cache size 0 max 16384 2024-04-22T07:21:21.834 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:21:21.834 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:21:21.834 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "d35fce20-c1fd-4a97-a908-e2ad68bc6de6", 2024-04-22T07:21:21.834 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:21:21.834 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd3d00a3d20 msgr2=0x7fd3d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd3d00a3d20 0x7fd3d00a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc032540 tx=0x7fd3dc009800 comp rx=0 tx=0).stop 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 msgr2=0x7fd3e419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 0x7fd3e419e5d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400ea60 tx=0x7fd3d400ef20 comp rx=0 tx=0).stop 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 shutdown_connections 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd3d00a3d20 0x7fd3d00a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3e4198690 0x7fd3e4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e41072b0 0x7fd3e419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 --2- 172.21.15.113:0/2505461653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3e4100af0 0x7fd3e419e5d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 >> 172.21.15.113:0/2505461653 conn(0x7fd3e40760e0 msgr2=0x7fd3e40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 shutdown_connections 2024-04-22T07:21:21.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.830+0000 7fd3e8f0e640 1 -- 172.21.15.113:0/2505461653 wait complete. 2024-04-22T07:21:21.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:21:21.839 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:21:21.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 -- 172.21.15.113:0/1088738803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 msgr2=0x7f38d8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 --2- 172.21.15.113:0/1088738803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d8101120 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009900 tx=0x7f38c802bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 -- 172.21.15.113:0/1088738803 shutdown_connections 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 --2- 172.21.15.113:0/1088738803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 0x7f38d8110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 --2- 172.21.15.113:0/1088738803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d8101660 0x7f38d810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 --2- 172.21.15.113:0/1088738803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d8101120 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 -- 172.21.15.113:0/1088738803 >> 172.21.15.113:0/1088738803 conn(0x7f38d8078000 msgr2=0x7f38d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 -- 172.21.15.113:0/1088738803 shutdown_connections 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.941+0000 7f38de775640 1 -- 172.21.15.113:0/1088738803 wait complete. 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 Processor -- start 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 -- start start 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d8101660 0x7f38d81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 0x7f38d819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8114570 con 0x7f38d8100d40 2024-04-22T07:21:21.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81143f0 con 0x7f38d8101660 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38de775640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81146f0 con 0x7f38d810e8f0 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58048/0 (socket says 172.21.15.113:58048) 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38d7fff640 1 -- 172.21.15.113:0/4035123088 learned_addr learned my addr 172.21.15.113:0/4035123088 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38d77fe640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d8101660 0x7f38d81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.942+0000 7f38dcd12640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 0x7f38d819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d7fff640 1 -- 172.21.15.113:0/4035123088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 msgr2=0x7f38d819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d7fff640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 0x7f38d819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d7fff640 1 -- 172.21.15.113:0/4035123088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d8101660 msgr2=0x7f38d81a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d7fff640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d8101660 0x7f38d81a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d7fff640 1 -- 172.21.15.113:0/4035123088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d819b020 con 0x7f38d8100d40 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38dcd12640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 0x7f38d819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d7fff640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d81a06e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005940 tx=0x7f38c802cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:21.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d57fa640 1 -- 172.21.15.113:0/4035123088 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8043020 con 0x7f38d8100d40 2024-04-22T07:21:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d57fa640 1 -- 172.21.15.113:0/4035123088 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f38c8004250 con 0x7f38d8100d40 2024-04-22T07:21:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d819b2b0 con 0x7f38d8100d40 2024-04-22T07:21:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38d57fa640 1 -- 172.21.15.113:0/4035123088 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003a60 con 0x7f38d8100d40 2024-04-22T07:21:21.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.943+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d81a7530 con 0x7f38d8100d40 2024-04-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38d57fa640 1 -- 172.21.15.113:0/4035123088 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f38c8046020 con 0x7f38d8100d40 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 Processor -- start 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 -- start start 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38d57fa640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f38c00a3d20 0x7f38c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc936960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc941e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cfdc89b1e0 con 0x55cfdc936580 2024-04-22T07:21:22.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cfdc838390 con 0x55cfdc936ea0 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.945+0000 7f38e1364740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cfdc8971a0 con 0x55cfdc93fa80 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc936960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38dcd12640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f38c00a3d20 0x7f38c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc936960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58060/0 (socket says 172.21.15.113:58060) 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 -- 172.21.15.113:0/53404091 learned_addr learned my addr 172.21.15.113:0/53404091 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d77fe640 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d57fa640 1 -- 172.21.15.113:0/4035123088 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f38c80f2480 con 0x7f38d8100d40 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38dcd12640 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc941e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 -- 172.21.15.113:0/53404091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 msgr2=0x55cfdc941e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc941e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 -- 172.21.15.113:0/53404091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 msgr2=0x55cfdc93f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 -- 172.21.15.113:0/53404091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfdc6777d0 con 0x55cfdc936580 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d77fe640 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38dcd12640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f38c00a3d20 0x7f38c00a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38d806b6f0 tx=0x7f38d0013a90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38d7fff640 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc936960 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f38c8000c00 tx=0x7f38c802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c525e8f7e32f58c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38be7fc640 1 -- 172.21.15.113:0/53404091 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80d1630 con 0x55cfdc936580 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38be7fc640 1 -- 172.21.15.113:0/53404091 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f38c80bea90 con 0x55cfdc936580 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.946+0000 7f38be7fc640 1 -- 172.21.15.113:0/53404091 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80d1450 con 0x55cfdc936580 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 -- 172.21.15.113:0/53404091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 msgr2=0x55cfdc936960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc936960 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f38c8000c00 tx=0x7f38c802bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 -- 172.21.15.113:0/53404091 shutdown_connections 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc941e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 --2- 172.21.15.113:0/53404091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc936960 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 -- 172.21.15.113:0/53404091 >> 172.21.15.113:0/53404091 conn(0x55cfdc930cb0 msgr2=0x55cfdc931090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 -- 172.21.15.113:0/53404091 shutdown_connections 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 -- 172.21.15.113:0/53404091 wait complete. 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 -- start start 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.947+0000 7f38e1364740 1 Processor -- start 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cfdc89b1e0 con 0x55cfdc936580 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cfdc8542f0 con 0x55cfdc936ea0 2024-04-22T07:21:22.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cfdc942630 con 0x55cfdc93fa80 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 10 client.0 _ll_register_callbacks cb 0x55cfdc861580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58070/0 (socket says 172.21.15.113:58070) 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 -- 172.21.15.113:0/3358060321 learned_addr learned my addr 172.21.15.113:0/3358060321 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d77fe640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38dcd12640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 -- 172.21.15.113:0/3358060321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 msgr2=0x55cfdc93d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 -- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 msgr2=0x55cfdc93cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 -- 172.21.15.113:0/3358060321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfdc5ad3f0 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38d7fff640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93ca80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f38c80be320 tx=0x7f38c80bac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ae3fa8c6f6501 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38b77fe640 1 -- 172.21.15.113:0/3358060321 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80be390 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38b77fe640 1 -- 172.21.15.113:0/3358060321 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f38c80f2280 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.948+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c80bea90 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38b77fe640 1 -- 172.21.15.113:0/3358060321 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f38c80c2740 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 10 client.17121 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cfdc56d0f0 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38b77fe640 1 -- 172.21.15.113:0/3358060321 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f38c80bea90 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38b77fe640 1 -- 172.21.15.113:0/3358060321 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f38c80c6b70 con 0x55cfdc936580 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 10 client.17121 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:22.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 20 client.17121 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc94f470 0x55cfdc951860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 4 client.17121 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfdc8a8c00 con 0x55cfdc94f470 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38d77fe640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc94f470 0x55cfdc951860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38d77fe640 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc94f470 0x55cfdc951860 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4874c8d9348a6aa5 server_cookie=bbcba7a8ffacbfac in_seq=0 out_seq=0 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.949+0000 7f38b77fe640 10 client.17121 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38b77fe640 1 -- 172.21.15.113:0/3358060321 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfdc8a8c00 con 0x55cfdc94f470 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38b77fe640 10 client.17121 handle_command_reply: tid=0 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 client.17121 shutdown 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 msgr2=0x55cfdc93ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93ca80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f38c80be320 tx=0x7f38c80bac10 comp rx=0 tx=0).stop 2024-04-22T07:21:22.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 shutdown_connections 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc94f470 0x55cfdc951860 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc93cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38b77fe640 0 client.17121 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 --2- 172.21.15.113:0/3358060321 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93ca80 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 >> 172.21.15.113:0/3358060321 conn(0x55cfdc930cb0 msgr2=0x55cfdc932a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 shutdown_connections 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 1 -- 172.21.15.113:0/3358060321 wait complete. 2024-04-22T07:21:22.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.950+0000 7f38e1364740 20 client.17121 trim_cache size 0 max 16384 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.967+0000 7f38e1364740 1 Processor -- start 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 -- start start 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc94b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81b2a00 con 0x55cfdc936580 2024-04-22T07:21:22.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81b02d0 con 0x55cfdc936ea0 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38e1364740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ae140 con 0x55cfdc93fa80 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58074/0 (socket says 172.21.15.113:58074) 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 -- 172.21.15.113:0/3147098362 learned_addr learned my addr 172.21.15.113:0/3147098362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d77fe640 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc94b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 -- 172.21.15.113:0/3147098362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 msgr2=0x55cfdc93f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 -- 172.21.15.113:0/3147098362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 msgr2=0x55cfdc94b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc94b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 -- 172.21.15.113:0/3147098362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfdc941e70 con 0x55cfdc936580 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38d7fff640 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93b9c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f38c80efbf0 tx=0x7f38c80d1810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b249524310dbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38bdffb640 1 -- 172.21.15.113:0/3147098362 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80bea90 con 0x55cfdc936580 2024-04-22T07:21:22.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38bdffb640 1 -- 172.21.15.113:0/3147098362 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f38c80f1030 con 0x55cfdc936580 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.968+0000 7f38bdffb640 1 -- 172.21.15.113:0/3147098362 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80bc430 con 0x55cfdc936580 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 -- 172.21.15.113:0/3147098362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 msgr2=0x55cfdc93b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93b9c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f38c80efbf0 tx=0x7f38c80d1810 comp rx=0 tx=0).stop 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 -- 172.21.15.113:0/3147098362 shutdown_connections 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc93fa80 0x55cfdc93f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc94b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 --2- 172.21.15.113:0/3147098362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc936580 0x55cfdc93b9c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 -- 172.21.15.113:0/3147098362 >> 172.21.15.113:0/3147098362 conn(0x55cfdc930cb0 msgr2=0x55cfdc92fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 -- 172.21.15.113:0/3147098362 shutdown_connections 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 -- 172.21.15.113:0/3147098362 wait complete. 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 -- start start 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.969+0000 7f38e1364740 1 Processor -- start 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc936580 0x55cfdc9435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc943b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 0x55cfdc9b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ae790 con 0x55cfdc93fa80 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81afc00 con 0x55cfdc936ea0 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81aee60 con 0x55cfdc936580 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 10 client.0 _ll_register_callbacks cb 0x7f38cc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 0x55cfdc9b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 0x55cfdc9b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58090/0 (socket says 172.21.15.113:58090) 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 -- 172.21.15.113:0/2668072537 learned_addr learned my addr 172.21.15.113:0/2668072537 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38d77fe640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc943b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38d7fff640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc936580 0x55cfdc9435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 -- 172.21.15.113:0/2668072537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc936580 msgr2=0x55cfdc9435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc936580 0x55cfdc9435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 -- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 msgr2=0x55cfdc943b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc943b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 -- 172.21.15.113:0/2668072537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfdc9b4d50 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38dcd12640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 0x55cfdc9b4810 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f38d0014fd0 tx=0x7f38d0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb9814b1ad0c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38b77fe640 1 -- 172.21.15.113:0/2668072537 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0025070 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38b77fe640 1 -- 172.21.15.113:0/2668072537 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f38d001f470 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c80f1030 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38b77fe640 1 -- 172.21.15.113:0/2668072537 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f38d0031050 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 10 client.17127 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.970+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cfdc9b5ec0 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38b77fe640 1 -- 172.21.15.113:0/2668072537 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f38c80f1030 con 0x55cfdc93fa80 2024-04-22T07:21:22.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38b77fe640 1 -- 172.21.15.113:0/2668072537 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f38d001d040 con 0x55cfdc93fa80 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 10 client.17127 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 20 client.17127 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc9b6640 0x55cfdc9b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 4 client.17127 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfdc8a2a70 con 0x55cfdc9b6640 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38d7fff640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc9b6640 0x55cfdc9b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38b77fe640 10 client.17127 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:22.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38d7fff640 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc9b6640 0x55cfdc9b8a60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31603dab05610a60 server_cookie=eda96292606b3bd in_seq=0 out_seq=0 2024-04-22T07:21:22.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38b77fe640 1 -- 172.21.15.113:0/2668072537 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cfdc8a2a70 con 0x55cfdc9b6640 2024-04-22T07:21:22.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38b77fe640 10 client.17127 handle_command_reply: tid=0 2024-04-22T07:21:22.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.971+0000 7f38e1364740 1 client.17127 shutdown 2024-04-22T07:21:22.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 msgr2=0x55cfdc9b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:22.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 0x55cfdc9b4810 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f38d0014fd0 tx=0x7f38d0004b50 comp rx=0 tx=0).stop 2024-04-22T07:21:22.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 shutdown_connections 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cfdc9b6640 0x55cfdc9b8a60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cfdc93fa80 0x55cfdc9b4810 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfdc936ea0 0x55cfdc943b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38b77fe640 0 client.17127 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 --2- 172.21.15.113:0/2668072537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cfdc936580 0x55cfdc9435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 >> 172.21.15.113:0/2668072537 conn(0x55cfdc930cb0 msgr2=0x55cfdc931fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 shutdown_connections 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 1 -- 172.21.15.113:0/2668072537 wait complete. 2024-04-22T07:21:22.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38e1364740 20 client.17127 trim_cache size 0 max 16384 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: "d35fce20-c1fd-4a97-a908-e2ad68bc6de6": { 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "d35fce20-c1fd-4a97-a908-e2ad68bc6de6", 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:21:22.071 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:21:22.072 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f38c00a3d20 msgr2=0x7f38c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f38c00a3d20 0x7f38c00a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38d806b6f0 tx=0x7f38d0013a90 comp rx=0 tx=0).stop 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 msgr2=0x7f38d81a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d81a06e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005940 tx=0x7f38c802cc40 comp rx=0 tx=0).stop 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 shutdown_connections 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f38c00a3d20 0x7f38c00a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38d810e8f0 0x7f38d819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d8101660 0x7f38d81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 --2- 172.21.15.113:0/4035123088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d8100d40 0x7f38d81a06e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:22.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 >> 172.21.15.113:0/4035123088 conn(0x7f38d8078000 msgr2=0x7f38d810fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:22.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 shutdown_connections 2024-04-22T07:21:22.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:21.972+0000 7f38de775640 1 -- 172.21.15.113:0/4035123088 wait complete. 2024-04-22T07:21:22.074 INFO:tasks.cephfs.filesystem:scrub status for tag:d35fce20-c1fd-4a97-a908-e2ad68bc6de6 - {'path': '/', 'tag': 'd35fce20-c1fd-4a97-a908-e2ad68bc6de6', 'options': 'recursive,force'} 2024-04-22T07:21:22.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:22 smithi133 ceph-mon[31198]: pgmap v205: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 7 op/s 2024-04-22T07:21:22.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:22 smithi133 ceph-mon[31198]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:22.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:22 smithi154 ceph-mon[31862]: pgmap v205: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 7 op/s 2024-04-22T07:21:22.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:22 smithi154 ceph-mon[31862]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:22 smithi113 ceph-mon[27933]: pgmap v205: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 7 op/s 2024-04-22T07:21:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:22 smithi113 ceph-mon[27933]: fsmap cephfs:3 {0=e=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-22T07:21:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:23 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:21:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:23 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:23 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:21:23.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:23 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:24.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:23 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:21:24.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:23 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:24.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:23 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:21:24.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:23 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:24.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:23 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:21:24.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:23 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:24.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:23 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:21:24.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:23 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:24.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:24 smithi113 ceph-mon[27933]: pgmap v206: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:25.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:24 smithi133 ceph-mon[31198]: pgmap v206: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:25.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:24 smithi154 ceph-mon[31862]: pgmap v206: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:21:26.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:26 smithi113 ceph-mon[27933]: pgmap v207: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:21:26.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:26 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:21:27.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:26 smithi133 ceph-mon[31198]: pgmap v207: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:21:27.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:26 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:21:27.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:26 smithi154 ceph-mon[31862]: pgmap v207: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:21:27.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:26 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:21:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:28 smithi113 ceph-mon[27933]: pgmap v208: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:21:29.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:28 smithi133 ceph-mon[31198]: pgmap v208: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:21:29.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:28 smithi154 ceph-mon[31862]: pgmap v208: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:21:30.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:30 smithi113 ceph-mon[27933]: pgmap v209: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:21:31.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:30 smithi133 ceph-mon[31198]: pgmap v209: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:21:31.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:30 smithi154 ceph-mon[31862]: pgmap v209: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:21:32.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:32 smithi113 ceph-mon[27933]: pgmap v210: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:21:33.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:32 smithi133 ceph-mon[31198]: pgmap v210: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:21:33.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:32 smithi154 ceph-mon[31862]: pgmap v210: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:21:35.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:34 smithi133 ceph-mon[31198]: pgmap v211: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-22T07:21:35.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:34 smithi154 ceph-mon[31862]: pgmap v211: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-22T07:21:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:34 smithi113 ceph-mon[27933]: pgmap v211: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-22T07:21:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:36 smithi133 ceph-mon[31198]: pgmap v212: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-22T07:21:37.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:36 smithi154 ceph-mon[31862]: pgmap v212: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-22T07:21:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:36 smithi113 ceph-mon[27933]: pgmap v212: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-22T07:21:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:38 smithi133 ceph-mon[31198]: pgmap v213: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:39.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:38 smithi154 ceph-mon[31862]: pgmap v213: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:39.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:38 smithi113 ceph-mon[27933]: pgmap v213: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:41.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:40 smithi133 ceph-mon[31198]: pgmap v214: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:41.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:40 smithi154 ceph-mon[31862]: pgmap v214: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:41.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:40 smithi113 ceph-mon[27933]: pgmap v214: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:42.866 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:42 smithi133 ceph-mon[31198]: pgmap v215: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:42.868 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:42 smithi154 ceph-mon[31862]: pgmap v215: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:43.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:42 smithi113 ceph-mon[27933]: pgmap v215: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:45.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:44 smithi133 ceph-mon[31198]: pgmap v216: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:45.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:44 smithi154 ceph-mon[31862]: pgmap v216: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:45.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:44 smithi113 ceph-mon[27933]: pgmap v216: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:46 smithi133 ceph-mon[31198]: pgmap v217: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:47.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:46 smithi154 ceph-mon[31862]: pgmap v217: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:46 smithi113 ceph-mon[27933]: pgmap v217: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:48 smithi133 ceph-mon[31198]: pgmap v218: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:49.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:48 smithi154 ceph-mon[31862]: pgmap v218: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:49.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:48 smithi113 ceph-mon[27933]: pgmap v218: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:50 smithi133 ceph-mon[31198]: pgmap v219: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:51.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:50 smithi154 ceph-mon[31862]: pgmap v219: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:51.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:50 smithi113 ceph-mon[27933]: pgmap v219: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:52.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:21:52.075 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:21:52.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 -- 172.21.15.113:0/2974111803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 msgr2=0x7ff7b010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 --2- 172.21.15.113:0/2974111803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b010dd00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009ca0 tx=0x7ff7a802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 -- 172.21.15.113:0/2974111803 shutdown_connections 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 --2- 172.21.15.113:0/2974111803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b010dd00 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 --2- 172.21.15.113:0/2974111803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101a10 0x7ff7b0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 --2- 172.21.15.113:0/2974111803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b0101060 0x7ff7b0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 -- 172.21.15.113:0/2974111803 >> 172.21.15.113:0/2974111803 conn(0x7ff7b00771d0 msgr2=0x7ff7b00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 -- 172.21.15.113:0/2974111803 shutdown_connections 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.189+0000 7ff7b6031640 1 -- 172.21.15.113:0/2974111803 wait complete. 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 Processor -- start 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 -- start start 2024-04-22T07:21:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101060 0x7ff7b019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b01985e0 0x7ff7b0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01111d0 con 0x7ff7b0106130 2024-04-22T07:21:52.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0111050 con 0x7ff7b0101060 2024-04-22T07:21:52.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7b6031640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0111350 con 0x7ff7b01985e0 2024-04-22T07:21:52.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7aeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7aeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48752/0 (socket says 172.21.15.113:48752) 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3947520864 learned_addr learned my addr 172.21.15.113:0/3947520864 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.190+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101060 0x7ff7b019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7affff640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b01985e0 0x7ff7b0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3947520864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b01985e0 msgr2=0x7ff7b0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b01985e0 0x7ff7b0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3947520864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101060 msgr2=0x7ff7b019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101060 0x7ff7b019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3947520864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b0199100 con 0x7ff7b0106130 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101060 0x7ff7b019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b019e970 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400e9c0 tx=0x7ff7a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7acff9640 1 -- 172.21.15.113:0/3947520864 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400cd80 con 0x7ff7b0106130 2024-04-22T07:21:52.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b01993f0 con 0x7ff7b0106130 2024-04-22T07:21:52.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7acff9640 1 -- 172.21.15.113:0/3947520864 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff7a40054f0 con 0x7ff7b0106130 2024-04-22T07:21:52.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7acff9640 1 -- 172.21.15.113:0/3947520864 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4010690 con 0x7ff7b0106130 2024-04-22T07:21:52.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.191+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a5690 con 0x7ff7b0106130 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.193+0000 7ff7acff9640 1 -- 172.21.15.113:0/3947520864 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff7a40046c0 con 0x7ff7b0106130 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 Processor -- start 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 -- start start 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f815960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f81e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f820e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56097f77a1e0 con 0x56097f815580 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56097f717390 con 0x56097f815ea0 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56097f7761a0 con 0x56097f81ea80 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7acff9640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff7980a3d20 0x7ff7980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7affff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f815960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7affff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f815960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48756/0 (socket says 172.21.15.113:48756) 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7affff640 1 -- 172.21.15.113:0/2443280562 learned_addr learned my addr 172.21.15.113:0/2443280562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7affff640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff7980a3d20 0x7ff7980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f81e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7acff9640 1 -- 172.21.15.113:0/3947520864 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7ff7a40d0ff0 con 0x7ff7b0106130 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.194+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f820e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 -- 172.21.15.113:0/2443280562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 msgr2=0x56097f820e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f820e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 -- 172.21.15.113:0/2443280562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 msgr2=0x56097f81e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f81e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 -- 172.21.15.113:0/2443280562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097f5567d0 con 0x56097f815580 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f820e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f81e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f815960 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8032540 tx=0x7ff7a80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aece89963831c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff7affff640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff7980a3d20 0x7ff7980a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7a802c630 tx=0x7ff7a8040000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff795ffb640 1 -- 172.21.15.113:0/2443280562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8039040 con 0x56097f815580 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff795ffb640 1 -- 172.21.15.113:0/2443280562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff7a8050070 con 0x56097f815580 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.195+0000 7ff795ffb640 1 -- 172.21.15.113:0/2443280562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80346a0 con 0x56097f815580 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2443280562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 msgr2=0x56097f815960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f815960 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8032540 tx=0x7ff7a80327b0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2443280562 shutdown_connections 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f820e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f81e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2443280562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f815960 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2443280562 >> 172.21.15.113:0/2443280562 conn(0x56097f80fcb0 msgr2=0x56097f810090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2443280562 shutdown_connections 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2443280562 wait complete. 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- start start 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 Processor -- start 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815580 0x56097f81bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 0x56097f81c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f81c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56097f821810 con 0x56097f815ea0 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56097f7332f0 con 0x56097f81ea80 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.196+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56097f821990 con 0x56097f815580 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7b8c23740 10 client.0 _ll_register_callbacks cb 0x56097f740580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 0x56097f81c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 0x56097f81c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48768/0 (socket says 172.21.15.113:48768) 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3824348233 learned_addr learned my addr 172.21.15.113:0/3824348233 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3824348233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815580 msgr2=0x56097f81bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815580 0x56097f81bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 msgr2=0x56097f81c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7affff640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f81c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f81c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 -- 172.21.15.113:0/3824348233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097f48c3f0 con 0x56097f815ea0 2024-04-22T07:21:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7affff640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f81c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 0x56097f81c0c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009230 tx=0x7ff7a409c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f425a395559a9649 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff78effd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff78effd640 1 -- 172.21.15.113:0/3824348233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40cea00 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a8050070 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff78effd640 1 -- 172.21.15.113:0/3824348233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7ff7a4067b70 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff78effd640 1 -- 172.21.15.113:0/3824348233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff7a40a1180 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7b8c23740 10 client.17139 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.197+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56097f44c0f0 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff78effd640 1 -- 172.21.15.113:0/3824348233 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff7a8050070 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff78effd640 1 -- 172.21.15.113:0/3824348233 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7ff7a40bd0c0 con 0x56097f815ea0 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7b8c23740 10 client.17139 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7b8c23740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7b8c23740 20 client.17139 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f82e9f0 0x56097f830de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7b8c23740 4 client.17139 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56097f787c00 con 0x56097f82e9f0 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f82e9f0 0x56097f830de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f82e9f0 0x56097f830de0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8edecb8c4e921a35 server_cookie=a710ec58f3d89549 in_seq=0 out_seq=0 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.198+0000 7ff78effd640 10 client.17139 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff78effd640 1 -- 172.21.15.113:0/3824348233 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56097f787c00 con 0x56097f82e9f0 2024-04-22T07:21:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff78effd640 10 client.17139 handle_command_reply: tid=0 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 client.17139 shutdown 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 msgr2=0x56097f81c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 0x56097f81c0c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009230 tx=0x7ff7a409c9d0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 shutdown_connections 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f82e9f0 0x56097f830de0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f81c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815ea0 0x56097f81c0c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff78effd640 0 client.17139 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3824348233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815580 0x56097f81bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 >> 172.21.15.113:0/3824348233 conn(0x56097f80fcb0 msgr2=0x56097f811a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 shutdown_connections 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.199+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3824348233 wait complete. 2024-04-22T07:21:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.200+0000 7ff7b8c23740 20 client.17139 trim_cache size 0 max 16384 2024-04-22T07:21:52.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 Processor -- start 2024-04-22T07:21:52.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 -- start start 2024-04-22T07:21:52.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f82a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f811810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f8366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01b0770 con 0x56097f815580 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01ae040 con 0x56097f815ea0 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01abeb0 con 0x56097f81ea80 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f82a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f82a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48782/0 (socket says 172.21.15.113:48782) 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 -- 172.21.15.113:0/3109322614 learned_addr learned my addr 172.21.15.113:0/3109322614 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7affff640 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f8366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 -- 172.21.15.113:0/3109322614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 msgr2=0x56097f8366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f8366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 -- 172.21.15.113:0/3109322614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 msgr2=0x56097f811810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f811810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7af7fe640 1 -- 172.21.15.113:0/3109322614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097f831150 con 0x56097f815580 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.217+0000 7ff7affff640 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f8366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f82a6d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000cc70 tx=0x7ff7a000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f035f6350ca30c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7957fa640 1 -- 172.21.15.113:0/3109322614 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0010c20 con 0x56097f815580 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7957fa640 1 -- 172.21.15.113:0/3109322614 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff7a001d030 con 0x56097f815580 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7957fa640 1 -- 172.21.15.113:0/3109322614 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0003610 con 0x56097f815580 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3109322614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 msgr2=0x56097f82a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f82a6d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000cc70 tx=0x7ff7a000c810 comp rx=0 tx=0).stop 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3109322614 shutdown_connections 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f81ea80 0x56097f8366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f815ea0 0x56097f811810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/3109322614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f82a6d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3109322614 >> 172.21.15.113:0/3109322614 conn(0x56097f80fcb0 msgr2=0x56097f811100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3109322614 shutdown_connections 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.218+0000 7ff7b8c23740 1 -- 172.21.15.113:0/3109322614 wait complete. 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 -- start start 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 Processor -- start 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f826ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815ea0 0x56097f8270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f823120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01ac500 con 0x56097f815580 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01ad970 con 0x56097f81ea80 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01acbd0 con 0x56097f815ea0 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7b8c23740 10 client.0 _ll_register_callbacks cb 0x7ff7a400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f826ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f826ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48796/0 (socket says 172.21.15.113:48796) 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 -- 172.21.15.113:0/2251770379 learned_addr learned my addr 172.21.15.113:0/2251770379 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815ea0 0x56097f8270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7affff640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f823120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 -- 172.21.15.113:0/2251770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815ea0 msgr2=0x56097f8270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815ea0 0x56097f8270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 -- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 msgr2=0x56097f823120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f823120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7af7fe640 1 -- 172.21.15.113:0/2251770379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097f815cd0 con 0x56097f815580 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.219+0000 7ff7affff640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f823120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7af7fe640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f826ba0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0005bd0 tx=0x7ff7a0019860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e845ce28dcc6a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff78effd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff78effd640 1 -- 172.21.15.113:0/2251770379 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0010a20 con 0x56097f815580 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff78effd640 1 -- 172.21.15.113:0/2251770379 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7ff7a0010bc0 con 0x56097f815580 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a001d030 con 0x56097f815580 2024-04-22T07:21:52.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff78effd640 1 -- 172.21.15.113:0/2251770379 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff7a0004020 con 0x56097f815580 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 10 client.17145 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56097f8958f0 con 0x56097f815580 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff78effd640 1 -- 172.21.15.113:0/2251770379 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff7a001d030 con 0x56097f815580 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff78effd640 1 -- 172.21.15.113:0/2251770379 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7ff7a0004830 con 0x56097f815580 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 10 client.17145 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 20 client.17145 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f896000 0x56097f898420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 4 client.17145 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.220+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56097f781a70 con 0x56097f896000 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f896000 0x56097f898420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7aeffd640 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f896000 0x56097f898420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1b6492706569a6 server_cookie=50de43e51cff28be in_seq=0 out_seq=0 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff78effd640 10 client.17145 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff78effd640 1 -- 172.21.15.113:0/2251770379 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56097f781a70 con 0x56097f896000 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff78effd640 10 client.17145 handle_command_reply: tid=0 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 client.17145 shutdown 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 msgr2=0x56097f826ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f826ba0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0005bd0 tx=0x7ff7a0019860 comp rx=0 tx=0).stop 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 shutdown_connections 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56097f896000 0x56097f898420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56097f81ea80 0x56097f823120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56097f815ea0 0x56097f8270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 --2- 172.21.15.113:0/2251770379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56097f815580 0x56097f826ba0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 >> 172.21.15.113:0/2251770379 conn(0x56097f80fcb0 msgr2=0x56097f81c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff78effd640 0 client.17145 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 shutdown_connections 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.221+0000 7ff7b8c23740 1 -- 172.21.15.113:0/2251770379 wait complete. 2024-04-22T07:21:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b8c23740 20 client.17145 trim_cache size 0 max 16384 2024-04-22T07:21:52.231 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:21:52.231 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:21:52.231 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:21:52.231 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:21:52.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff7980a3d20 msgr2=0x7ff7980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff7980a3d20 0x7ff7980a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7a802c630 tx=0x7ff7a8040000 comp rx=0 tx=0).stop 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 msgr2=0x7ff7b019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b019e970 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400e9c0 tx=0x7ff7a400ee80 comp rx=0 tx=0).stop 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 shutdown_connections 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff7980a3d20 0x7ff7980a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7b01985e0 0x7ff7b0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b0106130 0x7ff7b019e970 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 --2- 172.21.15.113:0/3947520864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7b0101060 0x7ff7b019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 >> 172.21.15.113:0/3947520864 conn(0x7ff7b00771d0 msgr2=0x7ff7b006a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 shutdown_connections 2024-04-22T07:21:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.222+0000 7ff7b6031640 1 -- 172.21.15.113:0/3947520864 wait complete. 2024-04-22T07:21:52.234 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:21:52.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 -- 172.21.15.113:0/236774749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be0102b40 msgr2=0x7f6be0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 --2- 172.21.15.113:0/236774749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be0102b40 0x7f6be0106b80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 -- 172.21.15.113:0/236774749 shutdown_connections 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 --2- 172.21.15.113:0/236774749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be0107260 0x7f6be010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 --2- 172.21.15.113:0/236774749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be0102b40 0x7f6be0106b80 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 --2- 172.21.15.113:0/236774749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be0102190 0x7f6be0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 -- 172.21.15.113:0/236774749 >> 172.21.15.113:0/236774749 conn(0x7f6be00793e0 msgr2=0x7f6be00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 -- 172.21.15.113:0/236774749 shutdown_connections 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 -- 172.21.15.113:0/236774749 wait complete. 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.333+0000 7f6be842e640 1 Processor -- start 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 -- start start 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be0102190 0x7f6be019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be0107260 0x7f6be019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 0x7f6be0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6be0112340 con 0x7f6be01986a0 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6be01121c0 con 0x7f6be0107260 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6be01124c0 con 0x7f6be0102190 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 0x7f6be0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 0x7f6be0198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48812/0 (socket says 172.21.15.113:48812) 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 -- 172.21.15.113:0/1539931225 learned_addr learned my addr 172.21.15.113:0/1539931225 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be59c9640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be0107260 0x7f6be019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 -- 172.21.15.113:0/1539931225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be0102190 msgr2=0x7f6be019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be0102190 0x7f6be019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 -- 172.21.15.113:0/1539931225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be0107260 msgr2=0x7f6be019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be0107260 0x7f6be019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 -- 172.21.15.113:0/1539931225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be01992a0 con 0x7f6be01986a0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be69cb640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 0x7f6be0198b20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800b970 tx=0x7f6bd800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8019070 con 0x7f6be01986a0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6bd800cb40 con 0x7f6be01986a0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be01a5380 con 0x7f6be01986a0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8007650 con 0x7f6be01986a0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.334+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be01a5730 con 0x7f6be01986a0 2024-04-22T07:21:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.335+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be0103660 con 0x7f6be01986a0 2024-04-22T07:21:52.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.336+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6bd801c070 con 0x7f6be01986a0 2024-04-22T07:21:52.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.337+0000 7f6bd37fe640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6bcc0a3da0 0x7f6bcc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.337+0000 7f6be61ca640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6bcc0a3da0 0x7f6bcc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.337+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f6bd80d02d0 con 0x7f6be01986a0 2024-04-22T07:21:52.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.337+0000 7f6be61ca640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6bcc0a3da0 0x7f6bcc0a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40058c0 tx=0x7f6bd4005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.341+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bd809c290 con 0x7f6be01986a0 2024-04-22T07:21:52.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.556+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be0102b40 con 0x7f6be01986a0 2024-04-22T07:21:52.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.557+0000 7f6bd37fe640 1 -- 172.21.15.113:0/1539931225 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7f6bd80a0590 con 0x7f6be01986a0 2024-04-22T07:21:52.556 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:21:52.557 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:21:52.557 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.559+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6bcc0a3da0 msgr2=0x7f6bcc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.559+0000 7f6be842e640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6bcc0a3da0 0x7f6bcc0a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40058c0 tx=0x7f6bd4005850 comp rx=0 tx=0).stop 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.559+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 msgr2=0x7f6be0198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.559+0000 7f6be842e640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 0x7f6be0198b20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800b970 tx=0x7f6bd800be30 comp rx=0 tx=0).stop 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 shutdown_connections 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6bcc0a3da0 0x7f6bcc0a6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be01986a0 0x7f6be0198b20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be0107260 0x7f6be019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 --2- 172.21.15.113:0/1539931225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6be0102190 0x7f6be019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 >> 172.21.15.113:0/1539931225 conn(0x7f6be00793e0 msgr2=0x7f6be00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 shutdown_connections 2024-04-22T07:21:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.560+0000 7f6be842e640 1 -- 172.21.15.113:0/1539931225 wait complete. 2024-04-22T07:21:52.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:21:52.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 -- 172.21.15.113:0/63930206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204107260 msgr2=0x7fe20410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 --2- 172.21.15.113:0/63930206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204107260 0x7fe20410ee30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009ca0 tx=0x7fe1f402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 -- 172.21.15.113:0/63930206 shutdown_connections 2024-04-22T07:21:52.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 --2- 172.21.15.113:0/63930206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204107260 0x7fe20410ee30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 --2- 172.21.15.113:0/63930206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe204102b40 0x7fe204106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 --2- 172.21.15.113:0/63930206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe204102190 0x7fe204102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 -- 172.21.15.113:0/63930206 >> 172.21.15.113:0/63930206 conn(0x7fe2040793a0 msgr2=0x7fe2040797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 -- 172.21.15.113:0/63930206 shutdown_connections 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.673+0000 7fe20bba4640 1 -- 172.21.15.113:0/63930206 wait complete. 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 Processor -- start 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 -- start start 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe204102190 0x7fe204074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe204107260 0x7fe204074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 0x7fe2041a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041122e0 con 0x7fe204074cd0 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe204112160 con 0x7fe204102190 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20bba4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe204112460 con 0x7fe204107260 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 0x7fe2041a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 0x7fe2041a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48828/0 (socket says 172.21.15.113:48828) 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 -- 172.21.15.113:0/88492353 learned_addr learned my addr 172.21.15.113:0/88492353 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20913f640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe204107260 0x7fe204074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 -- 172.21.15.113:0/88492353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe204107260 msgr2=0x7fe204074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe204107260 0x7fe204074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 -- 172.21.15.113:0/88492353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe204102190 msgr2=0x7fe204074250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe204102190 0x7fe204074250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 -- 172.21.15.113:0/88492353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2041a7010 con 0x7fe204074cd0 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe20a141640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 0x7fe2041a6910 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4037040 tx=0x7fe1f4035d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe202ffd640 1 -- 172.21.15.113:0/88492353 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4039040 con 0x7fe204074cd0 2024-04-22T07:21:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.674+0000 7fe202ffd640 1 -- 172.21.15.113:0/88492353 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe1f4044070 con 0x7fe204074cd0 2024-04-22T07:21:52.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.675+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2041a72a0 con 0x7fe204074cd0 2024-04-22T07:21:52.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.675+0000 7fe202ffd640 1 -- 172.21.15.113:0/88492353 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4004330 con 0x7fe204074cd0 2024-04-22T07:21:52.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.675+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2041a77e0 con 0x7fe204074cd0 2024-04-22T07:21:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.677+0000 7fe202ffd640 1 -- 172.21.15.113:0/88492353 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe1f40044d0 con 0x7fe204074cd0 2024-04-22T07:21:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.677+0000 7fe20e790740 1 Processor -- start 2024-04-22T07:21:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 -- start start 2024-04-22T07:21:52.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 0x563c490b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 0x563c490b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563c490111f0 con 0x563c490ace80 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c48fae4b0 con 0x563c490ac4d0 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20e790740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563c4900d1b0 con 0x563c490b5a00 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe202ffd640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe1ec0a3dd0 0x7fe1ec0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 0x563c490b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 0x563c490b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48830/0 (socket says 172.21.15.113:48830) 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 -- 172.21.15.113:0/2051271283 learned_addr learned my addr 172.21.15.113:0/2051271283 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20a141640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe1ec0a3dd0 0x7fe1ec0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe20913f640 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe202ffd640 1 -- 172.21.15.113:0/88492353 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fe1f40f2d00 con 0x7fe204074cd0 2024-04-22T07:21:52.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 -- 172.21.15.113:0/2051271283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 msgr2=0x563c490b7e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 0x563c490b7e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 -- 172.21.15.113:0/2051271283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 msgr2=0x563c490ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 -- 172.21.15.113:0/2051271283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c48fa01b0 con 0x563c490ace80 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.678+0000 7fe209940640 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 0x563c490b5430 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00aa20 tx=0x7fe1fc00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c40a156bd3e86c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20a141640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe1ec0a3dd0 0x7fe1ec0a6290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40be8b0 tx=0x7fe1f40df040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe1ebfff640 1 -- 172.21.15.113:0/2051271283 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00ce40 con 0x563c490ace80 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe1ebfff640 1 -- 172.21.15.113:0/2051271283 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe1fc00f920 con 0x563c490ace80 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe1ebfff640 1 -- 172.21.15.113:0/2051271283 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00ea20 con 0x563c490ace80 2024-04-22T07:21:52.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 -- 172.21.15.113:0/2051271283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 msgr2=0x563c490b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 0x563c490b5430 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00aa20 tx=0x7fe1fc00aee0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 -- 172.21.15.113:0/2051271283 shutdown_connections 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 0x563c490b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ace80 0x563c490b5430 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 --2- 172.21.15.113:0/2051271283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.679+0000 7fe20e790740 1 -- 172.21.15.113:0/2051271283 >> 172.21.15.113:0/2051271283 conn(0x563c490a6ba0 msgr2=0x563c490a6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 -- 172.21.15.113:0/2051271283 shutdown_connections 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 -- 172.21.15.113:0/2051271283 wait complete. 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 -- start start 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 Processor -- start 2024-04-22T07:21:52.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490b5a00 0x563c490b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563c490b8970 con 0x563c490ac4d0 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c48fca500 con 0x563c490b5a00 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563c490b8af0 con 0x563c490ace80 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20e790740 10 client.0 _ll_register_callbacks cb 0x563c48f98b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20913f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20913f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48846/0 (socket says 172.21.15.113:48846) 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.680+0000 7fe20913f640 1 -- 172.21.15.113:0/358634616 learned_addr learned my addr 172.21.15.113:0/358634616 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20913f640 1 -- 172.21.15.113:0/358634616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 msgr2=0x563c490b26f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20913f640 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b26f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20913f640 1 -- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490b5a00 msgr2=0x563c490b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20913f640 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490b5a00 0x563c490b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20913f640 1 -- 172.21.15.113:0/358634616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c48d233f0 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20913f640 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490b5360 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800da60 tx=0x7fe1f800df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0ce8ec4aeec787 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe1e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe1e97fa640 1 -- 172.21.15.113:0/358634616 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8005020 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe1e97fa640 1 -- 172.21.15.113:0/358634616 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fe1f80051c0 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1fc00f920 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe1e97fa640 1 -- 172.21.15.113:0/358634616 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe1f8023050 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 10 client.17163 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563c48ded7d0 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe1e97fa640 1 -- 172.21.15.113:0/358634616 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe1fc00f920 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe1e97fa640 1 -- 172.21.15.113:0/358634616 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fe1f8018600 con 0x563c490ac4d0 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 10 client.17163 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 20 client.17163 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c490c5d30 0x563c490c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 4 client.17163 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.681+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c48fb0930 con 0x563c490c5d30 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.684+0000 7fe209940640 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c490c5d30 0x563c490c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.686+0000 7fe209940640 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c490c5d30 0x563c490c81d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1510b7d1a66030 server_cookie=f9b5195f5a590661 in_seq=0 out_seq=0 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.686+0000 7fe1e97fa640 10 client.17163 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.687+0000 7fe1e97fa640 1 -- 172.21.15.113:0/358634616 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c48fb0930 con 0x563c490c5d30 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.687+0000 7fe1e97fa640 10 client.17163 handle_command_reply: tid=0 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.687+0000 7fe20e790740 1 client.17163 shutdown 2024-04-22T07:21:52.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 msgr2=0x563c490b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490b5360 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800da60 tx=0x7fe1f800df20 comp rx=0 tx=0).stop 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 shutdown_connections 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c490c5d30 0x563c490c81d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490b5a00 0x563c490b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe1e97fa640 0 client.17163 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 --2- 172.21.15.113:0/358634616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490b5360 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 >> 172.21.15.113:0/358634616 conn(0x563c490a6ba0 msgr2=0x563c490a88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 shutdown_connections 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 1 -- 172.21.15.113:0/358634616 wait complete. 2024-04-22T07:21:52.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.688+0000 7fe20e790740 20 client.17163 trim_cache size 0 max 16384 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 Processor -- start 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 -- start start 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ace80 0x563c490b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 0x563c490b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b2a60 con 0x563c490ac4d0 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b0330 con 0x563c490ace80 2024-04-22T07:21:52.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.701+0000 7fe20e790740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041ae1a0 con 0x563c490b5a00 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490c1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48858/0 (socket says 172.21.15.113:48858) 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 -- 172.21.15.113:0/1610833564 learned_addr learned my addr 172.21.15.113:0/1610833564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe209940640 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ace80 0x563c490b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 -- 172.21.15.113:0/1610833564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 msgr2=0x563c490b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 0x563c490b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 -- 172.21.15.113:0/1610833564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ace80 msgr2=0x563c490b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ace80 0x563c490b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 -- 172.21.15.113:0/1610833564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c490ae2c0 con 0x563c490ac4d0 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe20913f640 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490c1860 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80059f0 tx=0x7fe1f8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ada52e2dbbafd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe1eb7fe640 1 -- 172.21.15.113:0/1610833564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80186c0 con 0x563c490ac4d0 2024-04-22T07:21:52.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe1eb7fe640 1 -- 172.21.15.113:0/1610833564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe1f802b030 con 0x563c490ac4d0 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.702+0000 7fe1eb7fe640 1 -- 172.21.15.113:0/1610833564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8027660 con 0x563c490ac4d0 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 -- 172.21.15.113:0/1610833564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 msgr2=0x563c490c1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490c1860 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80059f0 tx=0x7fe1f8003b00 comp rx=0 tx=0).stop 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 -- 172.21.15.113:0/1610833564 shutdown_connections 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490b5a00 0x563c490b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ace80 0x563c490b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 --2- 172.21.15.113:0/1610833564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490ac4d0 0x563c490c1860 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 -- 172.21.15.113:0/1610833564 >> 172.21.15.113:0/1610833564 conn(0x563c490a6ba0 msgr2=0x563c490a5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 -- 172.21.15.113:0/1610833564 shutdown_connections 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 -- 172.21.15.113:0/1610833564 wait complete. 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 -- start start 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.703+0000 7fe20e790740 1 Processor -- start 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 0x563c490ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041ae7f0 con 0x563c490b5a00 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041afc60 con 0x563c490ac4d0 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041aeec0 con 0x563c490ace80 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 10 client.0 _ll_register_callbacks cb 0x7fe1f8004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 0x563c490ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 0x563c490ba370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48864/0 (socket says 172.21.15.113:48864) 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 -- 172.21.15.113:0/3998566697 learned_addr learned my addr 172.21.15.113:0/3998566697 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe209940640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 -- 172.21.15.113:0/3998566697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 msgr2=0x563c490b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 -- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 msgr2=0x563c490b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 -- 172.21.15.113:0/3998566697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4912b470 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe209940640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20913f640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 0x563c490ba370 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8001eb0 tx=0x7fe1f8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d61c588865a5fef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe1e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe1e97fa640 1 -- 172.21.15.113:0/3998566697 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8018780 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe1e97fa640 1 -- 172.21.15.113:0/3998566697 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fe1f8011440 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f802b030 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.704+0000 7fe1e97fa640 1 -- 172.21.15.113:0/3998566697 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe1f8011640 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 10 client.17169 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563c4912c750 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe1e97fa640 1 -- 172.21.15.113:0/3998566697 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe1f802b030 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe1e97fa640 1 -- 172.21.15.113:0/3998566697 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fe1f803bb90 con 0x563c490b5a00 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 10 client.17169 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 20 client.17169 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c4912cc90 0x563c4912f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 4 client.17169 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c4901ec10 con 0x563c4912cc90 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe209940640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c4912cc90 0x563c4912f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe1e97fa640 10 client.17169 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.705+0000 7fe209940640 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c4912cc90 0x563c4912f0b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993fa2885c933db2 server_cookie=1b7387c68de030b0 in_seq=0 out_seq=0 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe1e97fa640 1 -- 172.21.15.113:0/3998566697 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c4901ec10 con 0x563c4912cc90 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe1e97fa640 10 client.17169 handle_command_reply: tid=0 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 client.17169 shutdown 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 msgr2=0x563c490ba370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 0x563c490ba370 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8001eb0 tx=0x7fe1f8003dc0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 shutdown_connections 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563c4912cc90 0x563c4912f0b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563c490b5a00 0x563c490ba370 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563c490ace80 0x563c490b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 --2- 172.21.15.113:0/3998566697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c490ac4d0 0x563c490b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe1e97fa640 0 client.17169 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 >> 172.21.15.113:0/3998566697 conn(0x563c490a6ba0 msgr2=0x563c490a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 shutdown_connections 2024-04-22T07:21:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 1 -- 172.21.15.113:0/3998566697 wait complete. 2024-04-22T07:21:52.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20e790740 20 client.17169 trim_cache size 0 max 16384 2024-04-22T07:21:52.714 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe1ec0a3dd0 msgr2=0x7fe1ec0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20bba4640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe1ec0a3dd0 0x7fe1ec0a6290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40be8b0 tx=0x7fe1f40df040 comp rx=0 tx=0).stop 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 msgr2=0x7fe2041a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.706+0000 7fe20bba4640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 0x7fe2041a6910 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4037040 tx=0x7fe1f4035d00 comp rx=0 tx=0).stop 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 shutdown_connections 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe1ec0a3dd0 0x7fe1ec0a6290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe204074cd0 0x7fe2041a6910 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe204107260 0x7fe204074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 --2- 172.21.15.113:0/88492353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe204102190 0x7fe204074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 >> 172.21.15.113:0/88492353 conn(0x7fe2040793a0 msgr2=0x7fe2040ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 shutdown_connections 2024-04-22T07:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.707+0000 7fe20bba4640 1 -- 172.21.15.113:0/88492353 wait complete. 2024-04-22T07:21:52.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 -- 172.21.15.113:0/2786631337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100e10 msgr2=0x7fab5810ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 --2- 172.21.15.113:0/2786631337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100e10 0x7fab5810ede0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fab50009cd0 tx=0x7fab5002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 -- 172.21.15.113:0/2786631337 shutdown_connections 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 --2- 172.21.15.113:0/2786631337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100e10 0x7fab5810ede0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 --2- 172.21.15.113:0/2786631337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab58100450 0x7fab581008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 --2- 172.21.15.113:0/2786631337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab58106f30 0x7fab58107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 -- 172.21.15.113:0/2786631337 >> 172.21.15.113:0/2786631337 conn(0x7fab580773e0 msgr2=0x7fab580fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 -- 172.21.15.113:0/2786631337 shutdown_connections 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.818+0000 7fab5ea11640 1 -- 172.21.15.113:0/2786631337 wait complete. 2024-04-22T07:21:52.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 Processor -- start 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 -- start start 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 0x7fab5819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab58100e10 0x7fab5819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab58106f30 0x7fab58198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112430 con 0x7fab58100450 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fab581122b0 con 0x7fab58106f30 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5ea11640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab581125b0 con 0x7fab58100e10 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 0x7fab5819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 0x7fab5819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48890/0 (socket says 172.21.15.113:48890) 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 -- 172.21.15.113:0/4077172478 learned_addr learned my addr 172.21.15.113:0/4077172478 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab5cfae640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab58106f30 0x7fab58198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 -- 172.21.15.113:0/4077172478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab58100e10 msgr2=0x7fab5819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab58100e10 0x7fab5819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 -- 172.21.15.113:0/4077172478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab58106f30 msgr2=0x7fab58198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab58106f30 0x7fab58198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.819+0000 7fab57fff640 1 -- 172.21.15.113:0/4077172478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab58199010 con 0x7fab58100450 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.820+0000 7fab57fff640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 0x7fab5819e640 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b4a0 tx=0x7fab4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.820+0000 7fab557fa640 1 -- 172.21.15.113:0/4077172478 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48014070 con 0x7fab58100450 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.820+0000 7fab557fa640 1 -- 172.21.15.113:0/4077172478 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fab48010ae0 con 0x7fab58100450 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.820+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab58199300 con 0x7fab58100450 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.820+0000 7fab557fa640 1 -- 172.21.15.113:0/4077172478 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48017ce0 con 0x7fab58100450 2024-04-22T07:21:52.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.820+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a55c0 con 0x7fab58100450 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 Processor -- start 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 -- start start 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab557fa640 1 -- 172.21.15.113:0/4077172478 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fab48017070 con 0x7fab58100450 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564091294d0 0x5564091298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 0x556409132430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409134e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55640908e1f0 con 0x556409129e80 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55640902b4b0 con 0x556409132a00 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab615fe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55640908a1b0 con 0x5564091294d0 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 0x556409132430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 0x556409132430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48894/0 (socket says 172.21.15.113:48894) 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 -- 172.21.15.113:0/42298436 learned_addr learned my addr 172.21.15.113:0/42298436 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab577fe640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab5cfae640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564091294d0 0x5564091298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 -- 172.21.15.113:0/42298436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564091294d0 msgr2=0x5564091298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564091294d0 0x5564091298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 -- 172.21.15.113:0/42298436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 msgr2=0x556409134e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409134e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 -- 172.21.15.113:0/42298436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55640901d1b0 con 0x556409129e80 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab577fe640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab5cfae640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564091294d0 0x5564091298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:52.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab557fa640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fab400a3d20 0x7fab400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.823+0000 7fab57fff640 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 0x556409132430 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fab480cd320 tx=0x7fab480d8830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3649d63579f606f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab5cfae640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fab400a3d20 0x7fab400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab557fa640 1 -- 172.21.15.113:0/4077172478 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fab480cfcf0 con 0x7fab58100450 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab3e7fc640 1 -- 172.21.15.113:0/42298436 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48097ea0 con 0x556409129e80 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab3e7fc640 1 -- 172.21.15.113:0/42298436 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fab4809c300 con 0x556409129e80 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab3e7fc640 1 -- 172.21.15.113:0/42298436 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4809b560 con 0x556409129e80 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 -- 172.21.15.113:0/42298436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 msgr2=0x556409132430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab5cfae640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fab400a3d20 0x7fab400a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab50006010 tx=0x7fab5002e040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 0x556409132430 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fab480cd320 tx=0x7fab480d8830 comp rx=0 tx=0).stop 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 -- 172.21.15.113:0/42298436 shutdown_connections 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409129e80 0x556409132430 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 --2- 172.21.15.113:0/42298436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5564091294d0 0x5564091298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 -- 172.21.15.113:0/42298436 >> 172.21.15.113:0/42298436 conn(0x556409123ba0 msgr2=0x556409123f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 -- 172.21.15.113:0/42298436 shutdown_connections 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.824+0000 7fab615fe740 1 -- 172.21.15.113:0/42298436 wait complete. 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 -- start start 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 Processor -- start 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x5564091322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x55640912f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x55640912fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556409135970 con 0x5564091294d0 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556409047500 con 0x556409132a00 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556409135af0 con 0x556409129e80 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab615fe740 10 client.0 _ll_register_callbacks cb 0x556409015b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x5564091322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x5564091322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48904/0 (socket says 172.21.15.113:48904) 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab57fff640 1 -- 172.21.15.113:0/3423097039 learned_addr learned my addr 172.21.15.113:0/3423097039 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab5cfae640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x55640912fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.825+0000 7fab577fe640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x55640912f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab57fff640 1 -- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 msgr2=0x55640912f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab57fff640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x55640912f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab57fff640 1 -- 172.21.15.113:0/3423097039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 msgr2=0x55640912fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab57fff640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x55640912fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab57fff640 1 -- 172.21.15.113:0/3423097039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556408da03f0 con 0x5564091294d0 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab57fff640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x5564091322f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fab480cd8a0 tx=0x7fab48009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d143d2aa326b4a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab377fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab377fe640 1 -- 172.21.15.113:0/3423097039 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4809b740 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab377fe640 1 -- 172.21.15.113:0/3423097039 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fab480bc0f0 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4809c300 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab377fe640 1 -- 172.21.15.113:0/3423097039 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fab4809b9d0 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 10 client.17181 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556408e6a7d0 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab377fe640 1 -- 172.21.15.113:0/3423097039 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fab4809c300 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab377fe640 1 -- 172.21.15.113:0/3423097039 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fab48098270 con 0x5564091294d0 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 10 client.17181 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 10 client.17181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 20 client.17181 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x556409142b00 0x556409144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 4 client.17181 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.826+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55640902d930 con 0x556409142b00 2024-04-22T07:21:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.827+0000 7fab577fe640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x556409142b00 0x556409144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.827+0000 7fab377fe640 10 client.17181 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.827+0000 7fab577fe640 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x556409142b00 0x556409144ef0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ea83f75f51e2865 server_cookie=b3b6bf8376d66c0b in_seq=0 out_seq=0 2024-04-22T07:21:52.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab377fe640 1 -- 172.21.15.113:0/3423097039 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55640902d930 con 0x556409142b00 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab377fe640 10 client.17181 handle_command_reply: tid=0 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 client.17181 shutdown 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 msgr2=0x5564091322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x5564091322f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fab480cd8a0 tx=0x7fab48009320 comp rx=0 tx=0).stop 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 shutdown_connections 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x556409142b00 0x556409144ef0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x55640912fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x55640912f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 --2- 172.21.15.113:0/3423097039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x5564091322f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab377fe640 0 client.17181 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 >> 172.21.15.113:0/3423097039 conn(0x556409123ba0 msgr2=0x5564091258f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 shutdown_connections 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 1 -- 172.21.15.113:0/3423097039 wait complete. 2024-04-22T07:21:52.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.828+0000 7fab615fe740 20 client.17181 trim_cache size 0 max 16384 2024-04-22T07:21:52.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:52 smithi113 ceph-mon[27933]: pgmap v220: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:52.838 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:52 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1539931225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:21:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.841+0000 7fab615fe740 1 Processor -- start 2024-04-22T07:21:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.841+0000 7fab615fe740 1 -- start start 2024-04-22T07:21:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.841+0000 7fab615fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564091294d0 0x556409122a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.841+0000 7fab615fe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x556409131db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.841+0000 7fab615fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 0x5564091322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.841+0000 7fab615fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab581b08b0 con 0x556409132a00 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab615fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fab581ae180 con 0x5564091294d0 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab615fe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab581abff0 con 0x556409129e80 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 0x5564091322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 0x5564091322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48910/0 (socket says 172.21.15.113:48910) 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 -- 172.21.15.113:0/1651891593 learned_addr learned my addr 172.21.15.113:0/1651891593 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 -- 172.21.15.113:0/1651891593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 msgr2=0x556409131db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x556409131db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 -- 172.21.15.113:0/1651891593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564091294d0 msgr2=0x556409122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564091294d0 0x556409122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 -- 172.21.15.113:0/1651891593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556409129c20 con 0x556409132a00 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab5cfae640 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 0x5564091322f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fab50009cd0 tx=0x7fab5003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be500e6ffa93882 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab3dffb640 1 -- 172.21.15.113:0/1651891593 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab50039040 con 0x556409132a00 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab3dffb640 1 -- 172.21.15.113:0/1651891593 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fab50051070 con 0x556409132a00 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.842+0000 7fab3dffb640 1 -- 172.21.15.113:0/1651891593 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5004c6a0 con 0x556409132a00 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 -- 172.21.15.113:0/1651891593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 msgr2=0x5564091322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 0x5564091322f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fab50009cd0 tx=0x7fab5003f000 comp rx=0 tx=0).stop 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 -- 172.21.15.113:0/1651891593 shutdown_connections 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556409132a00 0x5564091322f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x556409131db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 --2- 172.21.15.113:0/1651891593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564091294d0 0x556409122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 -- 172.21.15.113:0/1651891593 >> 172.21.15.113:0/1651891593 conn(0x556409123ba0 msgr2=0x556409122370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 -- 172.21.15.113:0/1651891593 shutdown_connections 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 -- 172.21.15.113:0/1651891593 wait complete. 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 -- start start 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.843+0000 7fab615fe740 1 Processor -- start 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x556409148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x5564091455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409145b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab581ac640 con 0x5564091294d0 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fab581adab0 con 0x556409132a00 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab581acd10 con 0x556409129e80 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 10 client.0 _ll_register_callbacks cb 0x7fab480d33e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x556409148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x556409148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48922/0 (socket says 172.21.15.113:48922) 2024-04-22T07:21:52.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 -- 172.21.15.113:0/3095518092 learned_addr learned my addr 172.21.15.113:0/3095518092 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab5cfae640 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 -- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 msgr2=0x5564091455f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x5564091455f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 -- 172.21.15.113:0/3095518092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 msgr2=0x556409145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 -- 172.21.15.113:0/3095518092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564091462e0 con 0x5564091294d0 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab57fff640 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x556409148260 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fab480985d0 tx=0x7fab480d0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24e35d6d02bfd5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab377fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab377fe640 1 -- 172.21.15.113:0/3095518092 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4809b740 con 0x5564091294d0 2024-04-22T07:21:52.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab377fe640 1 -- 172.21.15.113:0/3095518092 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7fab480a45e0 con 0x5564091294d0 2024-04-22T07:21:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab50051070 con 0x5564091294d0 2024-04-22T07:21:52.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab377fe640 1 -- 172.21.15.113:0/3095518092 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fab4809b560 con 0x5564091294d0 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 10 client.17187 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.844+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5564091462e0 con 0x5564091294d0 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab377fe640 1 -- 172.21.15.113:0/3095518092 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fab50051070 con 0x5564091294d0 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab377fe640 1 -- 172.21.15.113:0/3095518092 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fab48060b80 con 0x5564091294d0 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab615fe740 10 client.17187 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab615fe740 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab615fe740 20 client.17187 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab615fe740 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5564091a9010 0x5564091ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab615fe740 4 client.17187 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.845+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55640909bc10 con 0x5564091a9010 2024-04-22T07:21:52.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.846+0000 7fab577fe640 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5564091a9010 0x5564091ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab377fe640 10 client.17187 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab577fe640 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5564091a9010 0x5564091ab400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=979897c567c856a2 server_cookie=203ed57579653731 in_seq=0 out_seq=0 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab377fe640 1 -- 172.21.15.113:0/3095518092 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55640909bc10 con 0x5564091a9010 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab377fe640 10 client.17187 handle_command_reply: tid=0 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab615fe740 1 client.17187 shutdown 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 msgr2=0x556409148260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.847+0000 7fab615fe740 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x556409148260 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fab480985d0 tx=0x7fab480d0060 comp rx=0 tx=0).stop 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 shutdown_connections 2024-04-22T07:21:52.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x5564091a9010 0x5564091ab400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556409132a00 0x556409145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556409129e80 0x5564091455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab377fe640 0 client.17187 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 --2- 172.21.15.113:0/3095518092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5564091294d0 0x556409148260 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 >> 172.21.15.113:0/3095518092 conn(0x556409123ba0 msgr2=0x556409122c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 shutdown_connections 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 1 -- 172.21.15.113:0/3095518092 wait complete. 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab615fe740 20 client.17187 trim_cache size 0 max 16384 2024-04-22T07:21:52.853 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:21:52.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fab400a3d20 msgr2=0x7fab400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fab400a3d20 0x7fab400a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab50006010 tx=0x7fab5002e040 comp rx=0 tx=0).stop 2024-04-22T07:21:52.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 msgr2=0x7fab5819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 0x7fab5819e640 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b4a0 tx=0x7fab4800b960 comp rx=0 tx=0).stop 2024-04-22T07:21:52.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 shutdown_connections 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fab400a3d20 0x7fab400a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab58106f30 0x7fab58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab58100e10 0x7fab5819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 --2- 172.21.15.113:0/4077172478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab58100450 0x7fab5819e640 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 >> 172.21.15.113:0/4077172478 conn(0x7fab580773e0 msgr2=0x7fab58105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 shutdown_connections 2024-04-22T07:21:52.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.848+0000 7fab5ea11640 1 -- 172.21.15.113:0/4077172478 wait complete. 2024-04-22T07:21:52.857 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 -- 172.21.15.113:0/1941559672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102b40 msgr2=0x7ff014106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/1941559672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102b40 0x7ff014106b80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff010009900 tx=0x7ff01002bc50 comp rx=0 tx=0).stop 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 -- 172.21.15.113:0/1941559672 shutdown_connections 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/1941559672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff014107260 0x7ff01410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/1941559672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102b40 0x7ff014106b80 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/1941559672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff014102190 0x7ff014102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 -- 172.21.15.113:0/1941559672 >> 172.21.15.113:0/1941559672 conn(0x7ff0140793a0 msgr2=0x7ff0140797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 -- 172.21.15.113:0/1941559672 shutdown_connections 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 -- 172.21.15.113:0/1941559672 wait complete. 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.958+0000 7ff01bd8f640 1 Processor -- start 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 -- start start 2024-04-22T07:21:52.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 0x7ff01419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff014107260 0x7ff01419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0141986a0 0x7ff014198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff014112360 con 0x7ff014102190 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141121e0 con 0x7ff014107260 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141124e0 con 0x7ff0141986a0 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 0x7ff01419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 0x7ff01419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48950/0 (socket says 172.21.15.113:48950) 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 -- 172.21.15.113:0/3557914247 learned_addr learned my addr 172.21.15.113:0/3557914247 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01932a640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff014107260 0x7ff01419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 -- 172.21.15.113:0/3557914247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0141986a0 msgr2=0x7ff014198b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0141986a0 0x7ff014198b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 -- 172.21.15.113:0/3557914247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff014107260 msgr2=0x7ff01419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff014107260 0x7ff01419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 -- 172.21.15.113:0/3557914247 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0141992a0 con 0x7ff014102190 2024-04-22T07:21:52.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 0x7ff01419e5e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c920 tx=0x7ff00400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff00affd640 1 -- 172.21.15.113:0/3557914247 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004007c70 con 0x7ff014102190 2024-04-22T07:21:52.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff00affd640 1 -- 172.21.15.113:0/3557914247 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff004007e10 con 0x7ff014102190 2024-04-22T07:21:52.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0141a5380 con 0x7ff014102190 2024-04-22T07:21:52.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.959+0000 7ff00affd640 1 -- 172.21.15.113:0/3557914247 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040188c0 con 0x7ff014102190 2024-04-22T07:21:52.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.960+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a5730 con 0x7ff014102190 2024-04-22T07:21:52.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.962+0000 7ff00affd640 1 -- 172.21.15.113:0/3557914247 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff004018a60 con 0x7ff014102190 2024-04-22T07:21:52.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.962+0000 7ff01e97f740 1 Processor -- start 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.962+0000 7ff01e97f740 1 -- start start 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.962+0000 7ff00affd640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff0000a3d20 0x7ff0000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 0x557f7f59e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01e97f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557f7f5031f0 con 0x557f7f59e4d0 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01e97f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f7f4a04b0 con 0x557f7f59ee80 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01e97f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557f7f4ff1b0 con 0x557f7f5a7a00 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff00affd640 1 -- 172.21.15.113:0/3557914247 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7ff0040d0ba0 con 0x7ff014102190 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01a32c640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff0000a3d20 0x7ff0000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 0x557f7f59e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 0x557f7f59e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48958/0 (socket says 172.21.15.113:48958) 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 -- 172.21.15.113:0/452294598 learned_addr learned my addr 172.21.15.113:0/452294598 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01932a640 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 -- 172.21.15.113:0/452294598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 msgr2=0x557f7f5a9e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5a9e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 -- 172.21.15.113:0/452294598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 msgr2=0x557f7f5a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 -- 172.21.15.113:0/452294598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7f4921b0 con 0x557f7f59e4d0 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff01a32c640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff0000a3d20 0x7ff0000a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ab70 tx=0x7ff00c009290 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7ff019b2b640 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 0x557f7f59e8b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff004098e00 tx=0x7ff00400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9479844099cf9ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7feffffff640 1 -- 172.21.15.113:0/452294598 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00409c9a0 con 0x557f7f59e4d0 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7feffffff640 1 -- 172.21.15.113:0/452294598 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0040a0d50 con 0x557f7f59e4d0 2024-04-22T07:21:52.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.963+0000 7feffffff640 1 -- 172.21.15.113:0/452294598 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004099010 con 0x557f7f59e4d0 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 -- 172.21.15.113:0/452294598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 msgr2=0x557f7f59e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 0x557f7f59e8b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff004098e00 tx=0x7ff00400de20 comp rx=0 tx=0).stop 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 -- 172.21.15.113:0/452294598 shutdown_connections 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 --2- 172.21.15.113:0/452294598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59e4d0 0x557f7f59e8b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 -- 172.21.15.113:0/452294598 >> 172.21.15.113:0/452294598 conn(0x557f7f598ba0 msgr2=0x557f7f598f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 -- 172.21.15.113:0/452294598 shutdown_connections 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 -- 172.21.15.113:0/452294598 wait complete. 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 -- start start 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.964+0000 7ff01e97f740 1 Processor -- start 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557f7f5aa970 con 0x557f7f59ee80 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f7f4bc500 con 0x557f7f59e4d0 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557f7f5aaaf0 con 0x557f7f5a7a00 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01e97f740 10 client.0 _ll_register_callbacks cb 0x557f7f48ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48974/0 (socket says 172.21.15.113:48974) 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 -- 172.21.15.113:0/157036983 learned_addr learned my addr 172.21.15.113:0/157036983 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 -- 172.21.15.113:0/157036983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 msgr2=0x557f7f5a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 -- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 msgr2=0x557f7f5a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 -- 172.21.15.113:0/157036983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7f2153f0 con 0x557f7f59ee80 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7ff01932a640 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5a46c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff01406b820 tx=0x7ff010004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acc3dde02b33bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.965+0000 7feffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7feffd7fa640 1 -- 172.21.15.113:0/157036983 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010005230 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7feffd7fa640 1 -- 172.21.15.113:0/157036983 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7ff0100053d0 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0040a0d50 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7feffd7fa640 1 -- 172.21.15.113:0/157036983 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff0100055d0 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 10 client.17199 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557f7f2df7d0 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7feffd7fa640 1 -- 172.21.15.113:0/157036983 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0040a0d50 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7feffd7fa640 1 -- 172.21.15.113:0/157036983 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7ff010003920 con 0x557f7f59ee80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 10 client.17199 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 10 client.17199 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 20 client.17199 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f5b7c80 0x557f7f5ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 4 client.17199 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.966+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f7f4a2930 con 0x557f7f5b7c80 2024-04-22T07:21:52.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.967+0000 7ff019b2b640 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f5b7c80 0x557f7f5ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.967+0000 7ff019b2b640 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f5b7c80 0x557f7f5ba140 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcaaff0a89f3edd8 server_cookie=7c2be270d9910f47 in_seq=0 out_seq=0 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.967+0000 7feffd7fa640 10 client.17199 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7feffd7fa640 1 -- 172.21.15.113:0/157036983 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f7f4a2930 con 0x557f7f5b7c80 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7feffd7fa640 10 client.17199 handle_command_reply: tid=0 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 client.17199 shutdown 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 msgr2=0x557f7f5a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5a46c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff01406b820 tx=0x7ff010004040 comp rx=0 tx=0).stop 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 shutdown_connections 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f5b7c80 0x557f7f5ba140 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5a46c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 --2- 172.21.15.113:0/157036983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7feffd7fa640 0 client.17199 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 >> 172.21.15.113:0/157036983 conn(0x557f7f598ba0 msgr2=0x557f7f59a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 shutdown_connections 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 1 -- 172.21.15.113:0/157036983 wait complete. 2024-04-22T07:21:52.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.968+0000 7ff01e97f740 20 client.17199 trim_cache size 0 max 16384 2024-04-22T07:21:52.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 Processor -- start 2024-04-22T07:21:52.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 -- start start 2024-04-22T07:21:52.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f59e4d0 0x557f7f5b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 0x557f7f5a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141b0900 con 0x557f7f5a7a00 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ae1d0 con 0x557f7f59ee80 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01e97f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ac040 con 0x557f7f59e4d0 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01a32c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 0x557f7f5a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01a32c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 0x557f7f5a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48978/0 (socket says 172.21.15.113:48978) 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.981+0000 7ff01a32c640 1 -- 172.21.15.113:0/1294114506 learned_addr learned my addr 172.21.15.113:0/1294114506 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01a32c640 1 -- 172.21.15.113:0/1294114506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f59e4d0 msgr2=0x557f7f5b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01a32c640 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f59e4d0 0x557f7f5b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01a32c640 1 -- 172.21.15.113:0/1294114506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 msgr2=0x557f7f5a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01a32c640 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01a32c640 1 -- 172.21.15.113:0/1294114506 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7f5a5040 con 0x557f7f5a7a00 2024-04-22T07:21:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01a32c640 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 0x557f7f5a7330 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ce80 tx=0x7ff00c00e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf4ab17e84873eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7fefff7fe640 1 -- 172.21.15.113:0/1294114506 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c019420 con 0x557f7f5a7a00 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7fefff7fe640 1 -- 172.21.15.113:0/1294114506 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff00c019a90 con 0x557f7f5a7a00 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7fefff7fe640 1 -- 172.21.15.113:0/1294114506 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c016c70 con 0x557f7f5a7a00 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 -- 172.21.15.113:0/1294114506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 msgr2=0x557f7f5a7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 0x557f7f5a7330 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ce80 tx=0x7ff00c00e040 comp rx=0 tx=0).stop 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 -- 172.21.15.113:0/1294114506 shutdown_connections 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f5a7a00 0x557f7f5a7330 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59ee80 0x557f7f5a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 --2- 172.21.15.113:0/1294114506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f59e4d0 0x557f7f5b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.982+0000 7ff01e97f740 1 -- 172.21.15.113:0/1294114506 >> 172.21.15.113:0/1294114506 conn(0x557f7f598ba0 msgr2=0x557f7f59a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 -- 172.21.15.113:0/1294114506 shutdown_connections 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 -- 172.21.15.113:0/1294114506 wait complete. 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 -- start start 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 Processor -- start 2024-04-22T07:21:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ac690 con 0x557f7f59ee80 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141adb00 con 0x557f7f59e4d0 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141acd60 con 0x557f7f5a7a00 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01e97f740 10 client.0 _ll_register_callbacks cb 0x7ff01002cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48990/0 (socket says 172.21.15.113:48990) 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 -- 172.21.15.113:0/3666491269 learned_addr learned my addr 172.21.15.113:0/3666491269 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 -- 172.21.15.113:0/3666491269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 msgr2=0x557f7f5ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 -- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 msgr2=0x557f7f5afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.983+0000 7ff01932a640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01932a640 1 -- 172.21.15.113:0/3666491269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7f5abe30 con 0x557f7f59ee80 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01932a640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5ab300 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff010003dc0 tx=0x7ff01002c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405a0fd13021ce05 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7feffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7feffd7fa640 1 -- 172.21.15.113:0/3666491269 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010034760 con 0x557f7f59ee80 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7feffd7fa640 1 -- 172.21.15.113:0/3666491269 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7ff010034900 con 0x557f7f59ee80 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff00c019a90 con 0x557f7f59ee80 2024-04-22T07:21:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7feffd7fa640 1 -- 172.21.15.113:0/3666491269 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff010034b00 con 0x557f7f59ee80 2024-04-22T07:21:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 10 client.17205 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557f7f622470 con 0x557f7f59ee80 2024-04-22T07:21:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7feffd7fa640 1 -- 172.21.15.113:0/3666491269 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff00c019a90 con 0x557f7f59ee80 2024-04-22T07:21:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7feffd7fa640 1 -- 172.21.15.113:0/3666491269 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7ff010058b30 con 0x557f7f59ee80 2024-04-22T07:21:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 10 client.17205 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 10 client.17205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 20 client.17205 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f622b90 0x557f7f624fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 4 client.17205 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.984+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f7f510c10 con 0x557f7f622b90 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f622b90 0x557f7f624fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7ff019b2b640 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f622b90 0x557f7f624fb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3d6f14f599fe315 server_cookie=cbbc2623abea923b in_seq=0 out_seq=0 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7feffd7fa640 10 client.17205 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7feffd7fa640 1 -- 172.21.15.113:0/3666491269 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f7f510c10 con 0x557f7f622b90 2024-04-22T07:21:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7feffd7fa640 10 client.17205 handle_command_reply: tid=0 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7ff01e97f740 1 client.17205 shutdown 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 msgr2=0x557f7f5ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.985+0000 7ff01e97f740 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5ab300 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff010003dc0 tx=0x7ff01002c2d0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 shutdown_connections 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x557f7f622b90 0x557f7f624fb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557f7f5a7a00 0x557f7f5ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7feffd7fa640 0 client.17205 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557f7f59ee80 0x557f7f5ab300 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 --2- 172.21.15.113:0/3666491269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f7f59e4d0 0x557f7f5afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 >> 172.21.15.113:0/3666491269 conn(0x557f7f598ba0 msgr2=0x557f7f5a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 shutdown_connections 2024-04-22T07:21:52.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 1 -- 172.21.15.113:0/3666491269 wait complete. 2024-04-22T07:21:52.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01e97f740 20 client.17205 trim_cache size 0 max 16384 2024-04-22T07:21:52.995 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:21:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff0000a3d20 msgr2=0x7ff0000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff0000a3d20 0x7ff0000a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ab70 tx=0x7ff00c009290 comp rx=0 tx=0).stop 2024-04-22T07:21:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 msgr2=0x7ff01419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 0x7ff01419e5e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c920 tx=0x7ff00400cde0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 shutdown_connections 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff0000a3d20 0x7ff0000a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0141986a0 0x7ff014198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff014107260 0x7ff01419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 --2- 172.21.15.113:0/3557914247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff014102190 0x7ff01419e5e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 >> 172.21.15.113:0/3557914247 conn(0x7ff0140793a0 msgr2=0x7ff0140ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 shutdown_connections 2024-04-22T07:21:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:52.986+0000 7ff01bd8f640 1 -- 172.21.15.113:0/3557914247 wait complete. 2024-04-22T07:21:53.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:52 smithi133 ceph-mon[31198]: pgmap v220: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:53.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:52 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1539931225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:21:53.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:52 smithi154 ceph-mon[31862]: pgmap v220: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:21:53.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:52 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1539931225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:21:53.999 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:21:53.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:21:54.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:21:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.115+0000 7f5aba4d5640 1 -- 172.21.15.113:0/2955902068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4101090 msgr2=0x7f5ab4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.115+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/2955902068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4101090 0x7f5ab4101470 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009900 tx=0x7f5a9c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 -- 172.21.15.113:0/2955902068 shutdown_connections 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/2955902068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab4106160 0x7f5ab410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/2955902068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab4101a40 0x7f5ab4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/2955902068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab4101090 0x7f5ab4101470 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 -- 172.21.15.113:0/2955902068 >> 172.21.15.113:0/2955902068 conn(0x7f5ab4077f50 msgr2=0x7f5ab4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 -- 172.21.15.113:0/2955902068 shutdown_connections 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 -- 172.21.15.113:0/2955902068 wait complete. 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 Processor -- start 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.116+0000 7f5aba4d5640 1 -- start start 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab4101090 0x7f5ab4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab4106160 0x7f5ab4199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 0x7f5ab41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41111e0 con 0x7f5ab419a160 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4111060 con 0x7f5ab4106160 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4111360 con 0x7f5ab4101090 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 0x7f5ab41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 0x7f5ab41a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49016/0 (socket says 172.21.15.113:49016) 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 -- 172.21.15.113:0/1499032162 learned_addr learned my addr 172.21.15.113:0/1499032162 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab4101090 0x7f5ab4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab4106160 0x7f5ab4199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 -- 172.21.15.113:0/1499032162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab4101090 msgr2=0x7f5ab4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab4101090 0x7f5ab4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 -- 172.21.15.113:0/1499032162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab4106160 msgr2=0x7f5ab4199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab4106160 0x7f5ab4199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 -- 172.21.15.113:0/1499032162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41a4ee0 con 0x7f5ab419a160 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab8a72640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 0x7f5ab41a46c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00eaa0 tx=0x7f5aac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab17fa640 1 -- 172.21.15.113:0/1499032162 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00cdb0 con 0x7f5ab419a160 2024-04-22T07:21:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab41a51d0 con 0x7f5ab419a160 2024-04-22T07:21:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab17fa640 1 -- 172.21.15.113:0/1499032162 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aac005020 con 0x7f5ab419a160 2024-04-22T07:21:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5ab17fa640 1 -- 172.21.15.113:0/1499032162 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0116d0 con 0x7f5ab419a160 2024-04-22T07:21:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.117+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a5720 con 0x7f5ab419a160 2024-04-22T07:21:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab17fa640 1 -- 172.21.15.113:0/1499032162 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5aac0051d0 con 0x7f5ab419a160 2024-04-22T07:21:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 Processor -- start 2024-04-22T07:21:54.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 -- start start 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b4800fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 0x55b48010a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b480062f00 con 0x55b480107e10 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b480000590 con 0x55b4800ff290 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5abd19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b48005eec0 con 0x55b4800fe8e0 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 0x55b48010a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 0x55b48010a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49022/0 (socket says 172.21.15.113:49022) 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 -- 172.21.15.113:0/851864963 learned_addr learned my addr 172.21.15.113:0/851864963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab17fa640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5aa00a3d20 0x7f5aa00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b4800fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab8a72640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5aa00a3d20 0x7f5aa00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 -- 172.21.15.113:0/851864963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 msgr2=0x55b4800fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b4800fecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab37fe640 1 -- 172.21.15.113:0/851864963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 msgr2=0x55b480107840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.120+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480107840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5ab37fe640 1 -- 172.21.15.113:0/851864963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47fe3d7e0 con 0x55b480107e10 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5ab17fa640 1 -- 172.21.15.113:0/1499032162 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f5aac0d0b90 con 0x7f5ab419a160 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 0x55b48010a260 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40103a0 tx=0x7f5aa400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccf601792219256 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5ab8a72640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5aa00a3d20 0x7f5aa00a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0ce1f0 tx=0x7f5aac06f480 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5aaa7fc640 1 -- 172.21.15.113:0/851864963 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4018030 con 0x55b480107e10 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5aaa7fc640 1 -- 172.21.15.113:0/851864963 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aa40054f0 con 0x55b480107e10 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.121+0000 7f5aaa7fc640 1 -- 172.21.15.113:0/851864963 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40126b0 con 0x55b480107e10 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- 172.21.15.113:0/851864963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 msgr2=0x55b48010a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 0x55b48010a260 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40103a0 tx=0x7f5aa400e590 comp rx=0 tx=0).stop 2024-04-22T07:21:54.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- 172.21.15.113:0/851864963 shutdown_connections 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b480107e10 0x55b48010a260 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- 172.21.15.113:0/851864963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b4800fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- 172.21.15.113:0/851864963 >> 172.21.15.113:0/851864963 conn(0x55b4800f8fb0 msgr2=0x55b4800f9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- 172.21.15.113:0/851864963 shutdown_connections 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- 172.21.15.113:0/851864963 wait complete. 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- start start 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 Processor -- start 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480104de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480105320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b480105860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b48010ad80 con 0x55b4800ff290 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b48001c580 con 0x55b480107e10 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.122+0000 7f5abd19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b48010af00 con 0x55b4800fe8e0 2024-04-22T07:21:54.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5abd19b740 10 client.0 _ll_register_callbacks cb 0x55b480022650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480105320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480105320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49038/0 (socket says 172.21.15.113:49038) 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 -- 172.21.15.113:0/691261170 learned_addr learned my addr 172.21.15.113:0/691261170 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b480105860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 -- 172.21.15.113:0/691261170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 msgr2=0x55b480104de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480104de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 -- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 msgr2=0x55b480105860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b480105860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 -- 172.21.15.113:0/691261170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47fe9df70 con 0x55b4800ff290 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480105320 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009900 tx=0x7f5a9c02ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881f208546a5cc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5a937fe640 1 -- 172.21.15.113:0/691261170 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c005030 con 0x55b4800ff290 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5a937fe640 1 -- 172.21.15.113:0/691261170 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f5a9c0051d0 con 0x55b4800ff290 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa40054f0 con 0x55b4800ff290 2024-04-22T07:21:54.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5a937fe640 1 -- 172.21.15.113:0/691261170 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c0053d0 con 0x55b4800ff290 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5abd19b740 10 client.17217 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.123+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b47fd35170 con 0x55b4800ff290 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5a937fe640 1 -- 172.21.15.113:0/691261170 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aa40054f0 con 0x55b4800ff290 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5a937fe640 1 -- 172.21.15.113:0/691261170 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f5a9c0038c0 con 0x55b4800ff290 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5abd19b740 10 client.17217 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5abd19b740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5abd19b740 20 client.17217 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5abd19b740 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480118300 0x55b48011a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5abd19b740 4 client.17217 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b480070920 con 0x55b480118300 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480118300 0x55b48011a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5a937fe640 10 client.17217 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.124+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480118300 0x55b48011a750 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8064ce8ab30e46e server_cookie=6e713dc15f5797f7 in_seq=0 out_seq=0 2024-04-22T07:21:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5a937fe640 1 -- 172.21.15.113:0/691261170 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b480070920 con 0x55b480118300 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5a937fe640 10 client.17217 handle_command_reply: tid=0 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 client.17217 shutdown 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 msgr2=0x55b480105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480105320 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009900 tx=0x7f5a9c02ce00 comp rx=0 tx=0).stop 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 shutdown_connections 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480118300 0x55b48011a750 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b480105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480105320 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5a937fe640 0 client.17217 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 --2- 172.21.15.113:0/691261170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 >> 172.21.15.113:0/691261170 conn(0x55b4800f8fb0 msgr2=0x55b4800fad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 shutdown_connections 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.125+0000 7f5abd19b740 1 -- 172.21.15.113:0/691261170 wait complete. 2024-04-22T07:21:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.126+0000 7f5abd19b740 20 client.17217 trim_cache size 0 max 16384 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 Processor -- start 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 -- start start 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 0x55b480107510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480113c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b480107e10 0x55b4801054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41b08f0 con 0x55b4800fe8e0 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41ae1c0 con 0x55b4800ff290 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5abd19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41ac030 con 0x55b480107e10 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 0x55b480107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 0x55b480107510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49046/0 (socket says 172.21.15.113:49046) 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 -- 172.21.15.113:0/3183531352 learned_addr learned my addr 172.21.15.113:0/3183531352 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 -- 172.21.15.113:0/3183531352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b480107e10 msgr2=0x55b4801054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b480107e10 0x55b4801054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 -- 172.21.15.113:0/3183531352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 msgr2=0x55b480113c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.143+0000 7f5ab37fe640 1 -- 172.21.15.113:0/3183531352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b480103330 con 0x55b4800fe8e0 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 0x55b480107510 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400ef90 tx=0x7f5aa400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674fb849285a5251 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/3183531352 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4018040 con 0x55b4800fe8e0 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/3183531352 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aa4019040 con 0x55b4800fe8e0 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5aa9ffb640 1 -- 172.21.15.113:0/3183531352 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40052e0 con 0x55b4800fe8e0 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 -- 172.21.15.113:0/3183531352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 msgr2=0x55b480107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 0x55b480107510 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400ef90 tx=0x7f5aa400ede0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 -- 172.21.15.113:0/3183531352 shutdown_connections 2024-04-22T07:21:54.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b480107e10 0x55b4801054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4800ff290 0x55b480113c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 --2- 172.21.15.113:0/3183531352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800fe8e0 0x55b480107510 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 -- 172.21.15.113:0/3183531352 >> 172.21.15.113:0/3183531352 conn(0x55b4800f8fb0 msgr2=0x55b480104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 -- 172.21.15.113:0/3183531352 shutdown_connections 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.144+0000 7f5abd19b740 1 -- 172.21.15.113:0/3183531352 wait complete. 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 -- start start 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 Processor -- start 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480176a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480176fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b48017b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41ac680 con 0x55b4800ff290 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41adaf0 con 0x55b480107e10 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41acd50 con 0x55b4800fe8e0 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5abd19b740 10 client.0 _ll_register_callbacks cb 0x7f5a9c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:54.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480176fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480176fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49054/0 (socket says 172.21.15.113:49054) 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5ab3fff640 1 -- 172.21.15.113:0/1886963553 learned_addr learned my addr 172.21.15.113:0/1886963553 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b48017b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.145+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480176a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5ab3fff640 1 -- 172.21.15.113:0/1886963553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 msgr2=0x55b480176a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480176a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5ab3fff640 1 -- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 msgr2=0x55b48017b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b48017b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5ab3fff640 1 -- 172.21.15.113:0/1886963553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b48017bf40 con 0x55b4800ff290 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5ab3fff640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480176fc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c02c170 tx=0x7f5a9c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80c47bb7c457f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5a937fe640 1 -- 172.21.15.113:0/1886963553 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c058060 con 0x55b4800ff290 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5a937fe640 1 -- 172.21.15.113:0/1886963553 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f5a9c0035f0 con 0x55b4800ff290 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa4019040 con 0x55b4800ff290 2024-04-22T07:21:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5a937fe640 1 -- 172.21.15.113:0/1886963553 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c005200 con 0x55b4800ff290 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 10 client.17223 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b480183350 con 0x55b4800ff290 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5a937fe640 1 -- 172.21.15.113:0/1886963553 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5aa4019040 con 0x55b4800ff290 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5a937fe640 1 -- 172.21.15.113:0/1886963553 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f5a9c034510 con 0x55b4800ff290 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 10 client.17223 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 20 client.17223 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480183b00 0x55b480185f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 4 client.17223 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.146+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4800ffc10 con 0x55b480183b00 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.147+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480183b00 0x55b480185f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.147+0000 7f5ab37fe640 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480183b00 0x55b480185f20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f6eec4a192375 server_cookie=6e0c747e2f37df8c in_seq=0 out_seq=0 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.147+0000 7f5a937fe640 10 client.17223 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5a937fe640 1 -- 172.21.15.113:0/1886963553 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4800ffc10 con 0x55b480183b00 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5a937fe640 10 client.17223 handle_command_reply: tid=0 2024-04-22T07:21:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 client.17223 shutdown 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 msgr2=0x55b480176fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480176fc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c02c170 tx=0x7f5a9c009230 comp rx=0 tx=0).stop 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 shutdown_connections 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b480183b00 0x55b480185f20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b480107e10 0x55b48017b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5a937fe640 0 client.17223 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4800ff290 0x55b480176fc0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 --2- 172.21.15.113:0/1886963553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4800fe8e0 0x55b480176a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 >> 172.21.15.113:0/1886963553 conn(0x55b4800f8fb0 msgr2=0x55b4800f8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 shutdown_connections 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.148+0000 7f5abd19b740 1 -- 172.21.15.113:0/1886963553 wait complete. 2024-04-22T07:21:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5abd19b740 20 client.17223 trim_cache size 0 max 16384 2024-04-22T07:21:54.154 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:21:54.154 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:21:54.154 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "b24f99e7-435f-4b44-b4cd-af4e27390288", 2024-04-22T07:21:54.154 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:21:54.154 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5aa00a3d20 msgr2=0x7f5aa00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5aa00a3d20 0x7f5aa00a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0ce1f0 tx=0x7f5aac06f480 comp rx=0 tx=0).stop 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 msgr2=0x7f5ab41a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 0x7f5ab41a46c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00eaa0 tx=0x7f5aac00ef60 comp rx=0 tx=0).stop 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 shutdown_connections 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5aa00a3d20 0x7f5aa00a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ab419a160 0x7f5ab41a46c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab4106160 0x7f5ab4199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 --2- 172.21.15.113:0/1499032162 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab4101090 0x7f5ab4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 >> 172.21.15.113:0/1499032162 conn(0x7f5ab4077f50 msgr2=0x7f5ab40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 shutdown_connections 2024-04-22T07:21:54.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.149+0000 7f5aba4d5640 1 -- 172.21.15.113:0/1499032162 wait complete. 2024-04-22T07:21:54.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:21:54.159 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 -- 172.21.15.113:0/3794007993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9038101090 msgr2=0x7f9038101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3794007993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9038101090 0x7f9038101470 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9028009900 tx=0x7f902802bc30 comp rx=0 tx=0).stop 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 -- 172.21.15.113:0/3794007993 shutdown_connections 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3794007993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 0x7f903810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3794007993 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101a40 0x7f9038105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3794007993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9038101090 0x7f9038101470 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 -- 172.21.15.113:0/3794007993 >> 172.21.15.113:0/3794007993 conn(0x7f90380771d0 msgr2=0x7f90380775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 -- 172.21.15.113:0/3794007993 shutdown_connections 2024-04-22T07:21:54.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.260+0000 7f903d0eb640 1 -- 172.21.15.113:0/3794007993 wait complete. 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 Processor -- start 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 -- start start 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101090 0x7f9038199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 0x7f9038199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 0x7f90381a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9038111200 con 0x7f903819a180 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9038111080 con 0x7f9038106160 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903d0eb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9038111380 con 0x7f9038101090 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 0x7f90381a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 0x7f90381a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49062/0 (socket says 172.21.15.113:49062) 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 -- 172.21.15.113:0/3638868957 learned_addr learned my addr 172.21.15.113:0/3638868957 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f9036d9d640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101090 0x7f9038199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f902ffff640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 0x7f9038199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 -- 172.21.15.113:0/3638868957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101090 msgr2=0x7f9038199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101090 0x7f9038199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 -- 172.21.15.113:0/3638868957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 msgr2=0x7f9038199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 0x7f9038199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f903759e640 1 -- 172.21.15.113:0/3638868957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90381a4f00 con 0x7f903819a180 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f9036d9d640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101090 0x7f9038199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.261+0000 7f902ffff640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 0x7f9038199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.262+0000 7f903759e640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 0x7f90381a46e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9030009fd0 tx=0x7f903000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.262+0000 7f9034d99640 1 -- 172.21.15.113:0/3638868957 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030017070 con 0x7f903819a180 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.262+0000 7f9034d99640 1 -- 172.21.15.113:0/3638868957 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9030003e40 con 0x7f903819a180 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.262+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90381a51f0 con 0x7f903819a180 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.262+0000 7f9034d99640 1 -- 172.21.15.113:0/3638868957 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030004820 con 0x7f903819a180 2024-04-22T07:21:54.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.262+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90381a56f0 con 0x7f903819a180 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.264+0000 7f9034d99640 1 -- 172.21.15.113:0/3638868957 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9030003970 con 0x7f903819a180 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.264+0000 7f903fdbb740 1 Processor -- start 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.264+0000 7f903fdbb740 1 -- start start 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fd4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fdfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903fdbb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645d9f391e0 con 0x5645d9fdda80 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903fdbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5645d9ed6390 con 0x5645d9fd4ea0 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903fdbb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5645d9f351a0 con 0x5645d9fd4580 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f9034d99640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90240a3d20 0x7f90240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fdfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fdfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49078/0 (socket says 172.21.15.113:49078) 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 -- 172.21.15.113:0/2282179000 learned_addr learned my addr 172.21.15.113:0/2282179000 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fd4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f9036d9d640 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903759e640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90240a3d20 0x7f90240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f9034d99640 1 -- 172.21.15.113:0/3638868957 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f90300d0850 con 0x7f903819a180 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 -- 172.21.15.113:0/2282179000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 msgr2=0x5645d9fd4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fd4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 -- 172.21.15.113:0/2282179000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4ea0 msgr2=0x5645d9fdd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 -- 172.21.15.113:0/2282179000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d9d157d0 con 0x5645d9fdda80 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f902ffff640 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fdfe70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9020014a20 tx=0x7f902000c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a331ce7ba92762f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.265+0000 7f903759e640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90240a3d20 0x7f90240a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f903009c350 tx=0x7f9030091ae0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f902d7fa640 1 -- 172.21.15.113:0/2282179000 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200142d0 con 0x5645d9fdda80 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f902d7fa640 1 -- 172.21.15.113:0/2282179000 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f90200054f0 con 0x5645d9fdda80 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f902d7fa640 1 -- 172.21.15.113:0/2282179000 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902000f760 con 0x5645d9fdda80 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 -- 172.21.15.113:0/2282179000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 msgr2=0x5645d9fdfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fdfe70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9020014a20 tx=0x7f902000c540 comp rx=0 tx=0).stop 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 -- 172.21.15.113:0/2282179000 shutdown_connections 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fdfe70 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 --2- 172.21.15.113:0/2282179000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fd4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 -- 172.21.15.113:0/2282179000 >> 172.21.15.113:0/2282179000 conn(0x5645d9fcecb0 msgr2=0x5645d9fcf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 -- 172.21.15.113:0/2282179000 shutdown_connections 2024-04-22T07:21:54.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.266+0000 7f903fdbb740 1 -- 172.21.15.113:0/2282179000 wait complete. 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 -- start start 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 Processor -- start 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fdaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5645d9fe0810 con 0x5645d9fd4ea0 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5645d9ef22f0 con 0x5645d9fdda80 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5645d9fe0990 con 0x5645d9fd4580 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f903fdbb740 10 client.0 _ll_register_callbacks cb 0x5645d9eff580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49080/0 (socket says 172.21.15.113:49080) 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 -- 172.21.15.113:0/1843501804 learned_addr learned my addr 172.21.15.113:0/1843501804 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f902ffff640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f902ffff640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fdaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 -- 172.21.15.113:0/1843501804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 msgr2=0x5645d9fdaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fdaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 -- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 msgr2=0x5645d9fdb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.267+0000 7f9036d9d640 1 -- 172.21.15.113:0/1843501804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d9c4b3f0 con 0x5645d9fd4ea0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f9036d9d640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdafd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f902802c170 tx=0x7f9028005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2799de922f5fb81f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f901e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f901e7fc640 1 -- 172.21.15.113:0/1843501804 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028005030 con 0x5645d9fd4ea0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f901e7fc640 1 -- 172.21.15.113:0/1843501804 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f90280051d0 con 0x5645d9fd4ea0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90200054f0 con 0x5645d9fd4ea0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f901e7fc640 1 -- 172.21.15.113:0/1843501804 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f90280053d0 con 0x5645d9fd4ea0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 10 client.17235 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5645d9c0b0f0 con 0x5645d9fd4ea0 2024-04-22T07:21:54.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f901e7fc640 1 -- 172.21.15.113:0/1843501804 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f90200054f0 con 0x5645d9fd4ea0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f901e7fc640 1 -- 172.21.15.113:0/1843501804 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f9028003780 con 0x5645d9fd4ea0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 10 client.17235 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 20 client.17235 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645d9feff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 4 client.17235 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.268+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d9f46c00 con 0x5645d9fedae0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.269+0000 7f902ffff640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645d9feff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.269+0000 7f901e7fc640 10 client.17235 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.269+0000 7f902ffff640 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645d9feff00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148a8158d03d9af server_cookie=b1d7d8b1dc4a7d11 in_seq=0 out_seq=0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f901e7fc640 1 -- 172.21.15.113:0/1843501804 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645d9f46c00 con 0x5645d9fedae0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f901e7fc640 10 client.17235 handle_command_reply: tid=0 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 client.17235 shutdown 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 msgr2=0x5645d9fdafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdafd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f902802c170 tx=0x7f9028005a20 comp rx=0 tx=0).stop 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 shutdown_connections 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645d9feff00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f901e7fc640 0 client.17235 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4ea0 0x5645d9fdafd0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 --2- 172.21.15.113:0/1843501804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4580 0x5645d9fdaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 >> 172.21.15.113:0/1843501804 conn(0x5645d9fcecb0 msgr2=0x5645d9fd0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 shutdown_connections 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 1 -- 172.21.15.113:0/1843501804 wait complete. 2024-04-22T07:21:54.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.270+0000 7f903fdbb740 20 client.17235 trim_cache size 0 max 16384 2024-04-22T07:21:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.287+0000 7f903fdbb740 1 Processor -- start 2024-04-22T07:21:54.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.287+0000 7f903fdbb740 1 -- start start 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 0x5645d9fd99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fe96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f903fdbb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90381b07e0 con 0x5645d9fd4580 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f903fdbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90381ae0b0 con 0x5645d9fdda80 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f903fdbb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90381abf20 con 0x5645d9fd4ea0 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 0x5645d9fd99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 0x5645d9fd99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49084/0 (socket says 172.21.15.113:49084) 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 -- 172.21.15.113:0/48300718 learned_addr learned my addr 172.21.15.113:0/48300718 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 -- 172.21.15.113:0/48300718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 msgr2=0x5645d9fe96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fe96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 -- 172.21.15.113:0/48300718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 msgr2=0x5645d9fdb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 -- 172.21.15.113:0/48300718 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d9fce480 con 0x5645d9fd4580 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902ffff640 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 0x5645d9fd99d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9020004c30 tx=0x7f9020004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b20388fb46eb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902cff9640 1 -- 172.21.15.113:0/48300718 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020031020 con 0x5645d9fd4580 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902cff9640 1 -- 172.21.15.113:0/48300718 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f902000cde0 con 0x5645d9fd4580 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.288+0000 7f902cff9640 1 -- 172.21.15.113:0/48300718 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902002d730 con 0x5645d9fd4580 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 -- 172.21.15.113:0/48300718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 msgr2=0x5645d9fd99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 0x5645d9fd99d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9020004c30 tx=0x7f9020004ce0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 -- 172.21.15.113:0/48300718 shutdown_connections 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fdda80 0x5645d9fdb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fe96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 --2- 172.21.15.113:0/48300718 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fd4580 0x5645d9fd99d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 -- 172.21.15.113:0/48300718 >> 172.21.15.113:0/48300718 conn(0x5645d9fcecb0 msgr2=0x5645d9fcdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 -- 172.21.15.113:0/48300718 shutdown_connections 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 -- 172.21.15.113:0/48300718 wait complete. 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 -- start start 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.289+0000 7f903fdbb740 1 Processor -- start 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4580 0x5645d9fd7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fd7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fd81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90381ac570 con 0x5645d9fdda80 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90381ad9e0 con 0x5645d9fd4580 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f90381acc40 con 0x5645d9fd4ea0 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 10 client.0 _ll_register_callbacks cb 0x7f902802cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fd81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fd81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49086/0 (socket says 172.21.15.113:49086) 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 -- 172.21.15.113:0/3883970783 learned_addr learned my addr 172.21.15.113:0/3883970783 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:21:54.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f9036d9d640 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fd7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 -- 172.21.15.113:0/3883970783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 msgr2=0x5645d9fd7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fd7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 -- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4580 msgr2=0x5645d9fd7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4580 0x5645d9fd7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 -- 172.21.15.113:0/3883970783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645da053040 con 0x5645d9fdda80 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f902ffff640 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fd81a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f902000c2f0 tx=0x7f9020005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7acf4c9da7091 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f901e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f901e7fc640 1 -- 172.21.15.113:0/3883970783 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020031030 con 0x5645d9fdda80 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f901e7fc640 1 -- 172.21.15.113:0/3883970783 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8943+0+0 (secure 0 0 0) 0x7f9020004680 con 0x5645d9fdda80 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902000cde0 con 0x5645d9fdda80 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f901e7fc640 1 -- 172.21.15.113:0/3883970783 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9020004880 con 0x5645d9fdda80 2024-04-22T07:21:54.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 10 client.17241 fetch_fsmap learned FSMap version 52 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5645da0542d0 con 0x5645d9fdda80 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.290+0000 7f901e7fc640 1 -- 172.21.15.113:0/3883970783 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f902000cde0 con 0x5645d9fdda80 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f901e7fc640 1 -- 172.21.15.113:0/3883970783 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f9020004de0 con 0x5645d9fdda80 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f903fdbb740 10 client.17241 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f903fdbb740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f903fdbb740 20 client.17241 populate_metadata read hostname 'smithi113' 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f903fdbb740 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645da056830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f903fdbb740 4 client.17241 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645d9f40a70 con 0x5645d9fedae0 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f9036d9d640 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645da056830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f901e7fc640 10 client.17241 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.291+0000 7f9036d9d640 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645da056830 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf9a8f937a5a4b1 server_cookie=f60c0f874b72a230 in_seq=0 out_seq=0 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f901e7fc640 1 -- 172.21.15.113:0/3883970783 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5645d9f40a70 con 0x5645d9fedae0 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f901e7fc640 10 client.17241 handle_command_reply: tid=0 2024-04-22T07:21:54.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 client.17241 shutdown 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 msgr2=0x5645d9fd81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fd81a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f902000c2f0 tx=0x7f9020005970 comp rx=0 tx=0).stop 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 shutdown_connections 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5645d9fedae0 0x5645da056830 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5645d9fdda80 0x5645d9fd81a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f901e7fc640 0 client.17241 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5645d9fd4ea0 0x5645d9fd7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 --2- 172.21.15.113:0/3883970783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5645d9fd4580 0x5645d9fd7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 >> 172.21.15.113:0/3883970783 conn(0x5645d9fcecb0 msgr2=0x5645d9fcfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 shutdown_connections 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 1 -- 172.21.15.113:0/3883970783 wait complete. 2024-04-22T07:21:54.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.292+0000 7f903fdbb740 20 client.17241 trim_cache size 0 max 16384 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: "b24f99e7-435f-4b44-b4cd-af4e27390288": { 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "b24f99e7-435f-4b44-b4cd-af4e27390288", 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:21:54.298 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:21:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90240a3d20 msgr2=0x7f90240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90240a3d20 0x7f90240a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f903009c350 tx=0x7f9030091ae0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 msgr2=0x7f90381a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:21:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 0x7f90381a46e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9030009fd0 tx=0x7f903000b260 comp rx=0 tx=0).stop 2024-04-22T07:21:54.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 shutdown_connections 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f90240a3d20 0x7f90240a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f903819a180 0x7f90381a46e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9038106160 0x7f9038199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 --2- 172.21.15.113:0/3638868957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9038101090 0x7f9038199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 >> 172.21.15.113:0/3638868957 conn(0x7f90380771d0 msgr2=0x7f90380fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 shutdown_connections 2024-04-22T07:21:54.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:21:54.293+0000 7f903d0eb640 1 -- 172.21.15.113:0/3638868957 wait complete. 2024-04-22T07:21:54.302 INFO:tasks.cephfs.filesystem:scrub status for tag:b24f99e7-435f-4b44-b4cd-af4e27390288 - {'path': '/', 'tag': 'b24f99e7-435f-4b44-b4cd-af4e27390288', 'options': 'recursive,force'} 2024-04-22T07:21:55.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:54 smithi133 ceph-mon[31198]: pgmap v221: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:55.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:54 smithi154 ceph-mon[31862]: pgmap v221: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:54 smithi113 ceph-mon[27933]: pgmap v221: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:21:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:55 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:21:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:55 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:55 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:21:56.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:55 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:56.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:55 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:21:56.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:55 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:56.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:55 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:21:56.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:55 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:56.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:55 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:21:56.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:55 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:56.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:55 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:21:56.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:55 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:21:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:56 smithi133 ceph-mon[31198]: pgmap v222: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:21:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:56 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:21:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:56 smithi154 ceph-mon[31862]: pgmap v222: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:21:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:56 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:21:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:56 smithi113 ceph-mon[27933]: pgmap v222: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:21:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:56 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:21:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:21:58 smithi133 ceph-mon[31198]: pgmap v223: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:21:59.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:21:58 smithi154 ceph-mon[31862]: pgmap v223: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:21:59.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:21:58 smithi113 ceph-mon[27933]: pgmap v223: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:01.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:00 smithi133 ceph-mon[31198]: pgmap v224: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:01.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:00 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:22:01.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:00 smithi154 ceph-mon[31862]: pgmap v224: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:01.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:00 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:22:01.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:00 smithi113 ceph-mon[27933]: pgmap v224: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:01.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:00 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:22:02.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:01 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:22:02.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:01 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:22:02.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:01 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:22:02.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:01 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:22:02.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:01 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:22:02.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:01 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:22:02.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:01 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:22:02.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:01 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:22:02.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:01 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:22:03.316 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:02 smithi133 ceph-mon[31198]: pgmap v225: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:03.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:02 smithi154 ceph-mon[31862]: pgmap v225: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:03.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:02 smithi113 ceph-mon[27933]: pgmap v225: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:05.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:04 smithi133 ceph-mon[31198]: pgmap v226: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:05.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:04 smithi154 ceph-mon[31862]: pgmap v226: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:05.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:04 smithi113 ceph-mon[27933]: pgmap v226: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:07.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:06 smithi133 ceph-mon[31198]: pgmap v227: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:22:07.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:06 smithi154 ceph-mon[31862]: pgmap v227: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:22:07.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:06 smithi113 ceph-mon[27933]: pgmap v227: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:22:09.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:08 smithi133 ceph-mon[31198]: pgmap v228: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:22:09.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:08 smithi154 ceph-mon[31862]: pgmap v228: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:22:09.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:08 smithi113 ceph-mon[27933]: pgmap v228: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:22:11.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:10 smithi133 ceph-mon[31198]: pgmap v229: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:22:11.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:10 smithi154 ceph-mon[31862]: pgmap v229: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:22:11.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:10 smithi113 ceph-mon[27933]: pgmap v229: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:22:12.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:11 smithi133 ceph-mon[31198]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T07:22:12.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:11 smithi154 ceph-mon[31862]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T07:22:12.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:11 smithi113 ceph-mon[27933]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T07:22:13.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:12 smithi133 ceph-mon[31198]: pgmap v231: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:22:13.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:12 smithi154 ceph-mon[31862]: pgmap v231: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:22:13.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:12 smithi113 ceph-mon[27933]: pgmap v231: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:22:15.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:14 smithi133 ceph-mon[31198]: pgmap v232: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:22:15.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:14 smithi154 ceph-mon[31862]: pgmap v232: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:22:15.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:14 smithi113 ceph-mon[27933]: pgmap v232: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:22:17.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:16 smithi133 ceph-mon[31198]: pgmap v233: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:17.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:16 smithi154 ceph-mon[31862]: pgmap v233: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:17.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:16 smithi113 ceph-mon[27933]: pgmap v233: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:19.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:18 smithi133 ceph-mon[31198]: pgmap v234: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:19.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:18 smithi133 ceph-mon[31198]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T07:22:19.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:18 smithi154 ceph-mon[31862]: pgmap v234: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:19.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:18 smithi154 ceph-mon[31862]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T07:22:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:18 smithi113 ceph-mon[27933]: pgmap v234: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:19.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:18 smithi113 ceph-mon[27933]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T07:22:21.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:21 smithi133 ceph-mon[31198]: pgmap v236: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 120 B/s wr, 4 op/s 2024-04-22T07:22:21.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:21 smithi154 ceph-mon[31862]: pgmap v236: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 120 B/s wr, 4 op/s 2024-04-22T07:22:21.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:21 smithi113 ceph-mon[27933]: pgmap v236: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 120 B/s wr, 4 op/s 2024-04-22T07:22:23.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:23 smithi133 ceph-mon[31198]: pgmap v237: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:23.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:23 smithi154 ceph-mon[31862]: pgmap v237: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:23.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:23 smithi113 ceph-mon[27933]: pgmap v237: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:24.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:22:24.304 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:22:24.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- 172.21.15.113:0/3244063133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 msgr2=0x7f322010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- 172.21.15.113:0/3244063133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010dd30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3218009ca0 tx=0x7f321802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- 172.21.15.113:0/3244063133 shutdown_connections 2024-04-22T07:22:24.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- 172.21.15.113:0/3244063133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010dd30 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- 172.21.15.113:0/3244063133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3220101a40 0x7f3220105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- 172.21.15.113:0/3244063133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3220101090 0x7f3220101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- 172.21.15.113:0/3244063133 >> 172.21.15.113:0/3244063133 conn(0x7f32200771d0 msgr2=0x7f32200775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- 172.21.15.113:0/3244063133 shutdown_connections 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- 172.21.15.113:0/3244063133 wait complete. 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 Processor -- start 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- start start 2024-04-22T07:22:24.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3220101090 0x7f322010e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322010f140 0x7f32201a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3220112300 con 0x7f3220106160 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3220112180 con 0x7f322010f140 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.420+0000 7f3225df4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3220112480 con 0x7f3220101090 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44660/0 (socket says 172.21.15.113:44660) 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 -- 172.21.15.113:0/1116337996 learned_addr learned my addr 172.21.15.113:0/1116337996 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f321f7fe640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3220101090 0x7f322010e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f321ffff640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322010f140 0x7f32201a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 -- 172.21.15.113:0/1116337996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3220101090 msgr2=0x7f322010e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3220101090 0x7f322010e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 -- 172.21.15.113:0/1116337996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322010f140 msgr2=0x7f32201a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322010f140 0x7f32201a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 -- 172.21.15.113:0/1116337996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32201a7070 con 0x7f3220106160 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3217fff640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010ec00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f320800e9f0 tx=0x7f320800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f321d7fa640 1 -- 172.21.15.113:0/1116337996 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800ccc0 con 0x7f3220106160 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32201a7360 con 0x7f3220106160 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f321d7fa640 1 -- 172.21.15.113:0/1116337996 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32080054f0 con 0x7f3220106160 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f321d7fa640 1 -- 172.21.15.113:0/1116337996 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208010690 con 0x7f3220106160 2024-04-22T07:22:24.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.421+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32201a78b0 con 0x7f3220106160 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.423+0000 7f321d7fa640 1 -- 172.21.15.113:0/1116337996 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f32080046c0 con 0x7f3220106160 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 Processor -- start 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 -- start start 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 0x557ab83f4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4ea0 0x557ab83fd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557ab83591e0 con 0x557ab83f4580 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557ab82f6390 con 0x557ab83fda80 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f32289e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557ab83551a0 con 0x557ab83f4ea0 2024-04-22T07:22:24.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321d7fa640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f320c0a3d20 0x7f320c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 0x557ab83f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 0x557ab83f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44672/0 (socket says 172.21.15.113:44672) 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321ffff640 1 -- 172.21.15.113:0/3464978876 learned_addr learned my addr 172.21.15.113:0/3464978876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321ffff640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f320c0a3d20 0x7f320c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321f7fe640 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4ea0 0x557ab83fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f3217fff640 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.424+0000 7f321d7fa640 1 -- 172.21.15.113:0/1116337996 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f32080d0de0 con 0x7f3220106160 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 -- 172.21.15.113:0/3464978876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4ea0 msgr2=0x557ab83fd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4ea0 0x557ab83fd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 -- 172.21.15.113:0/3464978876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 msgr2=0x557ab83ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 -- 172.21.15.113:0/3464978876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab81357d0 con 0x557ab83f4580 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 0x557ab83f4960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3218009430 tx=0x7f3218032790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e727b1862419b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f321ffff640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f320c0a3d20 0x7f320c0a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3218032540 tx=0x7f3218041000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f3215ffb640 1 -- 172.21.15.113:0/3464978876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218051070 con 0x557ab83f4580 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f3215ffb640 1 -- 172.21.15.113:0/3464978876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3218048070 con 0x557ab83f4580 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.425+0000 7f3215ffb640 1 -- 172.21.15.113:0/3464978876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218039040 con 0x557ab83f4580 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- 172.21.15.113:0/3464978876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 msgr2=0x557ab83f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 0x557ab83f4960 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3218009430 tx=0x7f3218032790 comp rx=0 tx=0).stop 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- 172.21.15.113:0/3464978876 shutdown_connections 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4ea0 0x557ab83fd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- 172.21.15.113:0/3464978876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4580 0x557ab83f4960 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- 172.21.15.113:0/3464978876 >> 172.21.15.113:0/3464978876 conn(0x557ab83eecb0 msgr2=0x557ab83ef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- 172.21.15.113:0/3464978876 shutdown_connections 2024-04-22T07:22:24.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- 172.21.15.113:0/3464978876 wait complete. 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- start start 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 Processor -- start 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4580 0x557ab83fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab83fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.426+0000 7f32289e7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557ab8400810 con 0x557ab83f4ea0 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f32289e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557ab83122f0 con 0x557ab83fda80 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f32289e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557ab8400990 con 0x557ab83f4580 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f32289e7740 10 client.0 _ll_register_callbacks cb 0x557ab831f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab83fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab83fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44676/0 (socket says 172.21.15.113:44676) 2024-04-22T07:22:24.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 -- 172.21.15.113:0/1427828059 learned_addr learned my addr 172.21.15.113:0/1427828059 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f321f7fe640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4580 0x557ab83fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f321ffff640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 -- 172.21.15.113:0/1427828059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4580 msgr2=0x557ab83fab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4580 0x557ab83fab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 -- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 msgr2=0x557ab83fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83fb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 -- 172.21.15.113:0/1427828059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab806b3f0 con 0x557ab83f4ea0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f321ffff640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f3217fff640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab83fb0c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f32080995b0 tx=0x7f320809c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a09e3e6967d3dad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f31feffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f31feffd640 1 -- 172.21.15.113:0/1427828059 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080b5070 con 0x557ab83f4ea0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f31feffd640 1 -- 172.21.15.113:0/1427828059 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f32080a53c0 con 0x557ab83f4ea0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3218048070 con 0x557ab83f4ea0 2024-04-22T07:22:24.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f31feffd640 1 -- 172.21.15.113:0/1427828059 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f3208063e60 con 0x557ab83f4ea0 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f32289e7740 10 client.17253 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.427+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557ab802b0f0 con 0x557ab83f4ea0 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f31feffd640 1 -- 172.21.15.113:0/1427828059 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3218048070 con 0x557ab83f4ea0 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f31feffd640 1 -- 172.21.15.113:0/1427828059 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f32080ceae0 con 0x557ab83f4ea0 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f32289e7740 10 client.17253 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f32289e7740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f32289e7740 20 client.17253 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f32289e7740 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab840db50 0x557ab840ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f32289e7740 4 client.17253 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ab8366c00 con 0x557ab840db50 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f321f7fe640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab840db50 0x557ab840ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f31feffd640 10 client.17253 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.428+0000 7f321f7fe640 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab840db50 0x557ab840ff60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6469ea2d3cbcd server_cookie=626107ac96fb2a85 in_seq=0 out_seq=0 2024-04-22T07:22:24.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f31feffd640 1 -- 172.21.15.113:0/1427828059 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ab8366c00 con 0x557ab840db50 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f31feffd640 10 client.17253 handle_command_reply: tid=0 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 client.17253 shutdown 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 msgr2=0x557ab83fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab83fb0c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f32080995b0 tx=0x7f320809c9b0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 shutdown_connections 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab840db50 0x557ab840ff60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83fda80 0x557ab83fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab83fb0c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 --2- 172.21.15.113:0/1427828059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83f4580 0x557ab83fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f31feffd640 0 client.17253 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.429+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 >> 172.21.15.113:0/1427828059 conn(0x557ab83eecb0 msgr2=0x557ab83f0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.430+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 shutdown_connections 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.430+0000 7f32289e7740 1 -- 172.21.15.113:0/1427828059 wait complete. 2024-04-22T07:22:24.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.430+0000 7f32289e7740 20 client.17253 trim_cache size 0 max 16384 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 Processor -- start 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 -- start start 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab83fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab84096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 0x557ab83f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32201b2a50 con 0x557ab83f4ea0 2024-04-22T07:22:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32201b0320 con 0x557ab83f4580 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f32289e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32201ae190 con 0x557ab83fda80 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab84096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab84096d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44686/0 (socket says 172.21.15.113:44686) 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 -- 172.21.15.113:0/917996202 learned_addr learned my addr 172.21.15.113:0/917996202 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 -- 172.21.15.113:0/917996202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 msgr2=0x557ab83f9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 0x557ab83f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f321f7fe640 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab83fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 -- 172.21.15.113:0/917996202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 msgr2=0x557ab83fd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab83fd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f3217fff640 1 -- 172.21.15.113:0/917996202 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab83f90d0 con 0x557ab83f4ea0 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.447+0000 7f321f7fe640 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab83fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f3217fff640 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab84096d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f320809cb70 tx=0x7f32080096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b7a2ffab814603 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32157fa640 1 -- 172.21.15.113:0/917996202 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208081070 con 0x557ab83f4ea0 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32157fa640 1 -- 172.21.15.113:0/917996202 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32080056a0 con 0x557ab83f4ea0 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32157fa640 1 -- 172.21.15.113:0/917996202 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080b5070 con 0x557ab83f4ea0 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 -- 172.21.15.113:0/917996202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 msgr2=0x557ab84096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab84096d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f320809cb70 tx=0x7f32080096c0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 -- 172.21.15.113:0/917996202 shutdown_connections 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 0x557ab83f9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab84096d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 --2- 172.21.15.113:0/917996202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab83fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 -- 172.21.15.113:0/917996202 >> 172.21.15.113:0/917996202 conn(0x557ab83eecb0 msgr2=0x557ab83f0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 -- 172.21.15.113:0/917996202 shutdown_connections 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.448+0000 7f32289e7740 1 -- 172.21.15.113:0/917996202 wait complete. 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 -- start start 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 Processor -- start 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab8401850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab8401d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 0x557ab84022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32201ae7e0 con 0x557ab83f4ea0 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32201afc50 con 0x557ab83f4580 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32201aeeb0 con 0x557ab83fda80 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f32289e7740 10 client.0 _ll_register_callbacks cb 0x7f320800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab8401d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab8401d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44688/0 (socket says 172.21.15.113:44688) 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 -- 172.21.15.113:0/3547529117 learned_addr learned my addr 172.21.15.113:0/3547529117 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f321f7fe640 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab8401850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 -- 172.21.15.113:0/3547529117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 msgr2=0x557ab84022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 0x557ab84022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 -- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 msgr2=0x557ab8401850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.449+0000 7f3217fff640 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab8401850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f3217fff640 1 -- 172.21.15.113:0/3547529117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab84732f0 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f3217fff640 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab8401d90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f320800cf30 tx=0x7f32080a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bfb2a40bd5e52b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f31feffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f31feffd640 1 -- 172.21.15.113:0/3547529117 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080b5060 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f31feffd640 1 -- 172.21.15.113:0/3547529117 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f3208061d60 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32080056a0 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f31feffd640 1 -- 172.21.15.113:0/3547529117 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f3208081050 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 10 client.17259 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557ab8474510 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f31feffd640 1 -- 172.21.15.113:0/3547529117 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32080056a0 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f31feffd640 1 -- 172.21.15.113:0/3547529117 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f320809baa0 con 0x557ab83f4ea0 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 10 client.17259 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:24.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 20 client.17259 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab84749c0 0x557ab8476de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 4 client.17259 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.450+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ab8360a70 con 0x557ab84749c0 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f321f7fe640 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab84749c0 0x557ab8476de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f31feffd640 10 client.17259 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f321f7fe640 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab84749c0 0x557ab8476de0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fd2fc996a4a55f server_cookie=3dd808ab6ce306cf in_seq=0 out_seq=0 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f31feffd640 1 -- 172.21.15.113:0/3547529117 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557ab8360a70 con 0x557ab84749c0 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f31feffd640 10 client.17259 handle_command_reply: tid=0 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 client.17259 shutdown 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 msgr2=0x557ab8401d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab8401d90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f320800cf30 tx=0x7f32080a55f0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 shutdown_connections 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557ab84749c0 0x557ab8476de0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557ab83fda80 0x557ab84022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557ab83f4ea0 0x557ab8401d90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f31feffd640 0 client.17259 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 --2- 172.21.15.113:0/3547529117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557ab83f4580 0x557ab8401850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 >> 172.21.15.113:0/3547529117 conn(0x557ab83eecb0 msgr2=0x557ab83fab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 shutdown_connections 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 1 -- 172.21.15.113:0/3547529117 wait complete. 2024-04-22T07:22:24.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.451+0000 7f32289e7740 20 client.17259 trim_cache size 0 max 16384 2024-04-22T07:22:24.457 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:22:24.457 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:22:24.457 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:22:24.457 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f320c0a3d20 msgr2=0x7f320c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f320c0a3d20 0x7f320c0a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3218032540 tx=0x7f3218041000 comp rx=0 tx=0).stop 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 msgr2=0x7f322010ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010ec00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f320800e9f0 tx=0x7f320800eeb0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 shutdown_connections 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f320c0a3d20 0x7f320c0a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322010f140 0x7f32201a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3220106160 0x7f322010ec00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 --2- 172.21.15.113:0/1116337996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3220101090 0x7f322010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 >> 172.21.15.113:0/1116337996 conn(0x7f32200771d0 msgr2=0x7f32200fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 shutdown_connections 2024-04-22T07:22:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.452+0000 7f3225df4640 1 -- 172.21.15.113:0/1116337996 wait complete. 2024-04-22T07:22:24.461 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:22:24.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:22:24.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 -- 172.21.15.113:0/3377027178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c1072b0 msgr2=0x7f9d8c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 --2- 172.21.15.113:0/3377027178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c1072b0 0x7f9d8c10eec0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009ca0 tx=0x7f9d8402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 -- 172.21.15.113:0/3377027178 shutdown_connections 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 --2- 172.21.15.113:0/3377027178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c1072b0 0x7f9d8c10eec0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 --2- 172.21.15.113:0/3377027178 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c101110 0x7f9d8c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 --2- 172.21.15.113:0/3377027178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d8c100760 0x7f9d8c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 -- 172.21.15.113:0/3377027178 >> 172.21.15.113:0/3377027178 conn(0x7f9d8c077480 msgr2=0x7f9d8c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 -- 172.21.15.113:0/3377027178 shutdown_connections 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.562+0000 7f9d93e81640 1 -- 172.21.15.113:0/3377027178 wait complete. 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 Processor -- start 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 -- start start 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c100760 0x7f9d8c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d8c1072b0 0x7f9d8c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 0x7f9d8c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c112390 con 0x7f9d8c198670 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c112210 con 0x7f9d8c1072b0 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d93e81640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c112510 con 0x7f9d8c100760 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 0x7f9d8c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 0x7f9d8c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44700/0 (socket says 172.21.15.113:44700) 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 -- 172.21.15.113:0/2825228016 learned_addr learned my addr 172.21.15.113:0/2825228016 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9141c640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d8c1072b0 0x7f9d8c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d91c1d640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c100760 0x7f9d8c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 -- 172.21.15.113:0/2825228016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c100760 msgr2=0x7f9d8c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c100760 0x7f9d8c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 -- 172.21.15.113:0/2825228016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d8c1072b0 msgr2=0x7f9d8c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d8c1072b0 0x7f9d8c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.563+0000 7f9d9241e640 1 -- 172.21.15.113:0/2825228016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c199270 con 0x7f9d8c198670 2024-04-22T07:22:24.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.564+0000 7f9d9241e640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 0x7f9d8c198af0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9d8402c700 tx=0x7f9d84035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.564+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84039040 con 0x7f9d8c198670 2024-04-22T07:22:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.564+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9d84044070 con 0x7f9d8c198670 2024-04-22T07:22:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.564+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c199500 con 0x7f9d8c198670 2024-04-22T07:22:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.564+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840043e0 con 0x7f9d8c198670 2024-04-22T07:22:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.564+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c1a5690 con 0x7f9d8c198670 2024-04-22T07:22:24.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.565+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c103810 con 0x7f9d8c198670 2024-04-22T07:22:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.566+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f9d84004580 con 0x7f9d8c198670 2024-04-22T07:22:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.566+0000 7f9d82ffd640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d780a3da0 0x7f9d780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.566+0000 7f9d91c1d640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d780a3da0 0x7f9d780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.566+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f9d840f2e60 con 0x7f9d8c198670 2024-04-22T07:22:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.567+0000 7f9d91c1d640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d780a3da0 0x7f9d780a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005fd0 tx=0x7f9d8800b590 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.571+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d840bf470 con 0x7f9d8c198670 2024-04-22T07:22:24.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.785+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d8c101110 con 0x7f9d8c198670 2024-04-22T07:22:24.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.786+0000 7f9d82ffd640 1 -- 172.21.15.113:0/2825228016 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7f9d840bed70 con 0x7f9d8c198670 2024-04-22T07:22:24.784 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:22:24.784 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:22:24.784 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d780a3da0 msgr2=0x7f9d780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d780a3da0 0x7f9d780a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d88005fd0 tx=0x7f9d8800b590 comp rx=0 tx=0).stop 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 msgr2=0x7f9d8c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 0x7f9d8c198af0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9d8402c700 tx=0x7f9d84035d80 comp rx=0 tx=0).stop 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 shutdown_connections 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f9d780a3da0 0x7f9d780a6260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d8c198670 0x7f9d8c198af0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d8c1072b0 0x7f9d8c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 --2- 172.21.15.113:0/2825228016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d8c100760 0x7f9d8c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 >> 172.21.15.113:0/2825228016 conn(0x7f9d8c077480 msgr2=0x7f9d8c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 shutdown_connections 2024-04-22T07:22:24.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.788+0000 7f9d93e81640 1 -- 172.21.15.113:0/2825228016 wait complete. 2024-04-22T07:22:24.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:22:24.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 -- 172.21.15.113:0/1254933842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081072e0 msgr2=0x7f710810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1254933842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081072e0 0x7f710810eeb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7100009ca0 tx=0x7f710002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 -- 172.21.15.113:0/1254933842 shutdown_connections 2024-04-22T07:22:24.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1254933842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081072e0 0x7f710810eeb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1254933842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102bc0 0x7f7108106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1254933842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7108102210 0x7f71081025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 -- 172.21.15.113:0/1254933842 >> 172.21.15.113:0/1254933842 conn(0x7f71080fdab0 msgr2=0x7f71080ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 -- 172.21.15.113:0/1254933842 shutdown_connections 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.899+0000 7f710dc5c640 1 -- 172.21.15.113:0/1254933842 wait complete. 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 Processor -- start 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 -- start start 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102210 0x7f710819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71081072e0 0x7f710819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 0x7f7108198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7108112360 con 0x7f71081986c0 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71081121e0 con 0x7f71081072e0 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f710dc5c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f71081124e0 con 0x7f7108102210 2024-04-22T07:22:24.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f7107fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 0x7f7108198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f7107fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 0x7f7108198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44714/0 (socket says 172.21.15.113:44714) 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.900+0000 7f7107fff640 1 -- 172.21.15.113:0/1468408706 learned_addr learned my addr 172.21.15.113:0/1468408706 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f71077fe640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102210 0x7f710819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7107fff640 1 -- 172.21.15.113:0/1468408706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102210 msgr2=0x7f710819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7107fff640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102210 0x7f710819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7106ffd640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71081072e0 0x7f710819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7107fff640 1 -- 172.21.15.113:0/1468408706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71081072e0 msgr2=0x7f710819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7107fff640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71081072e0 0x7f710819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7107fff640 1 -- 172.21.15.113:0/1468408706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081992c0 con 0x7f71081986c0 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f71077fe640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102210 0x7f710819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:24.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7107fff640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 0x7f7108198b40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f710002c700 tx=0x7f7100035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7104ff9640 1 -- 172.21.15.113:0/1468408706 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100039040 con 0x7f71081986c0 2024-04-22T07:22:24.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7108199550 con 0x7f71081986c0 2024-04-22T07:22:24.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7104ff9640 1 -- 172.21.15.113:0/1468408706 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7100044070 con 0x7f71081986c0 2024-04-22T07:22:24.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f7104ff9640 1 -- 172.21.15.113:0/1468408706 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000043e0 con 0x7f71081986c0 2024-04-22T07:22:24.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.901+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081a5690 con 0x7f71081986c0 2024-04-22T07:22:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.903+0000 7f7104ff9640 1 -- 172.21.15.113:0/1468408706 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7100004580 con 0x7f71081986c0 2024-04-22T07:22:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.903+0000 7f7110851740 1 Processor -- start 2024-04-22T07:22:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.903+0000 7f7110851740 1 -- start start 2024-04-22T07:22:24.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7110851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f94d0 0x5596cf3f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7110851740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f9e80 0x5596cf402430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7110851740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf404e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7110851740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5596cf35e1f0 con 0x5596cf402a00 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7110851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5596cf2fb4b0 con 0x5596cf3f94d0 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7110851740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5596cf35a1b0 con 0x5596cf3f9e80 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7104ff9640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70f40a3d20 0x7f70f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf404e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf404e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44726/0 (socket says 172.21.15.113:44726) 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 -- 172.21.15.113:0/2180878346 learned_addr learned my addr 172.21.15.113:0/2180878346 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7107fff640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70f40a3d20 0x7f70f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f94d0 0x5596cf3f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f71077fe640 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f9e80 0x5596cf402430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7104ff9640 1 -- 172.21.15.113:0/1468408706 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f71000f2fa0 con 0x7f71081986c0 2024-04-22T07:22:24.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 -- 172.21.15.113:0/2180878346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f9e80 msgr2=0x5596cf402430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f9e80 0x5596cf402430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 -- 172.21.15.113:0/2180878346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f94d0 msgr2=0x5596cf3f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f94d0 0x5596cf3f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 -- 172.21.15.113:0/2180878346 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cf2ed1b0 con 0x5596cf402a00 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7107fff640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70f40a3d20 0x7f70f40a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71000f0b50 tx=0x7f71000093f0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf404e50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f70f0015820 tx=0x7f70f00092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a715584239d38a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.904+0000 7f70fdffb640 1 -- 172.21.15.113:0/2180878346 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0015e60 con 0x5596cf402a00 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f70fdffb640 1 -- 172.21.15.113:0/2180878346 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f70f00152f0 con 0x5596cf402a00 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f70fdffb640 1 -- 172.21.15.113:0/2180878346 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0004de0 con 0x5596cf402a00 2024-04-22T07:22:24.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 -- 172.21.15.113:0/2180878346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 msgr2=0x5596cf404e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf404e50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f70f0015820 tx=0x7f70f00092a0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 -- 172.21.15.113:0/2180878346 shutdown_connections 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf404e50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f9e80 0x5596cf402430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 --2- 172.21.15.113:0/2180878346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f94d0 0x5596cf3f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 -- 172.21.15.113:0/2180878346 >> 172.21.15.113:0/2180878346 conn(0x5596cf3f3ba0 msgr2=0x5596cf3f3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 -- 172.21.15.113:0/2180878346 shutdown_connections 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 -- 172.21.15.113:0/2180878346 wait complete. 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.905+0000 7f7110851740 1 -- start start 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 Processor -- start 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f94d0 0x5596cf402320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf3ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5596cf405970 con 0x5596cf402a00 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5596cf317500 con 0x5596cf3f9e80 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5596cf405af0 con 0x5596cf3f94d0 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7110851740 10 client.0 _ll_register_callbacks cb 0x5596cf2e5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf3ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf3ffc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44738/0 (socket says 172.21.15.113:44738) 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 -- 172.21.15.113:0/2117080890 learned_addr learned my addr 172.21.15.113:0/2117080890 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f71077fe640 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 -- 172.21.15.113:0/2117080890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f94d0 msgr2=0x5596cf402320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f94d0 0x5596cf402320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 -- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 msgr2=0x5596cf3ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 -- 172.21.15.113:0/2117080890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cf0703f0 con 0x5596cf402a00 2024-04-22T07:22:24.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f7106ffd640 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf3ffc50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009c20 tx=0x7f70f0015ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a1643fa0f3d1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.906+0000 7f70e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f70e6ffd640 1 -- 172.21.15.113:0/2117080890 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0012790 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f70e6ffd640 1 -- 172.21.15.113:0/2117080890 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f70f00048f0 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f00152f0 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f70e6ffd640 1 -- 172.21.15.113:0/2117080890 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f70f0012930 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 10 client.17277 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5596cf13a7d0 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f70e6ffd640 1 -- 172.21.15.113:0/2117080890 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f70f00152f0 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f70e6ffd640 1 -- 172.21.15.113:0/2117080890 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f70f000c030 con 0x5596cf402a00 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 10 client.17277 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 20 client.17277 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:24.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf412af0 0x5596cf414ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 4 client.17277 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.907+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596cf2fd930 con 0x5596cf412af0 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.908+0000 7f71077fe640 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf412af0 0x5596cf414ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.908+0000 7f70e6ffd640 10 client.17277 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.908+0000 7f71077fe640 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf412af0 0x5596cf414ee0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ee99949d910e19 server_cookie=71df010e880276bd in_seq=0 out_seq=0 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f70e6ffd640 1 -- 172.21.15.113:0/2117080890 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596cf2fd930 con 0x5596cf412af0 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f70e6ffd640 10 client.17277 handle_command_reply: tid=0 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 client.17277 shutdown 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 msgr2=0x5596cf3ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf3ffc50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009c20 tx=0x7f70f0015ef0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 shutdown_connections 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf412af0 0x5596cf414ee0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf402a00 0x5596cf3ffc50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f70e6ffd640 0 client.17277 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 --2- 172.21.15.113:0/2117080890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf3f94d0 0x5596cf402320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 >> 172.21.15.113:0/2117080890 conn(0x5596cf3f3ba0 msgr2=0x5596cf3f58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 shutdown_connections 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 1 -- 172.21.15.113:0/2117080890 wait complete. 2024-04-22T07:22:24.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.909+0000 7f7110851740 20 client.17277 trim_cache size 0 max 16384 2024-04-22T07:22:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 Processor -- start 2024-04-22T07:22:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 -- start start 2024-04-22T07:22:24.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 0x5596cf401de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b08a0 con 0x5596cf3f94d0 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ae170 con 0x5596cf3f9e80 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7110851740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f71081abfe0 con 0x5596cf402a00 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44744/0 (socket says 172.21.15.113:44744) 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 -- 172.21.15.113:0/3676836567 learned_addr learned my addr 172.21.15.113:0/3676836567 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f71077fe640 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 -- 172.21.15.113:0/3676836567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 msgr2=0x5596cf401de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 0x5596cf401de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 -- 172.21.15.113:0/3676836567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 msgr2=0x5596cf3f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 -- 172.21.15.113:0/3676836567 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596cf3f4770 con 0x5596cf3f94d0 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.922+0000 7f7106ffd640 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40e830 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005d80 tx=0x7f70f00099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be95629388bebd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f70fd7fa640 1 -- 172.21.15.113:0/3676836567 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0039070 con 0x5596cf3f94d0 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f70fd7fa640 1 -- 172.21.15.113:0/3676836567 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f70f0009400 con 0x5596cf3f94d0 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f70fd7fa640 1 -- 172.21.15.113:0/3676836567 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0010a20 con 0x5596cf3f94d0 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 -- 172.21.15.113:0/3676836567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 msgr2=0x5596cf40e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40e830 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005d80 tx=0x7f70f00099a0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 -- 172.21.15.113:0/3676836567 shutdown_connections 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 0x5596cf401de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 --2- 172.21.15.113:0/3676836567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40e830 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 -- 172.21.15.113:0/3676836567 >> 172.21.15.113:0/3676836567 conn(0x5596cf3f3ba0 msgr2=0x5596cf3f2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 -- 172.21.15.113:0/3676836567 shutdown_connections 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.923+0000 7f7110851740 1 -- 172.21.15.113:0/3676836567 wait complete. 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 -- start start 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 Processor -- start 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 0x5596cf3fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f71081ac630 con 0x5596cf3f94d0 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71081adaa0 con 0x5596cf3f9e80 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f71081acd00 con 0x5596cf402a00 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7110851740 10 client.0 _ll_register_callbacks cb 0x7f70f0015720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44758/0 (socket says 172.21.15.113:44758) 2024-04-22T07:22:24.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 -- 172.21.15.113:0/1143395557 learned_addr learned my addr 172.21.15.113:0/1143395557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f71077fe640 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 -- 172.21.15.113:0/1143395557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 msgr2=0x5596cf3fc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 0x5596cf3fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 -- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 msgr2=0x5596cf3fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.924+0000 7f7106ffd640 1 -- 172.21.15.113:0/1143395557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596cf3fcc20 con 0x5596cf3f94d0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7106ffd640 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40b010 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005fd0 tx=0x7f70f00048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d677eda760fd5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f70e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f70e6ffd640 1 -- 172.21.15.113:0/1143395557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0039040 con 0x5596cf3f94d0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f70e6ffd640 1 -- 172.21.15.113:0/1143395557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f70f00107a0 con 0x5596cf3f94d0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f0009400 con 0x5596cf3f94d0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f70e6ffd640 1 -- 172.21.15.113:0/1143395557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f70f0012ad0 con 0x5596cf3f94d0 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 10 client.17283 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:24.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5596cf478dc0 con 0x5596cf3f94d0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f70e6ffd640 1 -- 172.21.15.113:0/1143395557 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f70f0009400 con 0x5596cf3f94d0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f70e6ffd640 1 -- 172.21.15.113:0/1143395557 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f70f003c030 con 0x5596cf3f94d0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 10 client.17283 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 20 client.17283 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf4790b0 0x5596cf47b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 4 client.17283 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.925+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596cf36bc10 con 0x5596cf4790b0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f71077fe640 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf4790b0 0x5596cf47b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f70e6ffd640 10 client.17283 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f71077fe640 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf4790b0 0x5596cf47b4d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f003746ca596f531 server_cookie=f1ed64cae936cdf9 in_seq=0 out_seq=0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f70e6ffd640 1 -- 172.21.15.113:0/1143395557 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596cf36bc10 con 0x5596cf4790b0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f70e6ffd640 10 client.17283 handle_command_reply: tid=0 2024-04-22T07:22:24.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 client.17283 shutdown 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 msgr2=0x5596cf40b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40b010 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005fd0 tx=0x7f70f00048d0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 shutdown_connections 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5596cf4790b0 0x5596cf47b4d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5596cf402a00 0x5596cf3fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cf3f9e80 0x5596cf3fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 --2- 172.21.15.113:0/1143395557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5596cf3f94d0 0x5596cf40b010 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f70e6ffd640 0 client.17283 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.926+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 >> 172.21.15.113:0/1143395557 conn(0x5596cf3f3ba0 msgr2=0x5596cf3f2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 shutdown_connections 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f7110851740 1 -- 172.21.15.113:0/1143395557 wait complete. 2024-04-22T07:22:24.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f7110851740 20 client.17283 trim_cache size 0 max 16384 2024-04-22T07:22:24.932 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70f40a3d20 msgr2=0x7f70f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70f40a3d20 0x7f70f40a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f71000f0b50 tx=0x7f71000093f0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 msgr2=0x7f7108198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 0x7f7108198b40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f710002c700 tx=0x7f7100035d80 comp rx=0 tx=0).stop 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 shutdown_connections 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f70f40a3d20 0x7f70f40a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71081986c0 0x7f7108198b40 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71081072e0 0x7f710819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 --2- 172.21.15.113:0/1468408706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7108102210 0x7f710819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:24.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 >> 172.21.15.113:0/1468408706 conn(0x7f71080fdab0 msgr2=0x7f71080ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:24.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 shutdown_connections 2024-04-22T07:22:24.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:24.927+0000 7f710dc5c640 1 -- 172.21.15.113:0/1468408706 wait complete. 2024-04-22T07:22:24.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 -- 172.21.15.113:0/2515549334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0654106680 msgr2=0x7f0654106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 --2- 172.21.15.113:0/2515549334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0654106680 0x7f0654106a60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f064400aa10 tx=0x7f064402cf70 comp rx=0 tx=0).stop 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 -- 172.21.15.113:0/2515549334 shutdown_connections 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 --2- 172.21.15.113:0/2515549334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0654100f80 0x7f065410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 --2- 172.21.15.113:0/2515549334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06541005c0 0x7f0654100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 --2- 172.21.15.113:0/2515549334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0654106680 0x7f0654106a60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 -- 172.21.15.113:0/2515549334 >> 172.21.15.113:0/2515549334 conn(0x7f0654077410 msgr2=0x7f06540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 -- 172.21.15.113:0/2515549334 shutdown_connections 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 -- 172.21.15.113:0/2515549334 wait complete. 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.039+0000 7f0659c9a640 1 Processor -- start 2024-04-22T07:22:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 -- start start 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 0x7f0654198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0654100f80 0x7f0654198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0654106680 0x7f065419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0654110100 con 0x7f06541005c0 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f065410ff80 con 0x7f0654100f80 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0659c9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0654110280 con 0x7f0654106680 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 0x7f0654198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 0x7f0654198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44774/0 (socket says 172.21.15.113:44774) 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 -- 172.21.15.113:0/3705126561 learned_addr learned my addr 172.21.15.113:0/3705126561 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0652ffd640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0654100f80 0x7f0654198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 -- 172.21.15.113:0/3705126561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0654106680 msgr2=0x7f065419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0654106680 0x7f065419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 -- 172.21.15.113:0/3705126561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0654100f80 msgr2=0x7f0654198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0654100f80 0x7f0654198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 -- 172.21.15.113:0/3705126561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065419d730 con 0x7f06541005c0 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f0652ffd640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0654100f80 0x7f0654198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.040+0000 7f06537fe640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 0x7f0654198740 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0644006010 tx=0x7f0644035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.041+0000 7f0650ff9640 1 -- 172.21.15.113:0/3705126561 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644004450 con 0x7f06541005c0 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.041+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065419d9c0 con 0x7f06541005c0 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.041+0000 7f0650ff9640 1 -- 172.21.15.113:0/3705126561 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0644043070 con 0x7f06541005c0 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.041+0000 7f0650ff9640 1 -- 172.21.15.113:0/3705126561 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644003850 con 0x7f06541005c0 2024-04-22T07:22:25.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.041+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541a52b0 con 0x7f06541005c0 2024-04-22T07:22:25.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f0650ff9640 1 -- 172.21.15.113:0/3705126561 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0644004d90 con 0x7f06541005c0 2024-04-22T07:22:25.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 Processor -- start 2024-04-22T07:22:25.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 -- start start 2024-04-22T07:22:25.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac54d0 0x55a13dac58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 0x55a13dace430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a13da2a1f0 con 0x55a13dac5e80 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a13d9c74b0 con 0x55a13dacea00 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f065c96f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a13da261b0 con 0x55a13dac54d0 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.043+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 0x55a13dace430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 0x55a13dace430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44780/0 (socket says 172.21.15.113:44780) 2024-04-22T07:22:25.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 -- 172.21.15.113:0/1552776125 learned_addr learned my addr 172.21.15.113:0/1552776125 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0652ffd640 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0653fff640 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac54d0 0x55a13dac58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0650ff9640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f063c0a3d20 0x7f063c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 -- 172.21.15.113:0/1552776125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac54d0 msgr2=0x55a13dac58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac54d0 0x55a13dac58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 -- 172.21.15.113:0/1552776125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 msgr2=0x55a13dad0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 -- 172.21.15.113:0/1552776125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13d9b91b0 con 0x55a13dac5e80 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0653fff640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f063c0a3d20 0x7f063c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0650ff9640 1 -- 172.21.15.113:0/3705126561 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f06440f2b70 con 0x7f06541005c0 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f06537fe640 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 0x55a13dace430 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06440f01a0 tx=0x7f06440bae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5149f1dc579d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0639ffb640 1 -- 172.21.15.113:0/1552776125 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440be850 con 0x55a13dac5e80 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0639ffb640 1 -- 172.21.15.113:0/1552776125 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f06440df0f0 con 0x55a13dac5e80 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.044+0000 7f0639ffb640 1 -- 172.21.15.113:0/1552776125 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440c63f0 con 0x55a13dac5e80 2024-04-22T07:22:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 -- 172.21.15.113:0/1552776125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 msgr2=0x55a13dace430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 0x55a13dace430 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06440f01a0 tx=0x7f06440bae50 comp rx=0 tx=0).stop 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f0653fff640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f063c0a3d20 0x7f063c0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f064c009270 tx=0x7f064c004a40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 -- 172.21.15.113:0/1552776125 shutdown_connections 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac5e80 0x55a13dace430 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 --2- 172.21.15.113:0/1552776125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac54d0 0x55a13dac58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 -- 172.21.15.113:0/1552776125 >> 172.21.15.113:0/1552776125 conn(0x55a13dabfba0 msgr2=0x55a13dabff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 -- 172.21.15.113:0/1552776125 shutdown_connections 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 -- 172.21.15.113:0/1552776125 wait complete. 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.045+0000 7f065c96f740 1 -- start start 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 Processor -- start 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dace2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dac5e80 0x55a13dacb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dacea00 0x55a13dacbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a13dad1730 con 0x55a13dac54d0 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a13d9e3500 con 0x55a13dac5e80 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a13dad18b0 con 0x55a13dacea00 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 10 client.0 _ll_register_callbacks cb 0x55a13d9b1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dace2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dace2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44788/0 (socket says 172.21.15.113:44788) 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 -- 172.21.15.113:0/448861262 learned_addr learned my addr 172.21.15.113:0/448861262 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0653fff640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dacea00 0x55a13dacbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 -- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dacea00 msgr2=0x55a13dacbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0652ffd640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dac5e80 0x55a13dacb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dacea00 0x55a13dacbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 -- 172.21.15.113:0/448861262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dac5e80 msgr2=0x55a13dacb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dac5e80 0x55a13dacb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 -- 172.21.15.113:0/448861262 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a13d73c3f0 con 0x55a13dac54d0 2024-04-22T07:22:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0653fff640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dacea00 0x55a13dacbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0652ffd640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dac5e80 0x55a13dacb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f06537fe640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dace2e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f06440bb450 tx=0x7f064402cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8da63cb9035bc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0632ffd640 1 -- 172.21.15.113:0/448861262 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440c7050 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f0632ffd640 1 -- 172.21.15.113:0/448861262 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f0644005410 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.046+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06440df0f0 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f0632ffd640 1 -- 172.21.15.113:0/448861262 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f06440bd040 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 10 client.17295 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a13d8067d0 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f0632ffd640 1 -- 172.21.15.113:0/448861262 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f06440df0f0 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f0632ffd640 1 -- 172.21.15.113:0/448861262 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f064400b040 con 0x55a13dac54d0 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 10 client.17295 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 10 client.17295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 20 client.17295 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13dade750 0x55a13dae0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 4 client.17295 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a13d9c9930 con 0x55a13dade750 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.047+0000 7f0652ffd640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13dade750 0x55a13dae0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.048+0000 7f0632ffd640 10 client.17295 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.048+0000 7f0652ffd640 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13dade750 0x55a13dae0b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab49152fe8fa6cc3 server_cookie=fc659e77757aca13 in_seq=0 out_seq=0 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f0632ffd640 1 -- 172.21.15.113:0/448861262 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a13d9c9930 con 0x55a13dade750 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f0632ffd640 10 client.17295 handle_command_reply: tid=0 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 client.17295 shutdown 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 msgr2=0x55a13dace2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dace2e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f06440bb450 tx=0x7f064402cf70 comp rx=0 tx=0).stop 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 shutdown_connections 2024-04-22T07:22:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13dade750 0x55a13dae0b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dacea00 0x55a13dacbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dac5e80 0x55a13dacb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f0632ffd640 0 client.17295 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 --2- 172.21.15.113:0/448861262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dace2e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 >> 172.21.15.113:0/448861262 conn(0x55a13dabfba0 msgr2=0x55a13dac18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 shutdown_connections 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 1 -- 172.21.15.113:0/448861262 wait complete. 2024-04-22T07:22:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.049+0000 7f065c96f740 20 client.17295 trim_cache size 0 max 16384 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 Processor -- start 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 -- start start 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dada560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dacdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dace2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06541b06f0 con 0x55a13dac54d0 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f06541adfc0 con 0x55a13dacea00 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f065c96f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06541abe30 con 0x55a13dac5e80 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dada560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dada560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44804/0 (socket says 172.21.15.113:44804) 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f06537fe640 1 -- 172.21.15.113:0/2015039761 learned_addr learned my addr 172.21.15.113:0/2015039761 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.062+0000 7f0653fff640 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dace2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06537fe640 1 -- 172.21.15.113:0/2015039761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 msgr2=0x55a13dacdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f0652ffd640 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dacdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06537fe640 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dacdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06537fe640 1 -- 172.21.15.113:0/2015039761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 msgr2=0x55a13dace2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06537fe640 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dace2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06537fe640 1 -- 172.21.15.113:0/2015039761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13dacb9e0 con 0x55a13dac54d0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f0652ffd640 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dacdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06537fe640 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dada560 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f06440be780 tx=0x7f06440bcf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c8b2489f4159b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06397fa640 1 -- 172.21.15.113:0/2015039761 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440bd230 con 0x55a13dac54d0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06397fa640 1 -- 172.21.15.113:0/2015039761 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f06440df0f0 con 0x55a13dac54d0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f06397fa640 1 -- 172.21.15.113:0/2015039761 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440be3e0 con 0x55a13dac54d0 2024-04-22T07:22:25.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 -- 172.21.15.113:0/2015039761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 msgr2=0x55a13dada560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dada560 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f06440be780 tx=0x7f06440bcf90 comp rx=0 tx=0).stop 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 -- 172.21.15.113:0/2015039761 shutdown_connections 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dace2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dacdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 --2- 172.21.15.113:0/2015039761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dada560 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 -- 172.21.15.113:0/2015039761 >> 172.21.15.113:0/2015039761 conn(0x55a13dabfba0 msgr2=0x55a13dabec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 -- 172.21.15.113:0/2015039761 shutdown_connections 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.063+0000 7f065c96f740 1 -- 172.21.15.113:0/2015039761 wait complete. 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 -- start start 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 Processor -- start 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dad6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dad2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06541ac480 con 0x55a13dac54d0 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f06541ad8f0 con 0x55a13dacea00 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06541acb50 con 0x55a13dac5e80 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f065c96f740 10 client.0 _ll_register_callbacks cb 0x7f06440bad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dad6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dad6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44810/0 (socket says 172.21.15.113:44810) 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 -- 172.21.15.113:0/3247269976 learned_addr learned my addr 172.21.15.113:0/3247269976 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 -- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 msgr2=0x55a13dad2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dad2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 -- 172.21.15.113:0/3247269976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 msgr2=0x55a13dad2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.064+0000 7f06537fe640 1 -- 172.21.15.113:0/3247269976 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a13dad2ec0 con 0x55a13dac54d0 2024-04-22T07:22:25.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f06537fe640 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dad6de0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f06440baee0 tx=0x7f06440bae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5f9632077db24b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0632ffd640 1 -- 172.21.15.113:0/3247269976 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440be5c0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0632ffd640 1 -- 172.21.15.113:0/3247269976 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f06440894b0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06440df0f0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0632ffd640 1 -- 172.21.15.113:0/3247269976 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f06440be3e0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 10 client.17301 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a13dad2ec0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0632ffd640 1 -- 172.21.15.113:0/3247269976 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f06440df0f0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0632ffd640 1 -- 172.21.15.113:0/3247269976 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f06440bb0f0 con 0x55a13dac54d0 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 10 client.17301 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 20 client.17301 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13db44dd0 0x55a13db47200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 4 client.17301 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a13da37c10 con 0x55a13db44dd0 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.065+0000 7f0652ffd640 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13db44dd0 0x55a13db47200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f0632ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f0652ffd640 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13db44dd0 0x55a13db47200 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4ca82bf6374907e server_cookie=62b0b8edf5b8ec46 in_seq=0 out_seq=0 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f0632ffd640 1 -- 172.21.15.113:0/3247269976 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a13da37c10 con 0x55a13db44dd0 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f0632ffd640 10 client.17301 handle_command_reply: tid=0 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 client.17301 shutdown 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 msgr2=0x55a13dad6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dad6de0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f06440baee0 tx=0x7f06440bae50 comp rx=0 tx=0).stop 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 shutdown_connections 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55a13db44dd0 0x55a13db47200 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a13dacea00 0x55a13dad2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a13dac5e80 0x55a13dad2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 --2- 172.21.15.113:0/3247269976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a13dac54d0 0x55a13dad6de0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f0632ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:25.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 >> 172.21.15.113:0/3247269976 conn(0x55a13dabfba0 msgr2=0x55a13dac0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 shutdown_connections 2024-04-22T07:22:25.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 1 -- 172.21.15.113:0/3247269976 wait complete. 2024-04-22T07:22:25.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.066+0000 7f065c96f740 20 client.17301 trim_cache size 0 max 16384 2024-04-22T07:22:25.071 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f063c0a3d20 msgr2=0x7f063c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f063c0a3d20 0x7f063c0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f064c009270 tx=0x7f064c004a40 comp rx=0 tx=0).stop 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 msgr2=0x7f0654198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 0x7f0654198740 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0644006010 tx=0x7f0644035ca0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 shutdown_connections 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f063c0a3d20 0x7f063c0a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0654106680 0x7f065419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0654100f80 0x7f0654198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 --2- 172.21.15.113:0/3705126561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06541005c0 0x7f0654198740 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 >> 172.21.15.113:0/3705126561 conn(0x7f0654077410 msgr2=0x7f0654104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 shutdown_connections 2024-04-22T07:22:25.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.067+0000 7f0659c9a640 1 -- 172.21.15.113:0/3705126561 wait complete. 2024-04-22T07:22:25.074 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:25 smithi113 ceph-mon[27933]: pgmap v238: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:25.074 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:25 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2825228016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:22:25.075 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3518845012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0101090 msgr2=0x7fe5e0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3518845012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0101090 0x7fe5e0101470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009900 tx=0x7fe5d402bc30 comp rx=0 tx=0).stop 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3518845012 shutdown_connections 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3518845012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e0106160 0x7fe5e010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3518845012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5e0101a40 0x7fe5e0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3518845012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e0101090 0x7fe5e0101470 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3518845012 >> 172.21.15.113:0/3518845012 conn(0x7fe5e0077f50 msgr2=0x7fe5e0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3518845012 shutdown_connections 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3518845012 wait complete. 2024-04-22T07:22:25.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 Processor -- start 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- start start 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5e0101090 0x7fe5e0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e0106160 0x7fe5e0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 0x7fe5e019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01111e0 con 0x7fe5e019bf40 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0111060 con 0x7fe5e0101090 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.177+0000 7fe5e70c1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e0111360 con 0x7fe5e0106160 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 0x7fe5e019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 0x7fe5e019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44830/0 (socket says 172.21.15.113:44830) 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 -- 172.21.15.113:0/3297321381 learned_addr learned my addr 172.21.15.113:0/3297321381 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5dffff640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e0106160 0x7fe5e0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 -- 172.21.15.113:0/3297321381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e0106160 msgr2=0x7fe5e0197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5e0101090 0x7fe5e0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e0106160 0x7fe5e0197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 -- 172.21.15.113:0/3297321381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5e0101090 msgr2=0x7fe5e0197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5e0101090 0x7fe5e0197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 -- 172.21.15.113:0/3297321381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e019ca30 con 0x7fe5e019bf40 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e565e640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 0x7fe5e019c3f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800eaa0 tx=0x7fe5d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5ddffb640 1 -- 172.21.15.113:0/3297321381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800cdb0 con 0x7fe5e019bf40 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5ddffb640 1 -- 172.21.15.113:0/3297321381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe5d8005020 con 0x7fe5e019bf40 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5ddffb640 1 -- 172.21.15.113:0/3297321381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80116d0 con 0x7fe5e019bf40 2024-04-22T07:22:25.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e019cd20 con 0x7fe5e019bf40 2024-04-22T07:22:25.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.178+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01a44c0 con 0x7fe5e019bf40 2024-04-22T07:22:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.180+0000 7fe5ddffb640 1 -- 172.21.15.113:0/3297321381 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe5d80051d0 con 0x7fe5e019bf40 2024-04-22T07:22:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 Processor -- start 2024-04-22T07:22:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 -- start start 2024-04-22T07:22:25.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b7517750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517d20 0x55d1b7522640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5ddffb640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe5d00a3d20 0x7fe5d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 0x55d1b7525020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d1b747c1f0 con 0x55d1b7522c10 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1b74194b0 con 0x55d1b7517d20 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d1b74781b0 con 0x55d1b7517370 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe5d00a3d20 0x7fe5d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 0x55d1b7525020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 0x55d1b7525020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44840/0 (socket says 172.21.15.113:44840) 2024-04-22T07:22:25.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 -- 172.21.15.113:0/2536347631 learned_addr learned my addr 172.21.15.113:0/2536347631 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5dffff640 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b7517750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5dffff640 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517d20 0x55d1b7522640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5ddffb640 1 -- 172.21.15.113:0/3297321381 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7fe5d80d0c70 con 0x7fe5e019bf40 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 -- 172.21.15.113:0/2536347631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 msgr2=0x55d1b7517750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b7517750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 -- 172.21.15.113:0/2536347631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517d20 msgr2=0x55d1b7522640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517d20 0x55d1b7522640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e565e640 1 -- 172.21.15.113:0/2536347631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b740b1b0 con 0x55d1b7522c10 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.181+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe5d00a3d20 0x7fe5d00a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036040 tx=0x7fe5d4005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e565e640 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 0x55d1b7525020 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe5d809c770 tx=0x7fe5d8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc07e35b23a2fd23 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5caffd640 1 -- 172.21.15.113:0/2536347631 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d801f070 con 0x55d1b7522c10 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5caffd640 1 -- 172.21.15.113:0/2536347631 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe5d809d170 con 0x55d1b7522c10 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5caffd640 1 -- 172.21.15.113:0/2536347631 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8005580 con 0x55d1b7522c10 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 -- 172.21.15.113:0/2536347631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 msgr2=0x55d1b7525020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 0x55d1b7525020 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe5d809c770 tx=0x7fe5d8005850 comp rx=0 tx=0).stop 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 -- 172.21.15.113:0/2536347631 shutdown_connections 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7522c10 0x55d1b7525020 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517d20 0x55d1b7522640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/2536347631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b7517750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 -- 172.21.15.113:0/2536347631 >> 172.21.15.113:0/2536347631 conn(0x55d1b7511ba0 msgr2=0x55d1b7511f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 -- 172.21.15.113:0/2536347631 shutdown_connections 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.182+0000 7fe5e9d94740 1 -- 172.21.15.113:0/2536347631 wait complete. 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 -- start start 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 Processor -- start 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 0x55d1b75224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b751f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7522c10 0x55d1b751fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7525a80 con 0x55d1b7517370 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7435500 con 0x55d1b7522c10 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7525c00 con 0x55d1b7517d20 2024-04-22T07:22:25.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e9d94740 10 client.0 _ll_register_callbacks cb 0x55d1b7403b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e4e5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 0x55d1b75224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e4e5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 0x55d1b75224b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44848/0 (socket says 172.21.15.113:44848) 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4027652128 learned_addr learned my addr 172.21.15.113:0/4027652128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e565e640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7522c10 0x55d1b751fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4027652128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 msgr2=0x55d1b751f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5dffff640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b751f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.183+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b751f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7522c10 msgr2=0x55d1b751fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7522c10 0x55d1b751fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4027652128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b718e3f0 con 0x55d1b7517370 2024-04-22T07:22:25.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e565e640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7522c10 0x55d1b751fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 0x55d1b75224b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40042f0 tx=0x7fe5d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74fbf0ecbd4b08f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5bffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5bffff640 1 -- 172.21.15.113:0/4027652128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d404c020 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5bffff640 1 -- 172.21.15.113:0/4027652128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7fe5d4005050 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d809d170 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5bffff640 1 -- 172.21.15.113:0/4027652128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe5d4057050 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 10 client.17313 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d1b72587d0 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5bffff640 1 -- 172.21.15.113:0/4027652128 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe5d809d170 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5bffff640 1 -- 172.21.15.113:0/4027652128 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fe5d4004c00 con 0x55d1b7517370 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 10 client.17313 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 10 client.17313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:22:25.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 20 client.17313 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7532d00 0x55d1b75351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 4 client.17313 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.184+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b741b930 con 0x55d1b7532d00 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.185+0000 7fe5dffff640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7532d00 0x55d1b75351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.185+0000 7fe5bffff640 10 client.17313 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.185+0000 7fe5dffff640 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7532d00 0x55d1b75351c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ac699dc1e250ed4 server_cookie=311dea72819f1247 in_seq=0 out_seq=0 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5bffff640 1 -- 172.21.15.113:0/4027652128 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1b741b930 con 0x55d1b7532d00 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5bffff640 10 client.17313 handle_command_reply: tid=0 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 client.17313 shutdown 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 msgr2=0x55d1b75224b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 0x55d1b75224b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40042f0 tx=0x7fe5d4009260 comp rx=0 tx=0).stop 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 shutdown_connections 2024-04-22T07:22:25.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7532d00 0x55d1b75351c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7522c10 0x55d1b751fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5bffff640 0 client.17313 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b751f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4027652128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517370 0x55d1b75224b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 >> 172.21.15.113:0/4027652128 conn(0x55d1b7511ba0 msgr2=0x55d1b75138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 shutdown_connections 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4027652128 wait complete. 2024-04-22T07:22:25.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.186+0000 7fe5e9d94740 20 client.17313 trim_cache size 0 max 16384 2024-04-22T07:22:25.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.199+0000 7fe5e9d94740 1 Processor -- start 2024-04-22T07:22:25.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.199+0000 7fe5e9d94740 1 -- start start 2024-04-22T07:22:25.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7511ba0 0x55d1b752e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b752b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 0x55d1b7521f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ae640 con 0x55d1b7517d20 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01abf10 con 0x55d1b7511ba0 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01a9d80 con 0x55d1b7517370 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 0x55d1b7521f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 0x55d1b7521f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44864/0 (socket says 172.21.15.113:44864) 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 -- 172.21.15.113:0/1835034422 learned_addr learned my addr 172.21.15.113:0/1835034422 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 -- 172.21.15.113:0/1835034422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 msgr2=0x55d1b752b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b752b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 -- 172.21.15.113:0/1835034422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7511ba0 msgr2=0x55d1b752e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7511ba0 0x55d1b752e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 -- 172.21.15.113:0/1835034422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b7525000 con 0x55d1b7517d20 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5e565e640 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 0x55d1b7521f70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80bd010 tx=0x7fe5d809c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c83824d49b21446 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5ca7fc640 1 -- 172.21.15.113:0/1835034422 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d801f040 con 0x55d1b7517d20 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5ca7fc640 1 -- 172.21.15.113:0/1835034422 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe5d80bd0b0 con 0x55d1b7517d20 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.200+0000 7fe5ca7fc640 1 -- 172.21.15.113:0/1835034422 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80b1150 con 0x55d1b7517d20 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 -- 172.21.15.113:0/1835034422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 msgr2=0x55d1b7521f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 0x55d1b7521f70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80bd010 tx=0x7fe5d809c970 comp rx=0 tx=0).stop 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 -- 172.21.15.113:0/1835034422 shutdown_connections 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7517d20 0x55d1b7521f70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517370 0x55d1b752b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/1835034422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7511ba0 0x55d1b752e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 -- 172.21.15.113:0/1835034422 >> 172.21.15.113:0/1835034422 conn(0x55d1b751fee0 msgr2=0x55d1b751ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 -- 172.21.15.113:0/1835034422 shutdown_connections 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 -- 172.21.15.113:0/1835034422 wait complete. 2024-04-22T07:22:25.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 -- start start 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.201+0000 7fe5e9d94740 1 Processor -- start 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 0x55d1b7518c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517370 0x55d1b7519160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b759a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01aa3d0 con 0x55d1b7511ba0 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01ab840 con 0x55d1b7517370 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e01aaaa0 con 0x55d1b7517d20 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 10 client.0 _ll_register_callbacks cb 0x7fe5d402cbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 0x55d1b7518c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 0x55d1b7518c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44870/0 (socket says 172.21.15.113:44870) 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4223831925 learned_addr learned my addr 172.21.15.113:0/4223831925 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e565e640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b759a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4223831925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 msgr2=0x55d1b759a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b759a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517370 msgr2=0x55d1b7519160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517370 0x55d1b7519160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 -- 172.21.15.113:0/4223831925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b759a8b0 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e565e640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b759a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e4e5d640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 0x55d1b7518c20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4003a40 tx=0x7fe5d4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5598b7b0e4260a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5bffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5bffff640 1 -- 172.21.15.113:0/4223831925 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4060060 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5bffff640 1 -- 172.21.15.113:0/4223831925 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7fe5d403fa40 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d80bd0b0 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5bffff640 1 -- 172.21.15.113:0/4223831925 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe5d4067050 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 10 client.17319 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.202+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d1b759bb50 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5bffff640 1 -- 172.21.15.113:0/4223831925 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe5d80bd0b0 con 0x55d1b7511ba0 2024-04-22T07:22:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5bffff640 1 -- 172.21.15.113:0/4223831925 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fe5d40046c0 con 0x55d1b7511ba0 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5e9d94740 10 client.17319 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5e9d94740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5e9d94740 20 client.17319 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7522c10 0x55d1b759e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5e9d94740 4 client.17319 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1b7489c10 con 0x55d1b7522c10 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5dffff640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7522c10 0x55d1b759e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5dffff640 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7522c10 0x55d1b759e330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b790972a0d45b24 server_cookie=14daeb6c0dc0a843 in_seq=0 out_seq=0 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5bffff640 10 client.17319 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5bffff640 1 -- 172.21.15.113:0/4223831925 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1b7489c10 con 0x55d1b7522c10 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.203+0000 7fe5bffff640 10 client.17319 handle_command_reply: tid=0 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 client.17319 shutdown 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 msgr2=0x55d1b7518c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 0x55d1b7518c20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4003a40 tx=0x7fe5d4004610 comp rx=0 tx=0).stop 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 shutdown_connections 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55d1b7522c10 0x55d1b759e330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d1b7517d20 0x55d1b759a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b7517370 0x55d1b7519160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5bffff640 0 client.17319 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 --2- 172.21.15.113:0/4223831925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d1b7511ba0 0x55d1b7518c20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 >> 172.21.15.113:0/4223831925 conn(0x55d1b751fee0 msgr2=0x55d1b7512cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 shutdown_connections 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 1 -- 172.21.15.113:0/4223831925 wait complete. 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e9d94740 20 client.17319 trim_cache size 0 max 16384 2024-04-22T07:22:25.208 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:22:25.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe5d00a3d20 msgr2=0x7fe5d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe5d00a3d20 0x7fe5d00a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036040 tx=0x7fe5d4005fd0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 msgr2=0x7fe5e019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 0x7fe5e019c3f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800eaa0 tx=0x7fe5d800ef60 comp rx=0 tx=0).stop 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 shutdown_connections 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe5d00a3d20 0x7fe5d00a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5e019bf40 0x7fe5e019c3f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e0106160 0x7fe5e0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 --2- 172.21.15.113:0/3297321381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5e0101090 0x7fe5e0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 >> 172.21.15.113:0/3297321381 conn(0x7fe5e0077f50 msgr2=0x7fe5e00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.204+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 shutdown_connections 2024-04-22T07:22:25.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:25.205+0000 7fe5e70c1640 1 -- 172.21.15.113:0/3297321381 wait complete. 2024-04-22T07:22:25.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:25 smithi133 ceph-mon[31198]: pgmap v238: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:25.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:25 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2825228016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:22:25.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:25 smithi154 ceph-mon[31862]: pgmap v238: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:22:25.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:25 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2825228016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:22:26.214 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:22:26.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:22:26.214 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 -- 172.21.15.113:0/2447051459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c101090 msgr2=0x7f3a7c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 --2- 172.21.15.113:0/2447051459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c101090 0x7f3a7c101470 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a7802bc30 comp rx=0 tx=0).stop 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 -- 172.21.15.113:0/2447051459 shutdown_connections 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 --2- 172.21.15.113:0/2447051459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a7c106160 0x7f3a7c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 --2- 172.21.15.113:0/2447051459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c101a40 0x7f3a7c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 --2- 172.21.15.113:0/2447051459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c101090 0x7f3a7c101470 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 -- 172.21.15.113:0/2447051459 >> 172.21.15.113:0/2447051459 conn(0x7f3a7c077f50 msgr2=0x7f3a7c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 -- 172.21.15.113:0/2447051459 shutdown_connections 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 -- 172.21.15.113:0/2447051459 wait complete. 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 Processor -- start 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.331+0000 7f3a835c7640 1 -- start start 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a7c101090 0x7f3a7c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c106160 0x7f3a7c1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 0x7f3a7c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c111200 con 0x7f3a7c19a830 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c111080 con 0x7f3a7c106160 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c111380 con 0x7f3a7c101090 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 0x7f3a7c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 0x7f3a7c19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55670/0 (socket says 172.21.15.113:55670) 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 -- 172.21.15.113:0/1179973377 learned_addr learned my addr 172.21.15.113:0/1179973377 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c106160 0x7f3a7c1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81363640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a7c101090 0x7f3a7c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 -- 172.21.15.113:0/1179973377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a7c101090 msgr2=0x7f3a7c1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a7c101090 0x7f3a7c1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 -- 172.21.15.113:0/1179973377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c106160 msgr2=0x7f3a7c1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c106160 0x7f3a7c1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 -- 172.21.15.113:0/1179973377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c19b3c0 con 0x7f3a7c19a830 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c106160 0x7f3a7c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a81b64640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 0x7f3a7c19ace0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400eaa0 tx=0x7f3a7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a727fc640 1 -- 172.21.15.113:0/1179973377 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400cdb0 con 0x7f3a7c19a830 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a727fc640 1 -- 172.21.15.113:0/1179973377 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3a74005020 con 0x7f3a7c19a830 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a727fc640 1 -- 172.21.15.113:0/1179973377 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740116d0 con 0x7f3a7c19a830 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c19b6b0 con 0x7f3a7c19a830 2024-04-22T07:22:26.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.332+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c1a7840 con 0x7f3a7c19a830 2024-04-22T07:22:26.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.334+0000 7f3a727fc640 1 -- 172.21.15.113:0/1179973377 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f3a740051d0 con 0x7f3a7c19a830 2024-04-22T07:22:26.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 Processor -- start 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 -- start start 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866c6e10 0x5648866c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564886621f00 con 0x5648866be290 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5648865bf590 con 0x5648866bd8e0 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a861bb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56488661dec0 con 0x5648866c6e10 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a727fc640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3a680a3da0 0x7f3a680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a81363640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a81363640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55680/0 (socket says 172.21.15.113:55680) 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a81363640 1 -- 172.21.15.113:0/1977271311 learned_addr learned my addr 172.21.15.113:0/1977271311 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.335+0000 7f3a81b64640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3a680a3da0 0x7f3a680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a727fc640 1 -- 172.21.15.113:0/1179973377 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f3a740d0b70 con 0x7f3a7c19a830 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866c6e10 0x5648866c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81363640 1 -- 172.21.15.113:0/1977271311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866c6e10 msgr2=0x5648866c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81363640 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866c6e10 0x5648866c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81363640 1 -- 172.21.15.113:0/1977271311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 msgr2=0x5648866bdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81363640 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866bdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81363640 1 -- 172.21.15.113:0/1977271311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648863fc7e0 con 0x5648866be290 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866c6e10 0x5648866c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81b64640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3a680a3da0 0x7f3a680a6260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a740ce1d0 tx=0x7f3a7406f460 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a81363640 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c6840 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a7802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690e82aecb12c5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a677fe640 1 -- 172.21.15.113:0/1977271311 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7802cb10 con 0x5648866be290 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a677fe640 1 -- 172.21.15.113:0/1977271311 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3a7802ccb0 con 0x5648866be290 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.336+0000 7f3a677fe640 1 -- 172.21.15.113:0/1977271311 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78004920 con 0x5648866be290 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 -- 172.21.15.113:0/1977271311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 msgr2=0x5648866c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c6840 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a7802c5d0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 -- 172.21.15.113:0/1977271311 shutdown_connections 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866c6e10 0x5648866c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c6840 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1977271311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 -- 172.21.15.113:0/1977271311 >> 172.21.15.113:0/1977271311 conn(0x5648866b7fb0 msgr2=0x5648866b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 -- 172.21.15.113:0/1977271311 shutdown_connections 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 -- 172.21.15.113:0/1977271311 wait complete. 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 -- start start 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.337+0000 7f3a861bb740 1 Processor -- start 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866bd8e0 0x5648866c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5648866c9d20 con 0x5648866be290 2024-04-22T07:22:26.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5648865db580 con 0x5648866c6e10 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5648866c9ea0 con 0x5648866bd8e0 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 10 client.0 _ll_register_callbacks cb 0x5648865e1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55696/0 (socket says 172.21.15.113:55696) 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 -- 172.21.15.113:0/3229686792 learned_addr learned my addr 172.21.15.113:0/3229686792 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a80b62640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866bd8e0 0x5648866c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a80b62640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 -- 172.21.15.113:0/3229686792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866bd8e0 msgr2=0x5648866c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866bd8e0 0x5648866c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 -- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 msgr2=0x5648866c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 -- 172.21.15.113:0/3229686792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488645cf70 con 0x5648866be290 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a80b62640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866bd8e0 0x5648866c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:26.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a81363640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c4340 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a78034810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a4a08fd5c394a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a64ff9640 1 -- 172.21.15.113:0/3229686792 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78034dd0 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56488662f920 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.338+0000 7f3a64ff9640 1 -- 172.21.15.113:0/3229686792 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f3a7804e420 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a64ff9640 1 -- 172.21.15.113:0/3229686792 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f3a7804e620 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 10 client.17331 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5648862f4170 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a64ff9640 1 -- 172.21.15.113:0/3229686792 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x56488662f920 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a64ff9640 1 -- 172.21.15.113:0/3229686792 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f3a7803d920 con 0x5648866be290 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 10 client.17331 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 20 client.17331 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5648866d7100 0x5648866d94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 4 client.17331 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564886629790 con 0x5648866d7100 2024-04-22T07:22:26.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.339+0000 7f3a80b62640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5648866d7100 0x5648866d94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a80b62640 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5648866d7100 0x5648866d94f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6eefe01ad5108b server_cookie=3c6ad2a6a0ad1a8c in_seq=0 out_seq=0 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a64ff9640 10 client.17331 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a64ff9640 1 -- 172.21.15.113:0/3229686792 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564886629790 con 0x5648866d7100 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a64ff9640 10 client.17331 handle_command_reply: tid=0 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a861bb740 1 client.17331 shutdown 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 msgr2=0x5648866c4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.340+0000 7f3a861bb740 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c4340 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a78034810 comp rx=0 tx=0).stop 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 shutdown_connections 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5648866d7100 0x5648866d94f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a64ff9640 0 client.17331 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866be290 0x5648866c4340 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 --2- 172.21.15.113:0/3229686792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866bd8e0 0x5648866c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 >> 172.21.15.113:0/3229686792 conn(0x5648866b7fb0 msgr2=0x5648866b9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 shutdown_connections 2024-04-22T07:22:26.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 1 -- 172.21.15.113:0/3229686792 wait complete. 2024-04-22T07:22:26.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.341+0000 7f3a861bb740 20 client.17331 trim_cache size 0 max 16384 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 Processor -- start 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 -- start start 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 0x5648866c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1b2af0 con 0x5648866c6e10 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1b03c0 con 0x5648866bd8e0 2024-04-22T07:22:26.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a861bb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1ae230 con 0x5648866be290 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a80b62640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 0x5648866c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a80b62640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 0x5648866c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55700/0 (socket says 172.21.15.113:55700) 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.358+0000 7f3a80b62640 1 -- 172.21.15.113:0/764794936 learned_addr learned my addr 172.21.15.113:0/764794936 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a81363640 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 -- 172.21.15.113:0/764794936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 msgr2=0x5648866d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 -- 172.21.15.113:0/764794936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 msgr2=0x5648866c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 -- 172.21.15.113:0/764794936 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648866c2350 con 0x5648866c6e10 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a80b62640 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 0x5648866c46a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00be20 tx=0x7f3a6c005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464739f3c33fa992 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a66ffd640 1 -- 172.21.15.113:0/764794936 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c004220 con 0x5648866c6e10 2024-04-22T07:22:26.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a66ffd640 1 -- 172.21.15.113:0/764794936 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3a6c0043c0 con 0x5648866c6e10 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.359+0000 7f3a66ffd640 1 -- 172.21.15.113:0/764794936 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c011440 con 0x5648866c6e10 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 -- 172.21.15.113:0/764794936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 msgr2=0x5648866c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 0x5648866c46a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00be20 tx=0x7f3a6c005d20 comp rx=0 tx=0).stop 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 -- 172.21.15.113:0/764794936 shutdown_connections 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866c6e10 0x5648866c46a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 --2- 172.21.15.113:0/764794936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866bd8e0 0x5648866c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 -- 172.21.15.113:0/764794936 >> 172.21.15.113:0/764794936 conn(0x5648866b7fb0 msgr2=0x5648866b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 -- 172.21.15.113:0/764794936 shutdown_connections 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 -- 172.21.15.113:0/764794936 wait complete. 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 -- start start 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.360+0000 7f3a861bb740 1 Processor -- start 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 0x5648866c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1ae880 con 0x5648866bd8e0 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1afcf0 con 0x5648866c6e10 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1aef50 con 0x5648866be290 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 10 client.0 _ll_register_callbacks cb 0x7f3a78005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 0x5648866c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 0x5648866c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55714/0 (socket says 172.21.15.113:55714) 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 -- 172.21.15.113:0/1377041803 learned_addr learned my addr 172.21.15.113:0/1377041803 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a81363640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 -- 172.21.15.113:0/1377041803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 msgr2=0x5648866c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 -- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 msgr2=0x5648866c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 -- 172.21.15.113:0/1377041803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488673c5b0 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a81363640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a80b62640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 0x5648866c0cd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x5648866cfb90 tx=0x7f3a6c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5424aeed181f346a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a64ff9640 1 -- 172.21.15.113:0/1377041803 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0037f0 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a64ff9640 1 -- 172.21.15.113:0/1377041803 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f3a6c003990 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.361+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6c0043c0 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a64ff9640 1 -- 172.21.15.113:0/1377041803 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f3a6c003e70 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 10 client.17337 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56488673d8b0 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a64ff9640 1 -- 172.21.15.113:0/1377041803 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3a6c0043c0 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a64ff9640 1 -- 172.21.15.113:0/1377041803 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f3a6c036020 con 0x5648866bd8e0 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 10 client.17337 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:26.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 20 client.17337 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56488673df50 0x564886740370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 4 client.17337 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648865a5c50 con 0x56488673df50 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a81363640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56488673df50 0x564886740370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a81363640 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56488673df50 0x564886740370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37d4d1480ad9152 server_cookie=1d2d48bb1b4f5848 in_seq=0 out_seq=0 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.362+0000 7f3a64ff9640 10 client.17337 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.363+0000 7f3a64ff9640 1 -- 172.21.15.113:0/1377041803 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648865a5c50 con 0x56488673df50 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.363+0000 7f3a64ff9640 10 client.17337 handle_command_reply: tid=0 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.363+0000 7f3a861bb740 1 client.17337 shutdown 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 msgr2=0x5648866c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 0x5648866c0cd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x5648866cfb90 tx=0x7f3a6c0056d0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 shutdown_connections 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56488673df50 0x564886740370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648866c6e10 0x5648866c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648866be290 0x5648866c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 --2- 172.21.15.113:0/1377041803 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5648866bd8e0 0x5648866c0cd0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 >> 172.21.15.113:0/1377041803 conn(0x5648866b7fb0 msgr2=0x5648866b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a64ff9640 0 client.17337 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 shutdown_connections 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 1 -- 172.21.15.113:0/1377041803 wait complete. 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a861bb740 20 client.17337 trim_cache size 0 max 16384 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "9f81cfea-a450-4dca-afe7-719b9fcef1e9", 2024-04-22T07:22:26.368 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:22:26.369 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:22:26.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3a680a3da0 msgr2=0x7f3a680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3a680a3da0 0x7f3a680a6260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a740ce1d0 tx=0x7f3a7406f460 comp rx=0 tx=0).stop 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 msgr2=0x7f3a7c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 0x7f3a7c19ace0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400eaa0 tx=0x7f3a7400ef60 comp rx=0 tx=0).stop 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 shutdown_connections 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f3a680a3da0 0x7f3a680a6260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a7c19a830 0x7f3a7c19ace0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a7c106160 0x7f3a7c1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 --2- 172.21.15.113:0/1179973377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3a7c101090 0x7f3a7c1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.364+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 >> 172.21.15.113:0/1179973377 conn(0x7f3a7c077f50 msgr2=0x7f3a7c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.365+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 shutdown_connections 2024-04-22T07:22:26.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.365+0000 7f3a835c7640 1 -- 172.21.15.113:0/1179973377 wait complete. 2024-04-22T07:22:26.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:22:26.373 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 -- 172.21.15.113:0/1842941216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e70106130 msgr2=0x7f8e7010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1842941216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e70106130 0x7f8e7010dd00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009ca0 tx=0x7f8e6802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 -- 172.21.15.113:0/1842941216 shutdown_connections 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1842941216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e70106130 0x7f8e7010dd00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1842941216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70101a10 0x7f8e70105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1842941216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e70101060 0x7f8e70101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 -- 172.21.15.113:0/1842941216 >> 172.21.15.113:0/1842941216 conn(0x7f8e700771d0 msgr2=0x7f8e700775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 -- 172.21.15.113:0/1842941216 shutdown_connections 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 -- 172.21.15.113:0/1842941216 wait complete. 2024-04-22T07:22:26.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.475+0000 7f8e748d2640 1 Processor -- start 2024-04-22T07:22:26.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 -- start start 2024-04-22T07:22:26.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e70101060 0x7f8e70197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70106130 0x7f8e70197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 0x7f8e7019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701111b0 con 0x7f8e7019bed0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70111030 con 0x7f8e70106130 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e748d2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70111330 con 0x7f8e70101060 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 0x7f8e7019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 0x7f8e7019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55720/0 (socket says 172.21.15.113:55720) 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/1602308643 learned_addr learned my addr 172.21.15.113:0/1602308643 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70106130 0x7f8e70197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e70101060 0x7f8e70197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/1602308643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e70101060 msgr2=0x7f8e70197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e70101060 0x7f8e70197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/1602308643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70106130 msgr2=0x7f8e70197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70106130 0x7f8e70197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/1602308643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7019c9c0 con 0x7f8e7019bed0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.476+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 0x7f8e7019c380 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8e6802c700 tx=0x7f8e68035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.477+0000 7f8e67fff640 1 -- 172.21.15.113:0/1602308643 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68039040 con 0x7f8e7019bed0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.477+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7019cc50 con 0x7f8e7019bed0 2024-04-22T07:22:26.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.477+0000 7f8e67fff640 1 -- 172.21.15.113:0/1602308643 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e68044070 con 0x7f8e7019bed0 2024-04-22T07:22:26.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.477+0000 7f8e67fff640 1 -- 172.21.15.113:0/1602308643 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68004540 con 0x7f8e7019bed0 2024-04-22T07:22:26.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.477+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a4410 con 0x7f8e7019bed0 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.479+0000 7f8e67fff640 1 -- 172.21.15.113:0/1602308643 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8e680046e0 con 0x7f8e7019bed0 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.479+0000 7f8e775a1740 1 Processor -- start 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.479+0000 7f8e775a1740 1 -- start start 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.479+0000 7f8e67fff640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8e5c0a3d20 0x7f8e5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764efb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764efe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e775a1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561764e551e0 con 0x561764ef0390 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e775a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561764df2390 con 0x561764efbc90 2024-04-22T07:22:26.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e775a1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561764e511a0 con 0x561764ef0d40 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8e5c0a3d20 0x7f8e5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55728/0 (socket says 172.21.15.113:55728) 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1660372787 learned_addr learned my addr 172.21.15.113:0/1660372787 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e67fff640 1 -- 172.21.15.113:0/1602308643 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f8e680f3180 con 0x7f8e7019bed0 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1660372787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 msgr2=0x561764efb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764efb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1660372787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 msgr2=0x561764efe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764efe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1660372787 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561764c317d0 con 0x561764ef0390 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8e5c0a3d20 0x7f8e5c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800a5c0 tx=0x7f8e580117b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef0770 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000cbf0 tx=0x7f8e6000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3a8334a410db9d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e64d99640 1 -- 172.21.15.113:0/1660372787 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60010630 con 0x561764ef0390 2024-04-22T07:22:26.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e64d99640 1 -- 172.21.15.113:0/1660372787 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e60010ca0 con 0x561764ef0390 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.480+0000 7f8e64d99640 1 -- 172.21.15.113:0/1660372787 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60019bf0 con 0x561764ef0390 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 -- 172.21.15.113:0/1660372787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 msgr2=0x561764ef0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef0770 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000cbf0 tx=0x7f8e6000e720 comp rx=0 tx=0).stop 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 -- 172.21.15.113:0/1660372787 shutdown_connections 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764efe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764efb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1660372787 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef0770 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 -- 172.21.15.113:0/1660372787 >> 172.21.15.113:0/1660372787 conn(0x561764eeacb0 msgr2=0x561764eeb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 -- 172.21.15.113:0/1660372787 shutdown_connections 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 -- 172.21.15.113:0/1660372787 wait complete. 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.481+0000 7f8e775a1740 1 -- start start 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 Processor -- start 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764ef9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561764efea50 con 0x561764ef0390 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561764e0e2f0 con 0x561764efbc90 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561764efebd0 con 0x561764ef0d40 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 10 client.0 _ll_register_callbacks cb 0x561764e1b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55732/0 (socket says 172.21.15.113:55732) 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1899561776 learned_addr learned my addr 172.21.15.113:0/1899561776 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764ef9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1899561776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 msgr2=0x561764ef9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764ef9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 msgr2=0x561764ef9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 -- 172.21.15.113:0/1899561776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561764b673f0 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef8cf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8e60005c50 tx=0x7f8e60005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6239fd671170a1c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e55ffb640 1 -- 172.21.15.113:0/1899561776 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60029060 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e55ffb640 1 -- 172.21.15.113:0/1899561776 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f8e600055f0 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.482+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e60010ca0 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e55ffb640 1 -- 172.21.15.113:0/1899561776 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8e60030050 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 10 client.17349 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561764b270f0 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e55ffb640 1 -- 172.21.15.113:0/1899561776 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e60010ca0 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e55ffb640 1 -- 172.21.15.113:0/1899561776 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f8e600046c0 con 0x561764ef0390 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 10 client.17349 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 20 client.17349 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:26.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f0bc90 0x561764f0e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 4 client.17349 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561764e62c00 con 0x561764f0bc90 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.483+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f0bc90 0x561764f0e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e55ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f0bc90 0x561764f0e080 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90acd0f0411c4ed server_cookie=3cb0cc0c0385e105 in_seq=0 out_seq=0 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e55ffb640 1 -- 172.21.15.113:0/1899561776 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561764e62c00 con 0x561764f0bc90 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e55ffb640 10 client.17349 handle_command_reply: tid=0 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 client.17349 shutdown 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 msgr2=0x561764ef8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef8cf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8e60005c50 tx=0x7f8e60005270 comp rx=0 tx=0).stop 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 shutdown_connections 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f0bc90 0x561764f0e080 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e55ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0d40 0x561764ef9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1899561776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0390 0x561764ef8cf0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.484+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 >> 172.21.15.113:0/1899561776 conn(0x561764eeacb0 msgr2=0x561764eeca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.485+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 shutdown_connections 2024-04-22T07:22:26.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.485+0000 7f8e775a1740 1 -- 172.21.15.113:0/1899561776 wait complete. 2024-04-22T07:22:26.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.485+0000 7f8e775a1740 20 client.17349 trim_cache size 0 max 16384 2024-04-22T07:22:26.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 Processor -- start 2024-04-22T07:22:26.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 -- start start 2024-04-22T07:22:26.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764efb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 0x561764f07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701ae550 con 0x561764ef0d40 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701abe20 con 0x561764efbc90 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e775a1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701a9c90 con 0x561764ef0390 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 0x561764f07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 0x561764f07980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55744/0 (socket says 172.21.15.113:55744) 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 -- 172.21.15.113:0/1014022434 learned_addr learned my addr 172.21.15.113:0/1014022434 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 -- 172.21.15.113:0/1014022434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 msgr2=0x561764efb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764efb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 -- 172.21.15.113:0/1014022434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 msgr2=0x561764ef7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.502+0000 7f8e65d9b640 1 -- 172.21.15.113:0/1014022434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561764eea4f0 con 0x561764ef0d40 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 0x561764f07980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8e700fe240 tx=0x7f8e58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dd2a0554716abe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e57fff640 1 -- 172.21.15.113:0/1014022434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5802b030 con 0x561764ef0d40 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e57fff640 1 -- 172.21.15.113:0/1014022434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e58016070 con 0x561764ef0d40 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e57fff640 1 -- 172.21.15.113:0/1014022434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58018070 con 0x561764ef0d40 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 -- 172.21.15.113:0/1014022434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 msgr2=0x561764f07980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 0x561764f07980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8e700fe240 tx=0x7f8e58014000 comp rx=0 tx=0).stop 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 -- 172.21.15.113:0/1014022434 shutdown_connections 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764efbc90 0x561764ef7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764ef0d40 0x561764f07980 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 --2- 172.21.15.113:0/1014022434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764efb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 -- 172.21.15.113:0/1014022434 >> 172.21.15.113:0/1014022434 conn(0x561764eeacb0 msgr2=0x561764eec0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 -- 172.21.15.113:0/1014022434 shutdown_connections 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.503+0000 7f8e775a1740 1 -- 172.21.15.113:0/1014022434 wait complete. 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 -- start start 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 Processor -- start 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764f04080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764ef0d40 0x561764ef5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 0x561764ef5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701aa2e0 con 0x561764efbc90 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701ab750 con 0x561764ef0d40 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701aa9b0 con 0x561764ef0390 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e775a1740 10 client.0 _ll_register_callbacks cb 0x7f8e60014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 0x561764ef5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 0x561764ef5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55756/0 (socket says 172.21.15.113:55756) 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/954236884 learned_addr learned my addr 172.21.15.113:0/954236884 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e65d9b640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764ef0d40 0x561764ef5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/954236884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 msgr2=0x561764f04080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764f04080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764f04080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764ef0d40 msgr2=0x561764ef5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764ef0d40 0x561764ef5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 -- 172.21.15.113:0/954236884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561764ef5fe0 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e6ed9d640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 0x561764ef5aa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009dc0 tx=0x7f8e680be9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b36d4622ca16bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.504+0000 7f8e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e55ffb640 1 -- 172.21.15.113:0/954236884 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680c3440 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e55ffb640 1 -- 172.21.15.113:0/954236884 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f8e680c30d0 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58016070 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e55ffb640 1 -- 172.21.15.113:0/954236884 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8e680c7d00 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 10 client.17355 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561764f71d00 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e55ffb640 1 -- 172.21.15.113:0/954236884 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e58016070 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e55ffb640 1 -- 172.21.15.113:0/954236884 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f8e6803c030 con 0x561764efbc90 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 10 client.17355 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:26.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 20 client.17355 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f72460 0x561764f74850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 4 client.17355 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561764e5ca70 con 0x561764f72460 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.505+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f72460 0x561764f74850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e55ffb640 10 client.17355 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e6e59c640 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f72460 0x561764f74850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0da20641dbeed2 server_cookie=a61c62c20ac089a8 in_seq=0 out_seq=0 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e55ffb640 1 -- 172.21.15.113:0/954236884 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561764e5ca70 con 0x561764f72460 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e55ffb640 10 client.17355 handle_command_reply: tid=0 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 client.17355 shutdown 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 msgr2=0x561764ef5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 0x561764ef5aa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009dc0 tx=0x7f8e680be9f0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 shutdown_connections 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x561764f72460 0x561764f74850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561764efbc90 0x561764ef5aa0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561764ef0d40 0x561764ef5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 --2- 172.21.15.113:0/954236884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561764ef0390 0x561764f04080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 >> 172.21.15.113:0/954236884 conn(0x561764eeacb0 msgr2=0x561764eec0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e55ffb640 0 client.17355 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 shutdown_connections 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 1 -- 172.21.15.113:0/954236884 wait complete. 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.506+0000 7f8e775a1740 20 client.17355 trim_cache size 0 max 16384 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:22:26.511 INFO:teuthology.orchestra.run.smithi113.stdout: "9f81cfea-a450-4dca-afe7-719b9fcef1e9": { 2024-04-22T07:22:26.512 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:22:26.512 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "9f81cfea-a450-4dca-afe7-719b9fcef1e9", 2024-04-22T07:22:26.512 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:22:26.512 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:22:26.512 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:22:26.512 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8e5c0a3d20 msgr2=0x7f8e5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8e5c0a3d20 0x7f8e5c0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800a5c0 tx=0x7f8e580117b0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 msgr2=0x7f8e7019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 0x7f8e7019c380 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8e6802c700 tx=0x7f8e68035d80 comp rx=0 tx=0).stop 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 shutdown_connections 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8e5c0a3d20 0x7f8e5c0a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e7019bed0 0x7f8e7019c380 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70106130 0x7f8e70197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 --2- 172.21.15.113:0/1602308643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e70101060 0x7f8e70197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 >> 172.21.15.113:0/1602308643 conn(0x7f8e700771d0 msgr2=0x7f8e700fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 shutdown_connections 2024-04-22T07:22:26.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:26.507+0000 7f8e748d2640 1 -- 172.21.15.113:0/1602308643 wait complete. 2024-04-22T07:22:26.515 INFO:tasks.cephfs.filesystem:scrub status for tag:9f81cfea-a450-4dca-afe7-719b9fcef1e9 - {'path': '/', 'tag': '9f81cfea-a450-4dca-afe7-719b9fcef1e9', 'options': 'recursive,force'} 2024-04-22T07:22:27.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:27 smithi133 ceph-mon[31198]: pgmap v239: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:22:27.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:27 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:22:27.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:27 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:27.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:27 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:22:27.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:27 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:27.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:27 smithi154 ceph-mon[31862]: pgmap v239: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:22:27.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:27 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:22:27.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:27 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:27.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:27 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:22:27.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:27 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:27.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:27 smithi113 ceph-mon[27933]: pgmap v239: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:22:27.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:27 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:22:27.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:27 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:27.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:27 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:22:27.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:27 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:29.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:29 smithi133 ceph-mon[31198]: pgmap v240: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:22:29.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:29 smithi154 ceph-mon[31862]: pgmap v240: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:22:29.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:29 smithi113 ceph-mon[27933]: pgmap v240: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:22:31.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:31 smithi133 ceph-mon[31198]: pgmap v241: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:22:31.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:31 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:22:31.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:31 smithi154 ceph-mon[31862]: pgmap v241: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:22:31.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:31 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:22:31.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:31 smithi113 ceph-mon[27933]: pgmap v241: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:22:31.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:31 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:22:33.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:33 smithi133 ceph-mon[31198]: pgmap v242: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:22:33.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:33 smithi154 ceph-mon[31862]: pgmap v242: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:22:33.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:33 smithi113 ceph-mon[27933]: pgmap v242: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T07:22:35.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:35 smithi133 ceph-mon[31198]: pgmap v243: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:35.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:35 smithi154 ceph-mon[31862]: pgmap v243: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:35.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:35 smithi113 ceph-mon[27933]: pgmap v243: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:37.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:37 smithi133 ceph-mon[31198]: pgmap v244: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:37.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:37 smithi154 ceph-mon[31862]: pgmap v244: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:37.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:37 smithi113 ceph-mon[27933]: pgmap v244: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:39.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:39 smithi133 ceph-mon[31198]: pgmap v245: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:39.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:39 smithi154 ceph-mon[31862]: pgmap v245: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:39.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:39 smithi113 ceph-mon[27933]: pgmap v245: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:41.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:41 smithi133 ceph-mon[31198]: pgmap v246: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:41.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:41 smithi154 ceph-mon[31862]: pgmap v246: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:41.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:41 smithi113 ceph-mon[27933]: pgmap v246: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:22:43.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:43 smithi133 ceph-mon[31198]: pgmap v247: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:43.317 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:43 smithi154 ceph-mon[31862]: pgmap v247: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:43.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:43 smithi113 ceph-mon[27933]: pgmap v247: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:22:45.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:45 smithi133 ceph-mon[31198]: pgmap v248: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:45.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:45 smithi154 ceph-mon[31862]: pgmap v248: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:45.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:45 smithi113 ceph-mon[27933]: pgmap v248: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:47.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:47 smithi133 ceph-mon[31198]: pgmap v249: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:47.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:47 smithi154 ceph-mon[31862]: pgmap v249: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:47.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:47 smithi113 ceph-mon[27933]: pgmap v249: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:49.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:49 smithi133 ceph-mon[31198]: pgmap v250: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:49.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:49 smithi154 ceph-mon[31862]: pgmap v250: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:49.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:49 smithi113 ceph-mon[27933]: pgmap v250: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:51.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:51 smithi133 ceph-mon[31198]: pgmap v251: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:51.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:51 smithi154 ceph-mon[31862]: pgmap v251: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:51.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:51 smithi113 ceph-mon[27933]: pgmap v251: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:53.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:53 smithi133 ceph-mon[31198]: pgmap v252: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:53.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:53 smithi154 ceph-mon[31862]: pgmap v252: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:53.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:53 smithi113 ceph-mon[27933]: pgmap v252: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:55.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:55 smithi133 ceph-mon[31198]: pgmap v253: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:55.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:55 smithi154 ceph-mon[31862]: pgmap v253: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:55.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:55 smithi113 ceph-mon[27933]: pgmap v253: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:56.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:22:56.516 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 -- 172.21.15.113:0/1752643552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f10101a60 msgr2=0x7f1f10105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 --2- 172.21.15.113:0/1752643552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f10101a60 0x7f1f10105aa0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009900 tx=0x7f1f0402bc30 comp rx=0 tx=0).stop 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 -- 172.21.15.113:0/1752643552 shutdown_connections 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 --2- 172.21.15.113:0/1752643552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f10106180 0x7f1f1010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 --2- 172.21.15.113:0/1752643552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f10101a60 0x7f1f10105aa0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 --2- 172.21.15.113:0/1752643552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 0x7f1f10101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 -- 172.21.15.113:0/1752643552 >> 172.21.15.113:0/1752643552 conn(0x7f1f100771d0 msgr2=0x7f1f100775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 -- 172.21.15.113:0/1752643552 shutdown_connections 2024-04-22T07:22:56.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 -- 172.21.15.113:0/1752643552 wait complete. 2024-04-22T07:22:56.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.632+0000 7f1f15006640 1 Processor -- start 2024-04-22T07:22:56.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 -- start start 2024-04-22T07:22:56.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 0x7f1f10199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f10106180 0x7f1f10199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 0x7f1f101a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10111200 con 0x7f1f1019a150 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10111080 con 0x7f1f101010b0 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f15006640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10111380 con 0x7f1f10106180 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 0x7f1f101a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 0x7f1f101a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52378/0 (socket says 172.21.15.113:52378) 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 -- 172.21.15.113:0/3296798664 learned_addr learned my addr 172.21.15.113:0/3296798664 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 0x7f1f10199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f10106180 0x7f1f10199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 -- 172.21.15.113:0/3296798664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f10106180 msgr2=0x7f1f10199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f10106180 0x7f1f10199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 -- 172.21.15.113:0/3296798664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 msgr2=0x7f1f10199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 0x7f1f10199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 -- 172.21.15.113:0/3296798664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f101a4f00 con 0x7f1f1019a150 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 0x7f1f10199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.633+0000 7f1f0f59e640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 0x7f1f101a4700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800eaa0 tx=0x7f1f0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.634+0000 7f1ef7fff640 1 -- 172.21.15.113:0/3296798664 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800cdb0 con 0x7f1f1019a150 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.634+0000 7f1ef7fff640 1 -- 172.21.15.113:0/3296798664 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f08005020 con 0x7f1f1019a150 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.634+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f101a51f0 con 0x7f1f1019a150 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.634+0000 7f1ef7fff640 1 -- 172.21.15.113:0/3296798664 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f080116d0 con 0x7f1f1019a150 2024-04-22T07:22:56.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.634+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f101a56f0 con 0x7f1f1019a150 2024-04-22T07:22:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.636+0000 7f1ef7fff640 1 -- 172.21.15.113:0/3296798664 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1f080051d0 con 0x7f1f1019a150 2024-04-22T07:22:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.636+0000 7f1f17bf4740 1 Processor -- start 2024-04-22T07:22:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.636+0000 7f1f17bf4740 1 -- start start 2024-04-22T07:22:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.636+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe43540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fd9f1e0 con 0x55cd0fe3a580 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fd3c390 con 0x55cd0fe43a80 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fd9b1a0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1ef7fff640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1ef80a3d20 0x7f1ef80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe3a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52392/0 (socket says 172.21.15.113:52392) 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2756992999 learned_addr learned my addr 172.21.15.113:0/2756992999 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0f59e640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1ef80a3d20 0x7f1ef80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe43540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1ef7fff640 1 -- 172.21.15.113:0/3296798664 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f1f080d0c80 con 0x7f1f1019a150 2024-04-22T07:22:56.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2756992999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3aea0 msgr2=0x55cd0fe43540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe43540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2756992999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 msgr2=0x55cd0fe45e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2756992999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0fb7b7d0 con 0x55cd0fe3a580 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe3a960 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1f04035000 tx=0x7f1f04005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d936bd45cd80af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1f0f59e640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1ef80a3d20 0x7f1ef80a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f080ce2e0 tx=0x7f1f080bd040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1ef57fa640 1 -- 172.21.15.113:0/2756992999 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04043020 con 0x55cd0fe3a580 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1ef57fa640 1 -- 172.21.15.113:0/2756992999 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f0402ca50 con 0x55cd0fe3a580 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.637+0000 7f1ef57fa640 1 -- 172.21.15.113:0/2756992999 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f040046c0 con 0x55cd0fe3a580 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2756992999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 msgr2=0x55cd0fe3a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe3a960 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1f04035000 tx=0x7f1f04005e70 comp rx=0 tx=0).stop 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2756992999 shutdown_connections 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe45e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe43540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2756992999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe3a960 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2756992999 >> 172.21.15.113:0/2756992999 conn(0x55cd0fe34cb0 msgr2=0x55cd0fe35090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2756992999 shutdown_connections 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.638+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2756992999 wait complete. 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 -- start start 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 Processor -- start 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3a580 0x55cd0fe40ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fe46810 con 0x55cd0fe3aea0 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fd582f0 con 0x55cd0fe43a80 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cd0fe46990 con 0x55cd0fe3a580 2024-04-22T07:22:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f17bf4740 10 client.0 _ll_register_callbacks cb 0x55cd0fd65580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe41010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52408/0 (socket says 172.21.15.113:52408) 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 -- 172.21.15.113:0/3385265185 learned_addr learned my addr 172.21.15.113:0/3385265185 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3a580 0x55cd0fe40ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 -- 172.21.15.113:0/3385265185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3a580 msgr2=0x55cd0fe40ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3a580 0x55cd0fe40ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 -- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 msgr2=0x55cd0fe41550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 -- 172.21.15.113:0/3385265185 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0fab13f0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3a580 0x55cd0fe40ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe41010 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000bfb0 tx=0x7f1f00011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37538df4bfcc23df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.639+0000 7f1ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/3385265185 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00012710 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/3385265185 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f1f000128b0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0402ca50 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/3385265185 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1f00012ab0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 10 client.17367 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cd0fa710f0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/3385265185 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f0402ca50 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/3385265185 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f1f0001a3d0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 10 client.17367 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 20 client.17367 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0fe53b10 0x55cd0fe55f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 4 client.17367 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd0fdacc00 con 0x55cd0fe53b10 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.640+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0fe53b10 0x55cd0fe55f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.641+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0fe53b10 0x55cd0fe55f50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffde7080ca03f8e server_cookie=797a68ccc104f71c in_seq=0 out_seq=0 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.641+0000 7f1ef2ff5640 10 client.17367 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/3385265185 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd0fdacc00 con 0x55cd0fe53b10 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1ef2ff5640 10 client.17367 handle_command_reply: tid=0 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 client.17367 shutdown 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 msgr2=0x55cd0fe41010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe41010 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000bfb0 tx=0x7f1f00011870 comp rx=0 tx=0).stop 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 shutdown_connections 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0fe53b10 0x55cd0fe55f50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe41010 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1ef2ff5640 0 client.17367 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/3385265185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe3a580 0x55cd0fe40ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 >> 172.21.15.113:0/3385265185 conn(0x55cd0fe34cb0 msgr2=0x55cd0fe36a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 shutdown_connections 2024-04-22T07:22:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 1 -- 172.21.15.113:0/3385265185 wait complete. 2024-04-22T07:22:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.642+0000 7f1f17bf4740 20 client.17367 trim_cache size 0 max 16384 2024-04-22T07:22:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 Processor -- start 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 -- start start 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101b08c0 con 0x55cd0fe3a580 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101ae190 con 0x55cd0fe3aea0 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101ac000 con 0x55cd0fe43a80 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f0e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f0e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52412/0 (socket says 172.21.15.113:52412) 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.659+0000 7f1f0e59c640 1 -- 172.21.15.113:0/976055988 learned_addr learned my addr 172.21.15.113:0/976055988 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f0e59c640 1 -- 172.21.15.113:0/976055988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 msgr2=0x55cd0fe41370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f0e59c640 1 -- 172.21.15.113:0/976055988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 msgr2=0x55cd0fe3fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f0e59c640 1 -- 172.21.15.113:0/976055988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0fe3acd0 con 0x55cd0fe3a580 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4f700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f040053f0 tx=0x7f1f04005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e88c84c323c22a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1ef4ff9640 1 -- 172.21.15.113:0/976055988 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04043040 con 0x55cd0fe3a580 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1ef4ff9640 1 -- 172.21.15.113:0/976055988 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f0402c850 con 0x55cd0fe3a580 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1ef4ff9640 1 -- 172.21.15.113:0/976055988 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003d30 con 0x55cd0fe3a580 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f17bf4740 1 -- 172.21.15.113:0/976055988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 msgr2=0x55cd0fe4f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.660+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4f700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f040053f0 tx=0x7f1f04005d50 comp rx=0 tx=0).stop 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- 172.21.15.113:0/976055988 shutdown_connections 2024-04-22T07:22:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 0x55cd0fe41370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe3fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/976055988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4f700 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- 172.21.15.113:0/976055988 >> 172.21.15.113:0/976055988 conn(0x55cd0fe34cb0 msgr2=0x55cd0fe36000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- 172.21.15.113:0/976055988 shutdown_connections 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- 172.21.15.113:0/976055988 wait complete. 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- start start 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 Processor -- start 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 0x55cd0fe3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101ac650 con 0x55cd0fe3a580 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101adac0 con 0x55cd0fe3aea0 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101acd20 con 0x55cd0fe43a80 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f17bf4740 10 client.0 _ll_register_callbacks cb 0x7f1f0000b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52418/0 (socket says 172.21.15.113:52418) 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2107697101 learned_addr learned my addr 172.21.15.113:0/2107697101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2107697101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 msgr2=0x55cd0fe3d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 0x55cd0fe3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 msgr2=0x55cd0fe4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0e59c640 1 -- 172.21.15.113:0/2107697101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0fe3d9a0 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.661+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f0e59c640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4bc10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1f0402c170 tx=0x7f1f0402caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea1337317edbd6e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/2107697101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04059070 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/2107697101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f1f0403f470 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0402c850 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/2107697101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1f04043070 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 10 client.17373 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cd0feb9c00 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/2107697101 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f0402c850 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/2107697101 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f1f0404c570 con 0x55cd0fe3a580 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 10 client.17373 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 20 client.17373 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0feb9ed0 0x55cd0febc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 4 client.17373 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.662+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd0fda6a70 con 0x55cd0feb9ed0 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0feb9ed0 0x55cd0febc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f0ed9d640 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0feb9ed0 0x55cd0febc2c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddcac26ade32f90 server_cookie=1f8544914e9ca88b in_seq=0 out_seq=0 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1ef2ff5640 10 client.17373 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1ef2ff5640 1 -- 172.21.15.113:0/2107697101 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd0fda6a70 con 0x55cd0feb9ed0 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1ef2ff5640 10 client.17373 handle_command_reply: tid=0 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 client.17373 shutdown 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 msgr2=0x55cd0fe4bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4bc10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1f0402c170 tx=0x7f1f0402caf0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 shutdown_connections 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55cd0feb9ed0 0x55cd0febc2c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cd0fe43a80 0x55cd0fe3d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd0fe3aea0 0x55cd0fe4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 --2- 172.21.15.113:0/2107697101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cd0fe3a580 0x55cd0fe4bc10 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1ef2ff5640 0 client.17373 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 >> 172.21.15.113:0/2107697101 conn(0x55cd0fe34cb0 msgr2=0x55cd0fe33d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.663+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 shutdown_connections 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f17bf4740 1 -- 172.21.15.113:0/2107697101 wait complete. 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f17bf4740 20 client.17373 trim_cache size 0 max 16384 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:22:56.669 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:22:56.670 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:22:56.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1ef80a3d20 msgr2=0x7f1ef80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1ef80a3d20 0x7f1ef80a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f080ce2e0 tx=0x7f1f080bd040 comp rx=0 tx=0).stop 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 msgr2=0x7f1f101a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 0x7f1f101a4700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800eaa0 tx=0x7f1f0800ef60 comp rx=0 tx=0).stop 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 shutdown_connections 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1ef80a3d20 0x7f1ef80a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1019a150 0x7f1f101a4700 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f10106180 0x7f1f10199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 --2- 172.21.15.113:0/3296798664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f101010b0 0x7f1f10199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 >> 172.21.15.113:0/3296798664 conn(0x7f1f100771d0 msgr2=0x7f1f100fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 shutdown_connections 2024-04-22T07:22:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.664+0000 7f1f15006640 1 -- 172.21.15.113:0/3296798664 wait complete. 2024-04-22T07:22:56.672 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:22:56.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:22:56.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 -- 172.21.15.113:0/1305560940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8101a50 msgr2=0x7f09d8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/1305560940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8101a50 0x7f09d8105a90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009900 tx=0x7f09cc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 -- 172.21.15.113:0/1305560940 shutdown_connections 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/1305560940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d8106170 0x7f09d810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/1305560940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8101a50 0x7f09d8105a90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/1305560940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 0x7f09d8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 -- 172.21.15.113:0/1305560940 >> 172.21.15.113:0/1305560940 conn(0x7f09d80771d0 msgr2=0x7f09d80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 -- 172.21.15.113:0/1305560940 shutdown_connections 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.773+0000 7f09dc9e6640 1 -- 172.21.15.113:0/1305560940 wait complete. 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 Processor -- start 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 -- start start 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 0x7f09d819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d8106170 0x7f09d819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 0x7f09d8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09d81111f0 con 0x7f09d8198650 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09d8111070 con 0x7f09d8106170 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09dc9e6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f09d8111370 con 0x7f09d81010a0 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 0x7f09d8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 0x7f09d8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52444/0 (socket says 172.21.15.113:52444) 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 -- 172.21.15.113:0/3107697638 learned_addr learned my addr 172.21.15.113:0/3107697638 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d5d9b640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d8106170 0x7f09d819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d659c640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 0x7f09d819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 -- 172.21.15.113:0/3107697638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 msgr2=0x7f09d819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 0x7f09d819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 -- 172.21.15.113:0/3107697638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d8106170 msgr2=0x7f09d819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d8106170 0x7f09d819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d6d9d640 1 -- 172.21.15.113:0/3107697638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d8199250 con 0x7f09d8198650 2024-04-22T07:22:56.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.774+0000 7f09d659c640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 0x7f09d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.775+0000 7f09d6d9d640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 0x7f09d8198ad0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009fd0 tx=0x7f09d000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.775+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0017070 con 0x7f09d8198650 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.775+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d81a5330 con 0x7f09d8198650 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.775+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f09d0003e40 con 0x7f09d8198650 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.775+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0004820 con 0x7f09d8198650 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.775+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d81a56e0 con 0x7f09d8198650 2024-04-22T07:22:56.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.776+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d8102570 con 0x7f09d8198650 2024-04-22T07:22:56.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.777+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f09d0003970 con 0x7f09d8198650 2024-04-22T07:22:56.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.777+0000 7f09c77fe640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09c00a3d20 0x7f09c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:56.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.777+0000 7f09d659c640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09c00a3d20 0x7f09c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:56.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.777+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f09d00d0700 con 0x7f09d8198650 2024-04-22T07:22:56.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.778+0000 7f09d659c640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09c00a3d20 0x7f09c00a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009c20 tx=0x7f09c8005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:56.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.782+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09d0003c60 con 0x7f09d8198650 2024-04-22T07:22:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.998+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d8069230 con 0x7f09d8198650 2024-04-22T07:22:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:56.999+0000 7f09c77fe640 1 -- 172.21.15.113:0/3107697638 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7f09d009cc00 con 0x7f09d8198650 2024-04-22T07:22:56.997 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:22:56.997 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:22:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:22:56.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09c00a3d20 msgr2=0x7f09c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:56.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09c00a3d20 0x7f09c00a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009c20 tx=0x7f09c8005f90 comp rx=0 tx=0).stop 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 msgr2=0x7f09d8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 0x7f09d8198ad0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009fd0 tx=0x7f09d000b260 comp rx=0 tx=0).stop 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 shutdown_connections 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f09c00a3d20 0x7f09c00a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d8198650 0x7f09d8198ad0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d8106170 0x7f09d819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 --2- 172.21.15.113:0/3107697638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09d81010a0 0x7f09d819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 >> 172.21.15.113:0/3107697638 conn(0x7f09d80771d0 msgr2=0x7f09d806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 shutdown_connections 2024-04-22T07:22:57.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.001+0000 7f09dc9e6640 1 -- 172.21.15.113:0/3107697638 wait complete. 2024-04-22T07:22:57.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:22:57.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 -- 172.21.15.113:0/537356538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406b1f0 msgr2=0x7fea44106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 --2- 172.21.15.113:0/537356538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406b1f0 0x7fea44106b90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fea38009900 tx=0x7fea3802bc30 comp rx=0 tx=0).stop 2024-04-22T07:22:57.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 -- 172.21.15.113:0/537356538 shutdown_connections 2024-04-22T07:22:57.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 --2- 172.21.15.113:0/537356538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea44107270 0x7fea4410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 --2- 172.21.15.113:0/537356538 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406b1f0 0x7fea44106b90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 --2- 172.21.15.113:0/537356538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea4406a840 0x7fea4406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 -- 172.21.15.113:0/537356538 >> 172.21.15.113:0/537356538 conn(0x7fea44077eb0 msgr2=0x7fea440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 -- 172.21.15.113:0/537356538 shutdown_connections 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.113+0000 7fea49f4b640 1 -- 172.21.15.113:0/537356538 wait complete. 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 Processor -- start 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 -- start start 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 0x7fea4419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea44107270 0x7fea4419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44198650 0x7fea44198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea44112350 con 0x7fea4406a840 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea441121d0 con 0x7fea44107270 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea49f4b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea441124d0 con 0x7fea44198650 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 0x7fea4419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 0x7fea4419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52474/0 (socket says 172.21.15.113:52474) 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea437fe640 1 -- 172.21.15.113:0/1992768111 learned_addr learned my addr 172.21.15.113:0/1992768111 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea42ffd640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea44107270 0x7fea4419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.114+0000 7fea43fff640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44198650 0x7fea44198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea437fe640 1 -- 172.21.15.113:0/1992768111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44198650 msgr2=0x7fea44198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea437fe640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44198650 0x7fea44198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea437fe640 1 -- 172.21.15.113:0/1992768111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea44107270 msgr2=0x7fea4419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea437fe640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea44107270 0x7fea4419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea437fe640 1 -- 172.21.15.113:0/1992768111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea44199170 con 0x7fea4406a840 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea43fff640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44198650 0x7fea44198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea437fe640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 0x7fea4419e450 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fea3400b4a0 tx=0x7fea3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea40ff9640 1 -- 172.21.15.113:0/1992768111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34014070 con 0x7fea4406a840 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea40ff9640 1 -- 172.21.15.113:0/1992768111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea34010ae0 con 0x7fea4406a840 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea44199460 con 0x7fea4406a840 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea40ff9640 1 -- 172.21.15.113:0/1992768111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34017ce0 con 0x7fea4406a840 2024-04-22T07:22:57.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.115+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441a5660 con 0x7fea4406a840 2024-04-22T07:22:57.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.117+0000 7fea40ff9640 1 -- 172.21.15.113:0/1992768111 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fea34017070 con 0x7fea4406a840 2024-04-22T07:22:57.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.117+0000 7fea4cc13740 1 Processor -- start 2024-04-22T07:22:57.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.117+0000 7fea4cc13740 1 -- start start 2024-04-22T07:22:57.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bfe80 0x55fc294c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea4cc13740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fc294241f0 con 0x55fc294bf4d0 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea4cc13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc293c14b0 con 0x55fc294bfe80 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea4cc13740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fc294201b0 con 0x55fc294c8a00 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294bf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52486/0 (socket says 172.21.15.113:52486) 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 -- 172.21.15.113:0/2190670732 learned_addr learned my addr 172.21.15.113:0/2190670732 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea437fe640 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bfe80 0x55fc294c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea40ff9640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea2c0a3da0 0x7fea2c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 -- 172.21.15.113:0/2190670732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 msgr2=0x55fc294cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 -- 172.21.15.113:0/2190670732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bfe80 msgr2=0x55fc294c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bfe80 0x55fc294c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 -- 172.21.15.113:0/2190670732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc293b31b0 con 0x55fc294bf4d0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea40ff9640 1 -- 172.21.15.113:0/1992768111 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7fea340cfb90 con 0x7fea4406a840 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea2c0a3da0 0x7fea2c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.118+0000 7fea43fff640 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294bf8b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e450 tx=0x7fea3c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d093e5aef94c5806 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea29ffb640 1 -- 172.21.15.113:0/2190670732 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c022070 con 0x55fc294bf4d0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea29ffb640 1 -- 172.21.15.113:0/2190670732 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea3c018070 con 0x55fc294bf4d0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea29ffb640 1 -- 172.21.15.113:0/2190670732 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0055f0 con 0x55fc294bf4d0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 -- 172.21.15.113:0/2190670732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 msgr2=0x55fc294bf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea43fff640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea2c0a3da0 0x7fea2c0a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea3c01f000 tx=0x7fea3c020000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294bf8b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e450 tx=0x7fea3c00e910 comp rx=0 tx=0).stop 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 -- 172.21.15.113:0/2190670732 shutdown_connections 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bfe80 0x55fc294c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2190670732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294bf8b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 -- 172.21.15.113:0/2190670732 >> 172.21.15.113:0/2190670732 conn(0x55fc294b9ba0 msgr2=0x55fc294b9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 -- 172.21.15.113:0/2190670732 shutdown_connections 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.119+0000 7fea4cc13740 1 -- 172.21.15.113:0/2190670732 wait complete. 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 -- start start 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 Processor -- start 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294bfe80 0x55fc294c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294c8a00 0x55fc294c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fc294cb970 con 0x55fc294bf4d0 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc293dd500 con 0x55fc294c8a00 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fc294cbaf0 con 0x55fc294bfe80 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea4cc13740 10 client.0 _ll_register_callbacks cb 0x55fc293abb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52498/0 (socket says 172.21.15.113:52498) 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 -- 172.21.15.113:0/511023521 learned_addr learned my addr 172.21.15.113:0/511023521 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea42ffd640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294bfe80 0x55fc294c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea43fff640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294c8a00 0x55fc294c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 -- 172.21.15.113:0/511023521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294bfe80 msgr2=0x55fc294c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294bfe80 0x55fc294c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 -- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294c8a00 msgr2=0x55fc294c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294c8a00 0x55fc294c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.120+0000 7fea437fe640 1 -- 172.21.15.113:0/511023521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc291363f0 con 0x55fc294bf4d0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea42ffd640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294bfe80 0x55fc294c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea43fff640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294c8a00 0x55fc294c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea437fe640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294c82d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fea34097f00 tx=0x7fea34009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd32c53aebb331f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea22ffd640 1 -- 172.21.15.113:0/511023521 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3401e070 con 0x55fc294bf4d0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea22ffd640 1 -- 172.21.15.113:0/511023521 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7fea34017360 con 0x55fc294bf4d0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c018070 con 0x55fc294bf4d0 2024-04-22T07:22:57.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea22ffd640 1 -- 172.21.15.113:0/511023521 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fea340bc0f0 con 0x55fc294bf4d0 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 10 client.17391 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fc292007d0 con 0x55fc294bf4d0 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea22ffd640 1 -- 172.21.15.113:0/511023521 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea3c018070 con 0x55fc294bf4d0 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea22ffd640 1 -- 172.21.15.113:0/511023521 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fea3409aef0 con 0x55fc294bf4d0 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 10 client.17391 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 20 client.17391 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc294d8b20 0x55fc294daf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 4 client.17391 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.121+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc293c3930 con 0x55fc294d8b20 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.122+0000 7fea42ffd640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc294d8b20 0x55fc294daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.122+0000 7fea22ffd640 10 client.17391 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:57.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.122+0000 7fea42ffd640 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc294d8b20 0x55fc294daf30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=affdf2a5d295d7b3 server_cookie=5e38664518a487d6 in_seq=0 out_seq=0 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea22ffd640 1 -- 172.21.15.113:0/511023521 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc293c3930 con 0x55fc294d8b20 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea22ffd640 10 client.17391 handle_command_reply: tid=0 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 client.17391 shutdown 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 msgr2=0x55fc294c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294c82d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fea34097f00 tx=0x7fea34009370 comp rx=0 tx=0).stop 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 shutdown_connections 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc294d8b20 0x55fc294daf30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294c8a00 0x55fc294c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294bfe80 0x55fc294c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea22ffd640 0 client.17391 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 --2- 172.21.15.113:0/511023521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bf4d0 0x55fc294c82d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 >> 172.21.15.113:0/511023521 conn(0x55fc294b9ba0 msgr2=0x55fc294bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 shutdown_connections 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 1 -- 172.21.15.113:0/511023521 wait complete. 2024-04-22T07:22:57.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.123+0000 7fea4cc13740 20 client.17391 trim_cache size 0 max 16384 2024-04-22T07:22:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 Processor -- start 2024-04-22T07:22:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 -- start start 2024-04-22T07:22:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea441b0810 con 0x55fc294bfe80 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea441ae0e0 con 0x55fc294bf4d0 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea4cc13740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea441abf50 con 0x55fc294c8a00 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294d4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52504/0 (socket says 172.21.15.113:52504) 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.136+0000 7fea42ffd640 1 -- 172.21.15.113:0/2154017175 learned_addr learned my addr 172.21.15.113:0/2154017175 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea437fe640 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea43fff640 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea42ffd640 1 -- 172.21.15.113:0/2154017175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 msgr2=0x55fc294d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea42ffd640 1 -- 172.21.15.113:0/2154017175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 msgr2=0x55fc294c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea42ffd640 1 -- 172.21.15.113:0/2154017175 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc294db560 con 0x55fc294bfe80 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea43fff640 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294d4830 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fea38009230 tx=0x7fea38005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8f529702b3e3de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea297fa640 1 -- 172.21.15.113:0/2154017175 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38045bc0 con 0x55fc294bfe80 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea297fa640 1 -- 172.21.15.113:0/2154017175 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea3804a070 con 0x55fc294bfe80 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea297fa640 1 -- 172.21.15.113:0/2154017175 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea380452c0 con 0x55fc294bfe80 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.137+0000 7fea4cc13740 1 -- 172.21.15.113:0/2154017175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 msgr2=0x55fc294d4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294d4830 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fea38009230 tx=0x7fea38005fd0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- 172.21.15.113:0/2154017175 shutdown_connections 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294d4830 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2154017175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- 172.21.15.113:0/2154017175 >> 172.21.15.113:0/2154017175 conn(0x55fc294b9ba0 msgr2=0x55fc294c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- 172.21.15.113:0/2154017175 shutdown_connections 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- 172.21.15.113:0/2154017175 wait complete. 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- start start 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 Processor -- start 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea441ac5a0 con 0x55fc294bfe80 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea441ada10 con 0x55fc294bf4d0 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fea441acc70 con 0x55fc294c8a00 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea4cc13740 10 client.0 _ll_register_callbacks cb 0x7fea340d33d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea42ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52518/0 (socket says 172.21.15.113:52518) 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.138+0000 7fea42ffd640 1 -- 172.21.15.113:0/2615316678 learned_addr learned my addr 172.21.15.113:0/2615316678 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea437fe640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea42ffd640 1 -- 172.21.15.113:0/2615316678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 msgr2=0x55fc294c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea42ffd640 1 -- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 msgr2=0x55fc294c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea42ffd640 1 -- 172.21.15.113:0/2615316678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc294c2120 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea437fe640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea42ffd640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294c3130 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fea38036040 tx=0x7fea38004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf32c82381529ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea22ffd640 1 -- 172.21.15.113:0/2615316678 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38052020 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3804a070 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea22ffd640 1 -- 172.21.15.113:0/2615316678 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7fea38005240 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea22ffd640 1 -- 172.21.15.113:0/2615316678 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fea3805d050 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 10 client.17397 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fc295439f0 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea22ffd640 1 -- 172.21.15.113:0/2615316678 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fea3804a070 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea22ffd640 1 -- 172.21.15.113:0/2615316678 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fea38003680 con 0x55fc294bfe80 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 10 client.17397 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:57.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 20 client.17397 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc29544140 0x55fc29546560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 4 client.17397 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.139+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc29431c10 con 0x55fc29544140 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea437fe640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc29544140 0x55fc29546560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea437fe640 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc29544140 0x55fc29546560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22960bc0c79e477 server_cookie=2da31bcfa861ea97 in_seq=0 out_seq=0 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea22ffd640 10 client.17397 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea22ffd640 1 -- 172.21.15.113:0/2615316678 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc29431c10 con 0x55fc29544140 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea22ffd640 10 client.17397 handle_command_reply: tid=0 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 client.17397 shutdown 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 msgr2=0x55fc294c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294c3130 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fea38036040 tx=0x7fea38004170 comp rx=0 tx=0).stop 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 shutdown_connections 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55fc29544140 0x55fc29546560 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fc294c8a00 0x55fc294c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc294bfe80 0x55fc294c3130 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea22ffd640 0 client.17397 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 --2- 172.21.15.113:0/2615316678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc294bf4d0 0x55fc294c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.140+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 >> 172.21.15.113:0/2615316678 conn(0x55fc294b9ba0 msgr2=0x55fc294bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 shutdown_connections 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea4cc13740 1 -- 172.21.15.113:0/2615316678 wait complete. 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea4cc13740 20 client.17397 trim_cache size 0 max 16384 2024-04-22T07:22:57.145 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:22:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea2c0a3da0 msgr2=0x7fea2c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea2c0a3da0 0x7fea2c0a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea3c01f000 tx=0x7fea3c020000 comp rx=0 tx=0).stop 2024-04-22T07:22:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 msgr2=0x7fea4419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 0x7fea4419e450 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fea3400b4a0 tx=0x7fea3400b960 comp rx=0 tx=0).stop 2024-04-22T07:22:57.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 shutdown_connections 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fea2c0a3da0 0x7fea2c0a6260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44198650 0x7fea44198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea44107270 0x7fea4419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 --2- 172.21.15.113:0/1992768111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea4406a840 0x7fea4419e450 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 >> 172.21.15.113:0/1992768111 conn(0x7fea44077eb0 msgr2=0x7fea440fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 shutdown_connections 2024-04-22T07:22:57.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.141+0000 7fea49f4b640 1 -- 172.21.15.113:0/1992768111 wait complete. 2024-04-22T07:22:57.149 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 -- 172.21.15.113:0/1290417020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c101090 msgr2=0x7f1a3c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 --2- 172.21.15.113:0/1290417020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c101090 0x7f1a3c101470 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009900 tx=0x7f1a2c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 -- 172.21.15.113:0/1290417020 shutdown_connections 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 --2- 172.21.15.113:0/1290417020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a3c106160 0x7f1a3c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 --2- 172.21.15.113:0/1290417020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c101a40 0x7f1a3c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 --2- 172.21.15.113:0/1290417020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c101090 0x7f1a3c101470 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 -- 172.21.15.113:0/1290417020 >> 172.21.15.113:0/1290417020 conn(0x7f1a3c077f50 msgr2=0x7f1a3c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 -- 172.21.15.113:0/1290417020 shutdown_connections 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.251+0000 7f1a42277640 1 -- 172.21.15.113:0/1290417020 wait complete. 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 Processor -- start 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 -- start start 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a3c101090 0x7f1a3c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c106160 0x7f1a3c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 0x7f1a3c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1111e0 con 0x7f1a3c19c000 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111060 con 0x7f1a3c106160 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a42277640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111360 con 0x7f1a3c101090 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a40814640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 0x7f1a3c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a40814640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 0x7f1a3c19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52542/0 (socket says 172.21.15.113:52542) 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.252+0000 7f1a40814640 1 -- 172.21.15.113:0/1924657896 learned_addr learned my addr 172.21.15.113:0/1924657896 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c106160 0x7f1a3c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a3c101090 0x7f1a3c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a40814640 1 -- 172.21.15.113:0/1924657896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a3c101090 msgr2=0x7f1a3c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a40814640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a3c101090 0x7f1a3c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a40814640 1 -- 172.21.15.113:0/1924657896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c106160 msgr2=0x7f1a3c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a40814640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c106160 0x7f1a3c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a40814640 1 -- 172.21.15.113:0/1924657896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c19caf0 con 0x7f1a3c19c000 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a40814640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 0x7f1a3c19c4b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400eaa0 tx=0x7f1a3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a397fa640 1 -- 172.21.15.113:0/1924657896 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400cdb0 con 0x7f1a3c19c000 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c19cde0 con 0x7f1a3c19c000 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a397fa640 1 -- 172.21.15.113:0/1924657896 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1a34005020 con 0x7f1a3c19c000 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a397fa640 1 -- 172.21.15.113:0/1924657896 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a340116d0 con 0x7f1a3c19c000 2024-04-22T07:22:57.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.253+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1a4580 con 0x7f1a3c19c000 2024-04-22T07:22:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.255+0000 7f1a397fa640 1 -- 172.21.15.113:0/1924657896 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1a340051d0 con 0x7f1a3c19c000 2024-04-22T07:22:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 Processor -- start 2024-04-22T07:22:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a397fa640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a240a3d20 0x7f1a240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 -- start start 2024-04-22T07:22:57.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a240a3d20 0x7f1a240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a397fa640 1 -- 172.21.15.113:0/1924657896 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f1a340d0bc0 con 0x7f1a3c19c000 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 0x564cdfcf5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564cdfc531f0 con 0x564cdfceecf0 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cdfbf04b0 con 0x564cdfcee340 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564cdfc4f1b0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 0x564cdfcf5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 0x564cdfcf5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52548/0 (socket says 172.21.15.113:52548) 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 -- 172.21.15.113:0/2273281163 learned_addr learned my addr 172.21.15.113:0/2273281163 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a240a3d20 0x7f1a240a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012500 tx=0x7f1a30012900 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 -- 172.21.15.113:0/2273281163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 msgr2=0x564cdfcf6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 -- 172.21.15.113:0/2273281163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 msgr2=0x564cdfcee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 -- 172.21.15.113:0/2273281163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cdfbe21b0 con 0x564cdfceecf0 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.256+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 0x564cdfcf5620 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012e00 tx=0x7f1a3001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c079e52b8384d6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a227fc640 1 -- 172.21.15.113:0/2273281163 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000ed60 con 0x564cdfceecf0 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a227fc640 1 -- 172.21.15.113:0/2273281163 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1a30023020 con 0x564cdfceecf0 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a227fc640 1 -- 172.21.15.113:0/2273281163 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3001fb50 con 0x564cdfceecf0 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2273281163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 msgr2=0x564cdfcf5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 0x564cdfcf5620 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012e00 tx=0x7f1a3001e000 comp rx=0 tx=0).stop 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2273281163 shutdown_connections 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfceecf0 0x564cdfcf5620 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2273281163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2273281163 >> 172.21.15.113:0/2273281163 conn(0x564cdfce8ba0 msgr2=0x564cdfce8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2273281163 shutdown_connections 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.257+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2273281163 wait complete. 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 -- start start 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 Processor -- start 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcfc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfcf9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564cdfcfcb30 con 0x564cdfcee340 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cdfc0c500 con 0x564cdfceecf0 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564cdfcfccb0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a44e5f740 10 client.0 _ll_register_callbacks cb 0x564cdfbdab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:57.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcfc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcfc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52558/0 (socket says 172.21.15.113:52558) 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 -- 172.21.15.113:0/685643419 learned_addr learned my addr 172.21.15.113:0/685643419 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfcf9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a40814640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 -- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 msgr2=0x564cdfcf9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 -- 172.21.15.113:0/685643419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 msgr2=0x564cdfcf9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfcf9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3bfff640 1 -- 172.21.15.113:0/685643419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cdf9653f0 con 0x564cdfcee340 2024-04-22T07:22:57.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfcf9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.258+0000 7f1a40814640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcfc0b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c02c170 tx=0x7f1a2c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5de52d756cf290 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/685643419 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c005030 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/685643419 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f1a2c0051d0 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a30023020 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/685643419 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1a2c0053f0 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 10 client.17409 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564cdfa2f7d0 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/685643419 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1a30023020 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/685643419 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f1a2c0038c0 con 0x564cdfcee340 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 10 client.17409 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:57.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 10 client.17409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 20 client.17409 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd09e50 0x564cdfd0c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 4 client.17409 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.259+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cdfbf2930 con 0x564cdfd09e50 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.260+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd09e50 0x564cdfd0c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.260+0000 7f1a1b7fe640 10 client.17409 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.260+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd09e50 0x564cdfd0c2b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=246afa30c4f3cc2c server_cookie=928c027a01bf8721 in_seq=0 out_seq=0 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/685643419 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cdfbf2930 con 0x564cdfd09e50 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a1b7fe640 10 client.17409 handle_command_reply: tid=0 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 client.17409 shutdown 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 msgr2=0x564cdfcfc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcfc0b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c02c170 tx=0x7f1a2c005dd0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 shutdown_connections 2024-04-22T07:22:57.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd09e50 0x564cdfd0c2b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfcf9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfcf9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a1b7fe640 0 client.17409 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/685643419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcfc0b0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 >> 172.21.15.113:0/685643419 conn(0x564cdfce8ba0 msgr2=0x564cdfcea8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 shutdown_connections 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 1 -- 172.21.15.113:0/685643419 wait complete. 2024-04-22T07:22:57.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.261+0000 7f1a44e5f740 20 client.17409 trim_cache size 0 max 16384 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 Processor -- start 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 -- start start 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcf7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfd05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfce7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ae650 con 0x564cdfcee340 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1abf20 con 0x564cdfceecf0 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1a9d90 con 0x564cdfcf5bf0 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.274+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcf7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcf7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52570/0 (socket says 172.21.15.113:52570) 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 -- 172.21.15.113:0/2604340661 learned_addr learned my addr 172.21.15.113:0/2604340661 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a40814640 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfce7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 -- 172.21.15.113:0/2604340661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 msgr2=0x564cdfce7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfce7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 -- 172.21.15.113:0/2604340661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 msgr2=0x564cdfd05a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfd05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 -- 172.21.15.113:0/2604340661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cdfcef160 con 0x564cdfcee340 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcf7060 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006010 tx=0x7f1a2c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588637b5c54f9e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a21ffb640 1 -- 172.21.15.113:0/2604340661 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c005590 con 0x564cdfcee340 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a21ffb640 1 -- 172.21.15.113:0/2604340661 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1a2c003ac0 con 0x564cdfcee340 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.275+0000 7f1a21ffb640 1 -- 172.21.15.113:0/2604340661 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c034d60 con 0x564cdfcee340 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2604340661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 msgr2=0x564cdfcf7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcf7060 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006010 tx=0x7f1a2c001e70 comp rx=0 tx=0).stop 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2604340661 shutdown_connections 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfcf5bf0 0x564cdfce7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfceecf0 0x564cdfd05a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/2604340661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcee340 0x564cdfcf7060 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2604340661 >> 172.21.15.113:0/2604340661 conn(0x564cdfce8ba0 msgr2=0x564cdfcf7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2604340661 shutdown_connections 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- 172.21.15.113:0/2604340661 wait complete. 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- start start 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 Processor -- start 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcfb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfceecf0 0x564cdfcfb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 0x564cdfcfbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1aa3e0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ab850 con 0x564cdfcee340 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1aaab0 con 0x564cdfceecf0 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a44e5f740 10 client.0 _ll_register_callbacks cb 0x7f1a2c02cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a40814640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 0x564cdfcfbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a40814640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 0x564cdfcfbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52572/0 (socket says 172.21.15.113:52572) 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.276+0000 7f1a40814640 1 -- 172.21.15.113:0/3708487097 learned_addr learned my addr 172.21.15.113:0/3708487097 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfceecf0 0x564cdfcfb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a40814640 1 -- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfceecf0 msgr2=0x564cdfcfb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcfb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a40814640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfceecf0 0x564cdfcfb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a40814640 1 -- 172.21.15.113:0/3708487097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 msgr2=0x564cdfcfb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a40814640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcfb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a40814640 1 -- 172.21.15.113:0/3708487097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cdfd6f6c0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a3b7fe640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfceecf0 0x564cdfcfb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a40814640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 0x564cdfcfbda0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1a34098e20 tx=0x7f1a3409d180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31457428c8bba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/3708487097 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3401f040 con 0x564cdfcf5bf0 2024-04-22T07:22:57.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/3708487097 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f1a340bbb80 con 0x564cdfcf5bf0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a2c003ac0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/3708487097 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1a340a5630 con 0x564cdfcf5bf0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 10 client.17415 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564cdfd709c0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/3708487097 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1a2c003ac0 con 0x564cdfcf5bf0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/3708487097 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f1a3400f040 con 0x564cdfcf5bf0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 10 client.17415 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 20 client.17415 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd710e0 0x564cdfd734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 4 client.17415 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.277+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cdfc60c10 con 0x564cdfd710e0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd710e0 0x564cdfd734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a3bfff640 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd710e0 0x564cdfd734d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ea08c2be2256690 server_cookie=d0cc3a49dce01e0e in_seq=0 out_seq=0 2024-04-22T07:22:57.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a1b7fe640 10 client.17415 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a1b7fe640 1 -- 172.21.15.113:0/3708487097 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cdfc60c10 con 0x564cdfd710e0 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a1b7fe640 10 client.17415 handle_command_reply: tid=0 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a44e5f740 1 client.17415 shutdown 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.278+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 msgr2=0x564cdfcfbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 0x564cdfcfbda0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1a34098e20 tx=0x7f1a3409d180 comp rx=0 tx=0).stop 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 shutdown_connections 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x564cdfd710e0 0x564cdfd734d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564cdfcf5bf0 0x564cdfcfbda0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564cdfceecf0 0x564cdfcfb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 --2- 172.21.15.113:0/3708487097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cdfcee340 0x564cdfcfb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a1b7fe640 0 client.17415 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 >> 172.21.15.113:0/3708487097 conn(0x564cdfce8ba0 msgr2=0x564cdfcf8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 shutdown_connections 2024-04-22T07:22:57.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 1 -- 172.21.15.113:0/3708487097 wait complete. 2024-04-22T07:22:57.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a44e5f740 20 client.17415 trim_cache size 0 max 16384 2024-04-22T07:22:57.285 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a240a3d20 msgr2=0x7f1a240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a240a3d20 0x7f1a240a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1a30012500 tx=0x7f1a30012900 comp rx=0 tx=0).stop 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 msgr2=0x7f1a3c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 0x7f1a3c19c4b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400eaa0 tx=0x7f1a3400ef60 comp rx=0 tx=0).stop 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 shutdown_connections 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a240a3d20 0x7f1a240a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a3c19c000 0x7f1a3c19c4b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a3c106160 0x7f1a3c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 --2- 172.21.15.113:0/1924657896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a3c101090 0x7f1a3c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 >> 172.21.15.113:0/1924657896 conn(0x7f1a3c077f50 msgr2=0x7f1a3c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 shutdown_connections 2024-04-22T07:22:57.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.279+0000 7f1a42277640 1 -- 172.21.15.113:0/1924657896 wait complete. 2024-04-22T07:22:57.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:57 smithi113 ceph-mon[27933]: pgmap v254: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:57.287 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:57 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3107697638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:22:57.288 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:22:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:57 smithi133 ceph-mon[31198]: pgmap v254: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:57.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:57 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3107697638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:22:57.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:57 smithi154 ceph-mon[31862]: pgmap v254: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:22:57.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:57 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3107697638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:22:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 -- 172.21.15.113:0/42912422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7324107270 msgr2=0x7f732410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 --2- 172.21.15.113:0/42912422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7324107270 0x7f732410ee40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f731c009ca0 tx=0x7f731c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 -- 172.21.15.113:0/42912422 shutdown_connections 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 --2- 172.21.15.113:0/42912422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7324107270 0x7f732410ee40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 --2- 172.21.15.113:0/42912422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f732406b230 0x7f7324106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 --2- 172.21.15.113:0/42912422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f732406a880 0x7f732406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 -- 172.21.15.113:0/42912422 >> 172.21.15.113:0/42912422 conn(0x7f7324077eb0 msgr2=0x7f73240782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 -- 172.21.15.113:0/42912422 shutdown_connections 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 -- 172.21.15.113:0/42912422 wait complete. 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 Processor -- start 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.391+0000 7f732a44e640 1 -- start start 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 0x7f7324198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7324107270 0x7f7324198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f732419cfc0 0x7f732419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73241122f0 con 0x7f732406a880 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7324112170 con 0x7f732419cfc0 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7324112470 con 0x7f7324107270 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 0x7f7324198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 0x7f7324198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52586/0 (socket says 172.21.15.113:52586) 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 -- 172.21.15.113:0/2019211430 learned_addr learned my addr 172.21.15.113:0/2019211430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f73237fe640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7324107270 0x7f7324198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f73289eb640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f732419cfc0 0x7f732419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 -- 172.21.15.113:0/2019211430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7324107270 msgr2=0x7f7324198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7324107270 0x7f7324198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 -- 172.21.15.113:0/2019211430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f732419cfc0 msgr2=0x7f732419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f732419cfc0 0x7f732419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 -- 172.21.15.113:0/2019211430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419dab0 con 0x7f732406a880 2024-04-22T07:22:57.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f73237fe640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7324107270 0x7f7324198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f7323fff640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 0x7f7324198750 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f731800c920 tx=0x7f731800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f73217fa640 1 -- 172.21.15.113:0/2019211430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318007c70 con 0x7f732406a880 2024-04-22T07:22:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f73217fa640 1 -- 172.21.15.113:0/2019211430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7318007e10 con 0x7f732406a880 2024-04-22T07:22:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419dda0 con 0x7f732406a880 2024-04-22T07:22:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f73217fa640 1 -- 172.21.15.113:0/2019211430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73180188c0 con 0x7f732406a880 2024-04-22T07:22:57.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.392+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a5590 con 0x7f732406a880 2024-04-22T07:22:57.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.394+0000 7f73217fa640 1 -- 172.21.15.113:0/2019211430 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7318018a60 con 0x7f732406a880 2024-04-22T07:22:57.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 Processor -- start 2024-04-22T07:22:57.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 -- start start 2024-04-22T07:22:57.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f73217fa640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f730c0a3d20 0x7f730c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 0x556ce58928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce5892e80 0x556ce589b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce589de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556ce57f71f0 con 0x556ce58924d0 2024-04-22T07:22:57.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ce57944b0 con 0x556ce5892e80 2024-04-22T07:22:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f732d041740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556ce57f31b0 con 0x556ce589ba00 2024-04-22T07:22:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f7323fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 0x556ce58928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f73289eb640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f730c0a3d20 0x7f730c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f7323fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 0x556ce58928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52600/0 (socket says 172.21.15.113:52600) 2024-04-22T07:22:57.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f7323fff640 1 -- 172.21.15.113:0/195440128 learned_addr learned my addr 172.21.15.113:0/195440128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f73217fa640 1 -- 172.21.15.113:0/2019211430 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f73180d0ba0 con 0x7f732406a880 2024-04-22T07:22:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f73237fe640 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce5892e80 0x556ce589b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f7323fff640 1 -- 172.21.15.113:0/195440128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 msgr2=0x556ce589de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f7323fff640 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce589de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.395+0000 7f7323fff640 1 -- 172.21.15.113:0/195440128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce5892e80 msgr2=0x556ce589b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f7323fff640 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce5892e80 0x556ce589b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f7323fff640 1 -- 172.21.15.113:0/195440128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce57861b0 con 0x556ce58924d0 2024-04-22T07:22:57.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f73237fe640 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce5892e80 0x556ce589b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f73289eb640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f730c0a3d20 0x7f730c0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f731c032540 tx=0x7f731c009800 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f7323fff640 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 0x556ce58928b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f731809c6a0 tx=0x7f731800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811edc899dc86c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f730a7fc640 1 -- 172.21.15.113:0/195440128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73180b6120 con 0x556ce58924d0 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f730a7fc640 1 -- 172.21.15.113:0/195440128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f731809d0a0 con 0x556ce58924d0 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f730a7fc640 1 -- 172.21.15.113:0/195440128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73180ce3d0 con 0x556ce58924d0 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f732d041740 1 -- 172.21.15.113:0/195440128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 msgr2=0x556ce58928b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.396+0000 7f732d041740 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 0x556ce58928b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f731809c6a0 tx=0x7f731800cf90 comp rx=0 tx=0).stop 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- 172.21.15.113:0/195440128 shutdown_connections 2024-04-22T07:22:57.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce589de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce5892e80 0x556ce589b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 --2- 172.21.15.113:0/195440128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce58924d0 0x556ce58928b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- 172.21.15.113:0/195440128 >> 172.21.15.113:0/195440128 conn(0x556ce588cba0 msgr2=0x556ce588cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- 172.21.15.113:0/195440128 shutdown_connections 2024-04-22T07:22:57.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- 172.21.15.113:0/195440128 wait complete. 2024-04-22T07:22:57.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- start start 2024-04-22T07:22:57.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 Processor -- start 2024-04-22T07:22:57.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce589b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce5892e80 0x556ce58986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 0x556ce5898cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556ce589e910 con 0x556ce589ba00 2024-04-22T07:22:57.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ce57b0500 con 0x556ce58924d0 2024-04-22T07:22:57.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556ce589ea90 con 0x556ce5892e80 2024-04-22T07:22:57.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f732d041740 10 client.0 _ll_register_callbacks cb 0x556ce577eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:57.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.397+0000 7f73289eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 0x556ce5898cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 0x556ce5898cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52602/0 (socket says 172.21.15.113:52602) 2024-04-22T07:22:57.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 -- 172.21.15.113:0/2430249729 learned_addr learned my addr 172.21.15.113:0/2430249729 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73237fe640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce5892e80 0x556ce58986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f7323fff640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce589b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 -- 172.21.15.113:0/2430249729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce5892e80 msgr2=0x556ce58986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce5892e80 0x556ce58986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 -- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 msgr2=0x556ce589b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce589b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 -- 172.21.15.113:0/2430249729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce55093f0 con 0x556ce589ba00 2024-04-22T07:22:57.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73237fe640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce5892e80 0x556ce58986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:57.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f7323fff640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce589b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:57.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73289eb640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 0x556ce5898cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f731c005c90 tx=0x7f731c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06d2ab9dc6e49ed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73037fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:57.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73037fe640 1 -- 172.21.15.113:0/2430249729 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c050060 con 0x556ce589ba00 2024-04-22T07:22:57.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73037fe640 1 -- 172.21.15.113:0/2430249729 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f731c004040 con 0x556ce589ba00 2024-04-22T07:22:57.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731809d0a0 con 0x556ce589ba00 2024-04-22T07:22:57.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73037fe640 1 -- 172.21.15.113:0/2430249729 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f731c057050 con 0x556ce589ba00 2024-04-22T07:22:57.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f732d041740 10 client.17427 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:57.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556ce55d37d0 con 0x556ce589ba00 2024-04-22T07:22:57.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.398+0000 7f73037fe640 1 -- 172.21.15.113:0/2430249729 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f731809d0a0 con 0x556ce589ba00 2024-04-22T07:22:57.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f73037fe640 1 -- 172.21.15.113:0/2430249729 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f731c042920 con 0x556ce589ba00 2024-04-22T07:22:57.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f732d041740 10 client.17427 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:57.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f732d041740 10 client.17427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:22:57.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f732d041740 20 client.17427 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:57.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f732d041740 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce58abbd0 0x556ce58ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f732d041740 4 client.17427 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:57.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ce5796930 con 0x556ce58abbd0 2024-04-22T07:22:57.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f7323fff640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce58abbd0 0x556ce58ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f73037fe640 10 client.17427 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:57.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.399+0000 7f7323fff640 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce58abbd0 0x556ce58ae060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19b31d6a95d45c4c server_cookie=4e1012ad4a358553 in_seq=0 out_seq=0 2024-04-22T07:22:57.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f73037fe640 1 -- 172.21.15.113:0/2430249729 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ce5796930 con 0x556ce58abbd0 2024-04-22T07:22:57.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f73037fe640 10 client.17427 handle_command_reply: tid=0 2024-04-22T07:22:57.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 client.17427 shutdown 2024-04-22T07:22:57.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 msgr2=0x556ce5898cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 0x556ce5898cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f731c005c90 tx=0x7f731c0095c0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 shutdown_connections 2024-04-22T07:22:57.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce58abbd0 0x556ce58ae060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce589ba00 0x556ce5898cd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce5892e80 0x556ce58986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f73037fe640 0 client.17427 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:57.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 --2- 172.21.15.113:0/2430249729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce589b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 >> 172.21.15.113:0/2430249729 conn(0x556ce588cba0 msgr2=0x556ce588e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 shutdown_connections 2024-04-22T07:22:57.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.400+0000 7f732d041740 1 -- 172.21.15.113:0/2430249729 wait complete. 2024-04-22T07:22:57.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.401+0000 7f732d041740 20 client.17427 trim_cache size 0 max 16384 2024-04-22T07:22:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.413+0000 7f732d041740 1 Processor -- start 2024-04-22T07:22:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.413+0000 7f732d041740 1 -- start start 2024-04-22T07:22:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f732d041740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce58924d0 0x556ce588be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f732d041740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce58a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f732d041740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce589ba00 0x556ce589adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f732d041740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b07c0 con 0x556ce5892e80 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f732d041740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ae090 con 0x556ce589ba00 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f732d041740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73241abf00 con 0x556ce58924d0 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce58a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce58a77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52612/0 (socket says 172.21.15.113:52612) 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 -- 172.21.15.113:0/1032001827 learned_addr learned my addr 172.21.15.113:0/1032001827 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 -- 172.21.15.113:0/1032001827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce58924d0 msgr2=0x556ce588be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73289eb640 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce589ba00 0x556ce589adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce58924d0 0x556ce588be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 -- 172.21.15.113:0/1032001827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce589ba00 msgr2=0x556ce589adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce589ba00 0x556ce589adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 -- 172.21.15.113:0/1032001827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce5891560 con 0x556ce5892e80 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73289eb640 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce589ba00 0x556ce589adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f73237fe640 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce58a77d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7314005fd0 tx=0x7f731400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97dbf44773d295a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f7309ffb640 1 -- 172.21.15.113:0/1032001827 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314012d20 con 0x556ce5892e80 2024-04-22T07:22:57.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f7309ffb640 1 -- 172.21.15.113:0/1032001827 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73140035f0 con 0x556ce5892e80 2024-04-22T07:22:57.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.414+0000 7f7309ffb640 1 -- 172.21.15.113:0/1032001827 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140106b0 con 0x556ce5892e80 2024-04-22T07:22:57.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 -- 172.21.15.113:0/1032001827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 msgr2=0x556ce58a77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce58a77d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7314005fd0 tx=0x7f731400bf20 comp rx=0 tx=0).stop 2024-04-22T07:22:57.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 -- 172.21.15.113:0/1032001827 shutdown_connections 2024-04-22T07:22:57.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce589ba00 0x556ce589adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce58a77d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 --2- 172.21.15.113:0/1032001827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce58924d0 0x556ce588be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 -- 172.21.15.113:0/1032001827 >> 172.21.15.113:0/1032001827 conn(0x556ce588cba0 msgr2=0x556ce5897710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 -- 172.21.15.113:0/1032001827 shutdown_connections 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 -- 172.21.15.113:0/1032001827 wait complete. 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 -- start start 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.415+0000 7f732d041740 1 Processor -- start 2024-04-22T07:22:57.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce58a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce5895140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce5895680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ac550 con 0x556ce5892e80 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ad9c0 con 0x556ce58924d0 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f73241acc20 con 0x556ce589ba00 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 10 client.0 _ll_register_callbacks cb 0x7f731c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce5895140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce5895140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52620/0 (socket says 172.21.15.113:52620) 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 -- 172.21.15.113:0/1566335876 learned_addr learned my addr 172.21.15.113:0/1566335876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73289eb640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce5895680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f7323fff640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce58a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 -- 172.21.15.113:0/1566335876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 msgr2=0x556ce5895680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce5895680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 -- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 msgr2=0x556ce58a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce58a40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 -- 172.21.15.113:0/1566335876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce5895eb0 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73289eb640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce5895680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73237fe640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce5895140 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7314003fd0 tx=0x7f73140040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897f53f8e6c9098f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73037fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73037fe640 1 -- 172.21.15.113:0/1566335876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314028020 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73037fe640 1 -- 172.21.15.113:0/1566335876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f7314005040 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73140035f0 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f73037fe640 1 -- 172.21.15.113:0/1566335876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7314033050 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 10 client.17433 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.416+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556ce5912170 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f73037fe640 1 -- 172.21.15.113:0/1566335876 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73140035f0 con 0x556ce5892e80 2024-04-22T07:22:57.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f73037fe640 1 -- 172.21.15.113:0/1566335876 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f7314019070 con 0x556ce5892e80 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f732d041740 10 client.17433 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f732d041740 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f732d041740 20 client.17433 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f732d041740 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce59128c0 0x556ce5914ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f732d041740 4 client.17433 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ce5804c10 con 0x556ce59128c0 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f7323fff640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce59128c0 0x556ce5914ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f73037fe640 10 client.17433 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f7323fff640 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce59128c0 0x556ce5914ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b800a88f8688b0e server_cookie=1e9e532b5e53ae22 in_seq=0 out_seq=0 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f73037fe640 1 -- 172.21.15.113:0/1566335876 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ce5804c10 con 0x556ce59128c0 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.417+0000 7f73037fe640 10 client.17433 handle_command_reply: tid=0 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 client.17433 shutdown 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 msgr2=0x556ce5895140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce5895140 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7314003fd0 tx=0x7f73140040a0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 shutdown_connections 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x556ce59128c0 0x556ce5914ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556ce589ba00 0x556ce5895680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f73037fe640 0 client.17433 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556ce5892e80 0x556ce5895140 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 --2- 172.21.15.113:0/1566335876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ce58924d0 0x556ce58a40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 >> 172.21.15.113:0/1566335876 conn(0x556ce588cba0 msgr2=0x556ce588df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 shutdown_connections 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 1 -- 172.21.15.113:0/1566335876 wait complete. 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732d041740 20 client.17433 trim_cache size 0 max 16384 2024-04-22T07:22:57.441 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f730c0a3d20 msgr2=0x7f730c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f730c0a3d20 0x7f730c0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f731c032540 tx=0x7f731c009800 comp rx=0 tx=0).stop 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 msgr2=0x7f7324198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 0x7f7324198750 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f731800c920 tx=0x7f731800cde0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 shutdown_connections 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f730c0a3d20 0x7f730c0a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f732419cfc0 0x7f732419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7324107270 0x7f7324198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 --2- 172.21.15.113:0/2019211430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f732406a880 0x7f7324198750 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:57.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 >> 172.21.15.113:0/2019211430 conn(0x7f7324077eb0 msgr2=0x7f73240fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:57.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 shutdown_connections 2024-04-22T07:22:57.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:57.418+0000 7f732a44e640 1 -- 172.21.15.113:0/2019211430 wait complete. 2024-04-22T07:22:58.444 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:22:58.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:22:58.445 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:22:58.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 -- 172.21.15.113:0/3312499953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5294107280 msgr2=0x7f529410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 --2- 172.21.15.113:0/3312499953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5294107280 0x7f529410ee30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f528c009ca0 tx=0x7f528c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 -- 172.21.15.113:0/3312499953 shutdown_connections 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 --2- 172.21.15.113:0/3312499953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5294107280 0x7f529410ee30 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 --2- 172.21.15.113:0/3312499953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f529406b170 0x7f5294106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 --2- 172.21.15.113:0/3312499953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f529406a7c0 0x7f529406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 -- 172.21.15.113:0/3312499953 >> 172.21.15.113:0/3312499953 conn(0x7f5294076c50 msgr2=0x7f5294077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 -- 172.21.15.113:0/3312499953 shutdown_connections 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.562+0000 7f52995b1640 1 -- 172.21.15.113:0/3312499953 wait complete. 2024-04-22T07:22:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 Processor -- start 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 -- start start 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f529406a7c0 0x7f5294198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5294107280 0x7f5294198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 0x7f529419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5294112320 con 0x7f529419d000 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52941121a0 con 0x7f529406a7c0 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52995b1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52941124a0 con 0x7f5294107280 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 0x7f529419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 0x7f529419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52636/0 (socket says 172.21.15.113:52636) 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 -- 172.21.15.113:0/2556221859 learned_addr learned my addr 172.21.15.113:0/2556221859 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f529406a7c0 0x7f5294198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 -- 172.21.15.113:0/2556221859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5294107280 msgr2=0x7f5294198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f528bfff640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5294107280 0x7f5294198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5294107280 0x7f5294198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 -- 172.21.15.113:0/2556221859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f529406a7c0 msgr2=0x7f5294198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f529406a7c0 0x7f5294198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.563+0000 7f52937fe640 1 -- 172.21.15.113:0/2556221859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529419daf0 con 0x7f529419d000 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.564+0000 7f52937fe640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 0x7f529419d4b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f528c02c700 tx=0x7f528c035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.564+0000 7f5290ff9640 1 -- 172.21.15.113:0/2556221859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c039040 con 0x7f529419d000 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.564+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529419dd80 con 0x7f529419d000 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.564+0000 7f5290ff9640 1 -- 172.21.15.113:0/2556221859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f528c044070 con 0x7f529419d000 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.564+0000 7f5290ff9640 1 -- 172.21.15.113:0/2556221859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0043e0 con 0x7f529419d000 2024-04-22T07:22:58.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.564+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941a55a0 con 0x7f529419d000 2024-04-22T07:22:58.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.566+0000 7f5290ff9640 1 -- 172.21.15.113:0/2556221859 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f528c004580 con 0x7f529419d000 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.566+0000 7f529c19c740 1 Processor -- start 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 -- start start 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8778db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e87796c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e877976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563e876f1f00 con 0x563e8778e0f0 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563e8768f590 con 0x563e8778d740 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f529c19c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e876edec0 con 0x563e87797250 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5290ff9640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f52800a3da0 0x7f52800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e87796c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e87796c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52638/0 (socket says 172.21.15.113:52638) 2024-04-22T07:22:58.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 -- 172.21.15.113:0/2892549813 learned_addr learned my addr 172.21.15.113:0/2892549813 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f528bfff640 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8778db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f52937fe640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f52800a3da0 0x7f52800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f528bfff640 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e877976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5290ff9640 1 -- 172.21.15.113:0/2556221859 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f528c0f2f30 con 0x7f529419d000 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 -- 172.21.15.113:0/2892549813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 msgr2=0x563e877976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e877976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 -- 172.21.15.113:0/2892549813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 msgr2=0x563e8778db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.567+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8778db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.568+0000 7f5292ffd640 1 -- 172.21.15.113:0/2892549813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e874cc7e0 con 0x563e8778e0f0 2024-04-22T07:22:58.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.568+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e87796c30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f528400bd10 tx=0x7f52840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ad7cd499082c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.568+0000 7f52937fe640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f52800a3da0 0x7f52800a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f528c0beae0 tx=0x7f528c08d790 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.568+0000 7f52897fa640 1 -- 172.21.15.113:0/2892549813 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284004030 con 0x563e8778e0f0 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.568+0000 7f52897fa640 1 -- 172.21.15.113:0/2892549813 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f52840041d0 con 0x563e8778e0f0 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.568+0000 7f52897fa640 1 -- 172.21.15.113:0/2892549813 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284011730 con 0x563e8778e0f0 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- 172.21.15.113:0/2892549813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 msgr2=0x563e87796c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e87796c30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f528400bd10 tx=0x7f52840107a0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- 172.21.15.113:0/2892549813 shutdown_connections 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e877976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e87796c30 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- 172.21.15.113:0/2892549813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8778db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- 172.21.15.113:0/2892549813 >> 172.21.15.113:0/2892549813 conn(0x563e87787fb0 msgr2=0x563e87788390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- 172.21.15.113:0/2892549813 shutdown_connections 2024-04-22T07:22:58.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- 172.21.15.113:0/2892549813 wait complete. 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- start start 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 Processor -- start 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e8779b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e8779b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563e8779e1d0 con 0x563e87797250 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563e876ab580 con 0x563e8778d740 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e8779e350 con 0x563e8778e0f0 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f529c19c740 10 client.0 _ll_register_callbacks cb 0x563e876b1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.569+0000 7f528bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e8779b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e8779b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52644/0 (socket says 172.21.15.113:52644) 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 -- 172.21.15.113:0/3200513776 learned_addr learned my addr 172.21.15.113:0/3200513776 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f5292ffd640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e8779b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 -- 172.21.15.113:0/3200513776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 msgr2=0x563e8779b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e8779b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 -- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 msgr2=0x563e8779acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 -- 172.21.15.113:0/3200513776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8752cf70 con 0x563e87797250 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f5292ffd640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e8779b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f528bfff640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e8779b770 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f529406a360 tx=0x7f527c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7d83acaaab0ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f527a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f527a7fc640 1 -- 172.21.15.113:0/3200513776 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c00cd40 con 0x563e87797250 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f527a7fc640 1 -- 172.21.15.113:0/3200513776 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f527c0213f0 con 0x563e87797250 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52840041d0 con 0x563e87797250 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f527a7fc640 1 -- 172.21.15.113:0/3200513776 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f527c0215f0 con 0x563e87797250 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f529c19c740 10 client.17445 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563e873c4170 con 0x563e87797250 2024-04-22T07:22:58.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.570+0000 7f527a7fc640 1 -- 172.21.15.113:0/3200513776 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f52840041d0 con 0x563e87797250 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f527a7fc640 1 -- 172.21.15.113:0/3200513776 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f527c00f040 con 0x563e87797250 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f529c19c740 10 client.17445 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f529c19c740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f529c19c740 20 client.17445 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f529c19c740 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e877ab7b0 0x563e877adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f529c19c740 4 client.17445 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e876ff920 con 0x563e877ab7b0 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.571+0000 7f5292ffd640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e877ab7b0 0x563e877adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.572+0000 7f527a7fc640 10 client.17445 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.572+0000 7f5292ffd640 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e877ab7b0 0x563e877adc00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3a526b4b0a3bcd server_cookie=92ef0c014c409f32 in_seq=0 out_seq=0 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f527a7fc640 1 -- 172.21.15.113:0/3200513776 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e876ff920 con 0x563e877ab7b0 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f527a7fc640 10 client.17445 handle_command_reply: tid=0 2024-04-22T07:22:58.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 client.17445 shutdown 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 msgr2=0x563e8779b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e8779b770 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f529406a360 tx=0x7f527c00ea70 comp rx=0 tx=0).stop 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 shutdown_connections 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e877ab7b0 0x563e877adc00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e8779b770 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f527a7fc640 0 client.17445 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e8779b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 --2- 172.21.15.113:0/3200513776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 >> 172.21.15.113:0/3200513776 conn(0x563e87787fb0 msgr2=0x563e87789d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 shutdown_connections 2024-04-22T07:22:58.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 1 -- 172.21.15.113:0/3200513776 wait complete. 2024-04-22T07:22:58.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.573+0000 7f529c19c740 20 client.17445 trim_cache size 0 max 16384 2024-04-22T07:22:58.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 Processor -- start 2024-04-22T07:22:58.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 -- start start 2024-04-22T07:22:58.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e877a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e87799e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52941ae630 con 0x563e8778e0f0 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52941abf00 con 0x563e8778d740 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f529c19c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52941a9d70 con 0x563e87797250 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e877a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e877a7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52646/0 (socket says 172.21.15.113:52646) 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 -- 172.21.15.113:0/2846807172 learned_addr learned my addr 172.21.15.113:0/2846807172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f528bfff640 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 -- 172.21.15.113:0/2846807172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 msgr2=0x563e87799e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e87799e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 -- 172.21.15.113:0/2846807172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 msgr2=0x563e8779d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.591+0000 7f5292ffd640 1 -- 172.21.15.113:0/2846807172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e87799240 con 0x563e8778e0f0 2024-04-22T07:22:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e877a7100 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x563e8778cab0 tx=0x7f5284003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbc64a575f79088 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f5288ff9640 1 -- 172.21.15.113:0/2846807172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528402e070 con 0x563e8778e0f0 2024-04-22T07:22:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f5288ff9640 1 -- 172.21.15.113:0/2846807172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f52840035f0 con 0x563e8778e0f0 2024-04-22T07:22:58.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f5288ff9640 1 -- 172.21.15.113:0/2846807172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840292b0 con 0x563e8778e0f0 2024-04-22T07:22:58.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 -- 172.21.15.113:0/2846807172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 msgr2=0x563e877a7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e877a7100 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x563e8778cab0 tx=0x7f5284003e20 comp rx=0 tx=0).stop 2024-04-22T07:22:58.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 -- 172.21.15.113:0/2846807172 shutdown_connections 2024-04-22T07:22:58.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e87797250 0x563e87799e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e8778e0f0 0x563e877a7100 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 --2- 172.21.15.113:0/2846807172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e8779d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.592+0000 7f529c19c740 1 -- 172.21.15.113:0/2846807172 >> 172.21.15.113:0/2846807172 conn(0x563e87787fb0 msgr2=0x563e8779acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 -- 172.21.15.113:0/2846807172 shutdown_connections 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 -- 172.21.15.113:0/2846807172 wait complete. 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 -- start start 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 Processor -- start 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e877a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e877a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e877a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52941aa3c0 con 0x563e87797250 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52941ab830 con 0x563e8778d740 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52941aaa90 con 0x563e8778e0f0 2024-04-22T07:22:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f529c19c740 10 client.0 _ll_register_callbacks cb 0x7f527c00edd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e877a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e877a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52652/0 (socket says 172.21.15.113:52652) 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 -- 172.21.15.113:0/2489739735 learned_addr learned my addr 172.21.15.113:0/2489739735 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e877a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 -- 172.21.15.113:0/2489739735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 msgr2=0x563e877a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e877a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 -- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 msgr2=0x563e877a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.593+0000 7f528bfff640 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e877a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f528bfff640 1 -- 172.21.15.113:0/2489739735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e877a0c20 con 0x563e87797250 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f528bfff640 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e877a05e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f527c00ef20 tx=0x7f527c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d46427b2a9f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f527a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f527a7fc640 1 -- 172.21.15.113:0/2489739735 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c00cd40 con 0x563e87797250 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f527a7fc640 1 -- 172.21.15.113:0/2489739735 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f527c0120b0 con 0x563e87797250 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52840035f0 con 0x563e87797250 2024-04-22T07:22:58.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f527a7fc640 1 -- 172.21.15.113:0/2489739735 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f527c00e550 con 0x563e87797250 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 10 client.17451 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563e878126b0 con 0x563e87797250 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f527a7fc640 1 -- 172.21.15.113:0/2489739735 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f52840035f0 con 0x563e87797250 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f527a7fc640 1 -- 172.21.15.113:0/2489739735 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f527c0230d0 con 0x563e87797250 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 10 client.17451 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 20 client.17451 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e87812e00 0x563e87815220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 4 client.17451 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.594+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563e876f9790 con 0x563e87812e00 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.595+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e87812e00 0x563e87815220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.595+0000 7f5292ffd640 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e87812e00 0x563e87815220 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61aa7bac78927024 server_cookie=cf4dad68ac33fed6 in_seq=0 out_seq=0 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.595+0000 7f527a7fc640 10 client.17451 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f527a7fc640 1 -- 172.21.15.113:0/2489739735 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563e876f9790 con 0x563e87812e00 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f527a7fc640 10 client.17451 handle_command_reply: tid=0 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 client.17451 shutdown 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 msgr2=0x563e877a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e877a05e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f527c00ef20 tx=0x7f527c005f30 comp rx=0 tx=0).stop 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 shutdown_connections 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e87812e00 0x563e87815220 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e87797250 0x563e877a05e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e8778e0f0 0x563e877a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f527a7fc640 0 client.17451 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 --2- 172.21.15.113:0/2489739735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e8778d740 0x563e877a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 >> 172.21.15.113:0/2489739735 conn(0x563e87787fb0 msgr2=0x563e8779acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 shutdown_connections 2024-04-22T07:22:58.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.596+0000 7f529c19c740 1 -- 172.21.15.113:0/2489739735 wait complete. 2024-04-22T07:22:58.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f529c19c740 20 client.17451 trim_cache size 0 max 16384 2024-04-22T07:22:58.600 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:22:58.600 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:22:58.600 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "d30275f0-4a20-41d4-bef6-95564e4e0892", 2024-04-22T07:22:58.600 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:22:58.600 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f52800a3da0 msgr2=0x7f52800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f52800a3da0 0x7f52800a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f528c0beae0 tx=0x7f528c08d790 comp rx=0 tx=0).stop 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 msgr2=0x7f529419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 0x7f529419d4b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f528c02c700 tx=0x7f528c035d80 comp rx=0 tx=0).stop 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 shutdown_connections 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f52800a3da0 0x7f52800a6260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f529419d000 0x7f529419d4b0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5294107280 0x7f5294198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 --2- 172.21.15.113:0/2556221859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f529406a7c0 0x7f5294198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 >> 172.21.15.113:0/2556221859 conn(0x7f5294076c50 msgr2=0x7f52940ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 shutdown_connections 2024-04-22T07:22:58.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.597+0000 7f52995b1640 1 -- 172.21.15.113:0/2556221859 wait complete. 2024-04-22T07:22:58.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:22:58.606 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 -- 172.21.15.113:0/3063763184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b8106150 msgr2=0x7f12b810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 --2- 172.21.15.113:0/3063763184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b8106150 0x7f12b810dd20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009ca0 tx=0x7f12b002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 -- 172.21.15.113:0/3063763184 shutdown_connections 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 --2- 172.21.15.113:0/3063763184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b8106150 0x7f12b810dd20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 --2- 172.21.15.113:0/3063763184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101a30 0x7f12b8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 --2- 172.21.15.113:0/3063763184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b8101080 0x7f12b8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 -- 172.21.15.113:0/3063763184 >> 172.21.15.113:0/3063763184 conn(0x7f12b8077fd0 msgr2=0x7f12b80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 -- 172.21.15.113:0/3063763184 shutdown_connections 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 -- 172.21.15.113:0/3063763184 wait complete. 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 Processor -- start 2024-04-22T07:22:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.708+0000 7f12be470640 1 -- start start 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101080 0x7f12b8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b8106150 0x7f12b8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 0x7f12b819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81122f0 con 0x7f12b819cfc0 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8112170 con 0x7f12b8101080 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8112470 con 0x7f12b8106150 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 0x7f12b819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 0x7f12b819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52670/0 (socket says 172.21.15.113:52670) 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 -- 172.21.15.113:0/3764924491 learned_addr learned my addr 172.21.15.113:0/3764924491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12b77fe640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b8106150 0x7f12b8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12b7fff640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101080 0x7f12b8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 -- 172.21.15.113:0/3764924491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b8106150 msgr2=0x7f12b8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b8106150 0x7f12b8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 -- 172.21.15.113:0/3764924491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101080 msgr2=0x7f12b8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101080 0x7f12b8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 -- 172.21.15.113:0/3764924491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b819dab0 con 0x7f12b819cfc0 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12b7fff640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101080 0x7f12b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12bca0d640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 0x7f12b819d470 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f12b002c700 tx=0x7f12b0035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12b57fa640 1 -- 172.21.15.113:0/3764924491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0039040 con 0x7f12b819cfc0 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b819dd40 con 0x7f12b819cfc0 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12b57fa640 1 -- 172.21.15.113:0/3764924491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f12b0044070 con 0x7f12b819cfc0 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12b57fa640 1 -- 172.21.15.113:0/3764924491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0004330 con 0x7f12b819cfc0 2024-04-22T07:22:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.709+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a5510 con 0x7f12b819cfc0 2024-04-22T07:22:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.711+0000 7f12b57fa640 1 -- 172.21.15.113:0/3764924491 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f12b00044d0 con 0x7f12b819cfc0 2024-04-22T07:22:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 Processor -- start 2024-04-22T07:22:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 -- start start 2024-04-22T07:22:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f944610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f951950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f954280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ff7f8a91e0 con 0x55ff7f944230 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ff7f846390 con 0x55ff7f944b50 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12c105c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ff7f8a51a0 con 0x55ff7f951e90 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b57fa640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f12a40a3d20 0x7f12a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f944610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f944610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52676/0 (socket says 172.21.15.113:52676) 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 -- 172.21.15.113:0/2367672519 learned_addr learned my addr 172.21.15.113:0/2367672519 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b7fff640 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f951950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12bca0d640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f12a40a3d20 0x7f12a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b57fa640 1 -- 172.21.15.113:0/3764924491 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f12b00f2ed0 con 0x7f12b819cfc0 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 -- 172.21.15.113:0/2367672519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 msgr2=0x55ff7f954280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f954280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 -- 172.21.15.113:0/2367672519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 msgr2=0x55ff7f951950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f951950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.712+0000 7f12b77fe640 1 -- 172.21.15.113:0/2367672519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff7f6857d0 con 0x55ff7f944230 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f944610 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f12a800bc80 tx=0x7f12a8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161bd838a3b1c229 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12bca0d640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f12a40a3d20 0x7f12a40a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12b00f0a80 tx=0x7f12b0091830 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12ae7fc640 1 -- 172.21.15.113:0/2367672519 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a801a400 con 0x55ff7f944230 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12ae7fc640 1 -- 172.21.15.113:0/2367672519 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f12a8005bf0 con 0x55ff7f944230 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12ae7fc640 1 -- 172.21.15.113:0/2367672519 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f7c0 con 0x55ff7f944230 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 -- 172.21.15.113:0/2367672519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 msgr2=0x55ff7f944610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f944610 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f12a800bc80 tx=0x7f12a8011750 comp rx=0 tx=0).stop 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 -- 172.21.15.113:0/2367672519 shutdown_connections 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f954280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f951950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 --2- 172.21.15.113:0/2367672519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f944610 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.713+0000 7f12c105c740 1 -- 172.21.15.113:0/2367672519 >> 172.21.15.113:0/2367672519 conn(0x55ff7f93ecb0 msgr2=0x55ff7f93f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 -- 172.21.15.113:0/2367672519 shutdown_connections 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 -- 172.21.15.113:0/2367672519 wait complete. 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 -- start start 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 Processor -- start 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f94ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f944b50 0x55ff7f94f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 0x55ff7f94f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ff7f954cd0 con 0x55ff7f951e90 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ff7f8622f0 con 0x55ff7f944230 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ff7f954e50 con 0x55ff7f944b50 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12c105c740 10 client.0 _ll_register_callbacks cb 0x55ff7f86f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 0x55ff7f94f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 0x55ff7f94f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52688/0 (socket says 172.21.15.113:52688) 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12b77fe640 1 -- 172.21.15.113:0/2656955032 learned_addr learned my addr 172.21.15.113:0/2656955032 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.714+0000 7f12b7fff640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f944b50 0x55ff7f94f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f94ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 -- 172.21.15.113:0/2656955032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f944b50 msgr2=0x55ff7f94f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f944b50 0x55ff7f94f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 -- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 msgr2=0x55ff7f94ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f94ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 -- 172.21.15.113:0/2656955032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff7f5bb3f0 con 0x55ff7f951e90 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 0x55ff7f94f8b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f12a80058c0 tx=0x7f12a800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca39ce5889504f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12977fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12977fe640 1 -- 172.21.15.113:0/2656955032 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8005050 con 0x55ff7f951e90 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12977fe640 1 -- 172.21.15.113:0/2656955032 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f12a80051f0 con 0x55ff7f951e90 2024-04-22T07:22:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff7f8b6c00 con 0x55ff7f951e90 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12977fe640 1 -- 172.21.15.113:0/2656955032 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f12a8031050 con 0x55ff7f951e90 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12c105c740 10 client.17463 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ff7f57b0f0 con 0x55ff7f951e90 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.715+0000 7f12977fe640 1 -- 172.21.15.113:0/2656955032 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x55ff7f8b6c00 con 0x55ff7f951e90 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12977fe640 1 -- 172.21.15.113:0/2656955032 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f12a8029020 con 0x55ff7f951e90 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12c105c740 10 client.17463 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12c105c740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12c105c740 20 client.17463 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12c105c740 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f964260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12c105c740 4 client.17463 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff7f8b0a70 con 0x55ff7f961e70 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12b7fff640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f964260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12977fe640 10 client.17463 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.716+0000 7f12b7fff640 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f964260 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d8f9de249726f5 server_cookie=cf47cedc299fc4b3 in_seq=0 out_seq=0 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12977fe640 1 -- 172.21.15.113:0/2656955032 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff7f8b0a70 con 0x55ff7f961e70 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12977fe640 10 client.17463 handle_command_reply: tid=0 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 client.17463 shutdown 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 msgr2=0x55ff7f94f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 0x55ff7f94f8b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f12a80058c0 tx=0x7f12a800ba60 comp rx=0 tx=0).stop 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 shutdown_connections 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f964260 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f951e90 0x55ff7f94f8b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f944b50 0x55ff7f94f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12977fe640 0 client.17463 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 --2- 172.21.15.113:0/2656955032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f94ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 >> 172.21.15.113:0/2656955032 conn(0x55ff7f93ecb0 msgr2=0x55ff7f940a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 shutdown_connections 2024-04-22T07:22:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.717+0000 7f12c105c740 1 -- 172.21.15.113:0/2656955032 wait complete. 2024-04-22T07:22:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.718+0000 7f12c105c740 20 client.17463 trim_cache size 0 max 16384 2024-04-22T07:22:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.734+0000 7f12c105c740 1 Processor -- start 2024-04-22T07:22:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.734+0000 7f12c105c740 1 -- start start 2024-04-22T07:22:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12c105c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f951560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12c105c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f94dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12c105c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f95dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12c105c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ae5d0 con 0x55ff7f944230 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12c105c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81abea0 con 0x55ff7f944b50 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12c105c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81a9d10 con 0x55ff7f951e90 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f951560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f951560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52696/0 (socket says 172.21.15.113:52696) 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 -- 172.21.15.113:0/2077115431 learned_addr learned my addr 172.21.15.113:0/2077115431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b7fff640 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f94dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 -- 172.21.15.113:0/2077115431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 msgr2=0x55ff7f95dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f95dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 -- 172.21.15.113:0/2077115431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 msgr2=0x55ff7f94dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f94dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 -- 172.21.15.113:0/2077115431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff7f944980 con 0x55ff7f944230 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12b77fe640 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f951560 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f12a800b680 tx=0x7f12a800b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335d518b99b4accf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12adffb640 1 -- 172.21.15.113:0/2077115431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a803a070 con 0x55ff7f944230 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12adffb640 1 -- 172.21.15.113:0/2077115431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f12a8020030 con 0x55ff7f944230 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.735+0000 7f12adffb640 1 -- 172.21.15.113:0/2077115431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80128f0 con 0x55ff7f944230 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 -- 172.21.15.113:0/2077115431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 msgr2=0x55ff7f951560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f951560 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f12a800b680 tx=0x7f12a800b500 comp rx=0 tx=0).stop 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 -- 172.21.15.113:0/2077115431 shutdown_connections 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f95dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944b50 0x55ff7f94dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 --2- 172.21.15.113:0/2077115431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944230 0x55ff7f951560 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 -- 172.21.15.113:0/2077115431 >> 172.21.15.113:0/2077115431 conn(0x55ff7f93ecb0 msgr2=0x55ff7f940990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 -- 172.21.15.113:0/2077115431 shutdown_connections 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 -- 172.21.15.113:0/2077115431 wait complete. 2024-04-22T07:22:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.736+0000 7f12c105c740 1 -- start start 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 Processor -- start 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f95a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 0x55ff7f955a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f956100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81aa360 con 0x55ff7f944b50 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ab7d0 con 0x55ff7f944230 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81aaa30 con 0x55ff7f951e90 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 10 client.0 _ll_register_callbacks cb 0x7f12a000b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 0x55ff7f955a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 0x55ff7f955a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52712/0 (socket says 172.21.15.113:52712) 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 -- 172.21.15.113:0/355133160 learned_addr learned my addr 172.21.15.113:0/355133160 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 -- 172.21.15.113:0/355133160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 msgr2=0x55ff7f956100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f956100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 -- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 msgr2=0x55ff7f95a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b77fe640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f95a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f95a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 -- 172.21.15.113:0/355133160 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff7f9c70d0 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b77fe640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f95a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12b7fff640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 0x55ff7f955a10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f12a0004700 tx=0x7f12a000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6874c55972bf8bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12977fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12977fe640 1 -- 172.21.15.113:0/355133160 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0020070 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12977fe640 1 -- 172.21.15.113:0/355133160 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8943+0+0 (secure 0 0 0) 0x7f12a001c5b0 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.737+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a8020030 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12977fe640 1 -- 172.21.15.113:0/355133160 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f12a002b050 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 10 client.17469 fetch_fsmap learned FSMap version 52 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ff7f9c8310 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12977fe640 1 -- 172.21.15.113:0/355133160 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f12a8020030 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12977fe640 1 -- 172.21.15.113:0/355133160 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f12a00053f0 con 0x55ff7f944b50 2024-04-22T07:22:58.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 10 client.17469 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 20 client.17469 populate_metadata read hostname 'smithi113' 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f9cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 4 client.17469 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff7f82c590 con 0x55ff7f961e70 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.738+0000 7f12b77fe640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f9cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12977fe640 10 client.17469 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12b77fe640 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f9cab00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f0df8406038b73 server_cookie=1253d47431f319fb in_seq=0 out_seq=0 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12977fe640 1 -- 172.21.15.113:0/355133160 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ff7f82c590 con 0x55ff7f961e70 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12977fe640 10 client.17469 handle_command_reply: tid=0 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 client.17469 shutdown 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 msgr2=0x55ff7f955a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 0x55ff7f955a10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f12a0004700 tx=0x7f12a000fb30 comp rx=0 tx=0).stop 2024-04-22T07:22:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 shutdown_connections 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55ff7f961e70 0x55ff7f9cab00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ff7f951e90 0x55ff7f956100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ff7f944b50 0x55ff7f955a10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 --2- 172.21.15.113:0/355133160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ff7f944230 0x55ff7f95a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12977fe640 0 client.17469 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 >> 172.21.15.113:0/355133160 conn(0x55ff7f93ecb0 msgr2=0x55ff7f940100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 shutdown_connections 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 1 -- 172.21.15.113:0/355133160 wait complete. 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12c105c740 20 client.17469 trim_cache size 0 max 16384 2024-04-22T07:22:58.743 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: "d30275f0-4a20-41d4-bef6-95564e4e0892": { 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "d30275f0-4a20-41d4-bef6-95564e4e0892", 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:22:58.744 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f12a40a3d20 msgr2=0x7f12a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12be470640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f12a40a3d20 0x7f12a40a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12b00f0a80 tx=0x7f12b0091830 comp rx=0 tx=0).stop 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 msgr2=0x7f12b819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.739+0000 7f12be470640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 0x7f12b819d470 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f12b002c700 tx=0x7f12b0035d80 comp rx=0 tx=0).stop 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 shutdown_connections 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f12a40a3d20 0x7f12a40a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12b819cfc0 0x7f12b819d470 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12b8106150 0x7f12b8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 --2- 172.21.15.113:0/3764924491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12b8101080 0x7f12b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:22:58.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 >> 172.21.15.113:0/3764924491 conn(0x7f12b8077fd0 msgr2=0x7f12b80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:22:58.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 shutdown_connections 2024-04-22T07:22:58.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:22:58.740+0000 7f12be470640 1 -- 172.21.15.113:0/3764924491 wait complete. 2024-04-22T07:22:58.748 INFO:tasks.cephfs.filesystem:scrub status for tag:d30275f0-4a20-41d4-bef6-95564e4e0892 - {'path': '/', 'tag': 'd30275f0-4a20-41d4-bef6-95564e4e0892', 'options': 'recursive,force'} 2024-04-22T07:22:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:59 smithi133 ceph-mon[31198]: pgmap v255: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:59 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:22:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:59 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:59 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:22:59.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:22:59 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:59.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:59 smithi154 ceph-mon[31862]: pgmap v255: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:59.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:59 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:22:59.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:59 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:59.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:59 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:22:59.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:22:59 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:59 smithi113 ceph-mon[27933]: pgmap v255: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:22:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:59 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:22:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:59 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:22:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:59 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:22:59.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:22:59 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:01 smithi133 ceph-mon[31198]: pgmap v256: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:01.314 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:01 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:23:01.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:01 smithi154 ceph-mon[31862]: pgmap v256: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:01.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:01 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:23:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:01 smithi113 ceph-mon[27933]: pgmap v256: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:01 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:23:02.315 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:02 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:23:02.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:02 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:23:02.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:02 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:23:02.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:02 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:23:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:02 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:23:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:02 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:23:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:02 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:23:02.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:02 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:23:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:02 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:23:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:02 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:23:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:02 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:23:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:02 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:23:03.316 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:03 smithi154 ceph-mon[31862]: pgmap v257: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:03.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:03 smithi113 ceph-mon[27933]: pgmap v257: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:03 smithi133 ceph-mon[31198]: pgmap v257: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:05.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:05 smithi113 ceph-mon[27933]: pgmap v258: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:05 smithi133 ceph-mon[31198]: pgmap v258: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:05.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:05 smithi154 ceph-mon[31862]: pgmap v258: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:07.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:07 smithi113 ceph-mon[27933]: pgmap v259: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:23:07.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:07 smithi133 ceph-mon[31198]: pgmap v259: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:23:07.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:07 smithi154 ceph-mon[31862]: pgmap v259: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:23:09.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:09 smithi113 ceph-mon[27933]: pgmap v260: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-22T07:23:09.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:09 smithi133 ceph-mon[31198]: pgmap v260: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-22T07:23:09.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:09 smithi154 ceph-mon[31862]: pgmap v260: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-22T07:23:11.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:11 smithi133 ceph-mon[31198]: pgmap v261: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-22T07:23:11.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:11 smithi154 ceph-mon[31862]: pgmap v261: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-22T07:23:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:11 smithi113 ceph-mon[27933]: pgmap v261: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-22T07:23:12.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:12 smithi133 ceph-mon[31198]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T07:23:12.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:12 smithi154 ceph-mon[31862]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T07:23:12.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:12 smithi113 ceph-mon[27933]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T07:23:13.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:13 smithi133 ceph-mon[31198]: pgmap v263: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:23:13.467 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:13 smithi154 ceph-mon[31862]: pgmap v263: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:23:13.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:13 smithi113 ceph-mon[27933]: pgmap v263: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:23:15.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:15 smithi133 ceph-mon[31198]: pgmap v264: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:23:15.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:15 smithi154 ceph-mon[31862]: pgmap v264: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:23:15.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:15 smithi113 ceph-mon[27933]: pgmap v264: 97 pgs: 97 active+clean; 273 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:23:17.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:17 smithi133 ceph-mon[31198]: pgmap v265: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:17.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:17 smithi154 ceph-mon[31862]: pgmap v265: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:17.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:17 smithi113 ceph-mon[27933]: pgmap v265: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:19.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:19 smithi133 ceph-mon[31198]: pgmap v266: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:19.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:19 smithi133 ceph-mon[31198]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T07:23:19.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:19 smithi154 ceph-mon[31862]: pgmap v266: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:19.567 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:19 smithi154 ceph-mon[31862]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T07:23:19.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:19 smithi113 ceph-mon[27933]: pgmap v266: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:19.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:19 smithi113 ceph-mon[27933]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T07:23:21.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:21 smithi133 ceph-mon[31198]: pgmap v268: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 121 B/s wr, 4 op/s 2024-04-22T07:23:21.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:21 smithi154 ceph-mon[31862]: pgmap v268: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 121 B/s wr, 4 op/s 2024-04-22T07:23:21.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:21 smithi113 ceph-mon[27933]: pgmap v268: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 121 B/s wr, 4 op/s 2024-04-22T07:23:23.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:23 smithi133 ceph-mon[31198]: pgmap v269: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:23.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:23 smithi154 ceph-mon[31862]: pgmap v269: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:23.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:23 smithi113 ceph-mon[27933]: pgmap v269: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:25.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:25 smithi133 ceph-mon[31198]: pgmap v270: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:25.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:25 smithi154 ceph-mon[31862]: pgmap v270: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:25.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:25 smithi113 ceph-mon[27933]: pgmap v270: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:23:27.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:27 smithi133 ceph-mon[31198]: pgmap v271: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:23:27.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:27 smithi154 ceph-mon[31862]: pgmap v271: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:23:27.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:27 smithi113 ceph-mon[27933]: pgmap v271: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:23:28.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:23:28.749 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:23:28.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 -- 172.21.15.113:0/2879163795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101a30 msgr2=0x7fa7f0105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/2879163795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101a30 0x7fa7f0105a70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009900 tx=0x7fa7d802bc30 comp rx=0 tx=0).stop 2024-04-22T07:23:28.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 -- 172.21.15.113:0/2879163795 shutdown_connections 2024-04-22T07:23:28.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/2879163795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f0106150 0x7fa7f010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/2879163795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101a30 0x7fa7f0105a70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/2879163795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0101080 0x7fa7f0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 -- 172.21.15.113:0/2879163795 >> 172.21.15.113:0/2879163795 conn(0x7fa7f0078000 msgr2=0x7fa7f0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 -- 172.21.15.113:0/2879163795 shutdown_connections 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.864+0000 7fa7f5c74640 1 -- 172.21.15.113:0/2879163795 wait complete. 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 Processor -- start 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 -- start start 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 0x7fa7f019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f0106150 0x7fa7f019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0198640 0x7fa7f0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0111230 con 0x7fa7f0101080 2024-04-22T07:23:28.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01110b0 con 0x7fa7f0198640 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7f5c74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01113b0 con 0x7fa7f0106150 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 0x7fa7f019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 0x7fa7f019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59984/0 (socket says 172.21.15.113:59984) 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/1468905319 learned_addr learned my addr 172.21.15.113:0/1468905319 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7e7fff640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f0106150 0x7fa7f019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7effff640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0198640 0x7fa7f0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/1468905319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f0106150 msgr2=0x7fa7f019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f0106150 0x7fa7f019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/1468905319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0198640 msgr2=0x7fa7f0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0198640 0x7fa7f0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/1468905319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0199240 con 0x7fa7f0101080 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7effff640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0198640 0x7fa7f0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:28.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 0x7fa7f019e580 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c890 tx=0x7fa7e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:28.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.865+0000 7fa7ed7fa640 1 -- 172.21.15.113:0/1468905319 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0004220 con 0x7fa7f0101080 2024-04-22T07:23:28.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.866+0000 7fa7ed7fa640 1 -- 172.21.15.113:0/1468905319 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa7e00043c0 con 0x7fa7f0101080 2024-04-22T07:23:28.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.866+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f01a5320 con 0x7fa7f0101080 2024-04-22T07:23:28.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.866+0000 7fa7ed7fa640 1 -- 172.21.15.113:0/1468905319 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00049c0 con 0x7fa7f0101080 2024-04-22T07:23:28.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.866+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01a56d0 con 0x7fa7f0101080 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7ed7fa640 1 -- 172.21.15.113:0/1468905319 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa7e0005140 con 0x7fa7f0101080 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 Processor -- start 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 -- start start 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7ed7fa640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa7dc0a3d20 0x7fa7dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbf4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc01950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbc04280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbb591e0 con 0x55d4fbc01e90 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbaf6390 con 0x55d4fbbf4230 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.868+0000 7fa7f886e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbb551a0 con 0x55d4fbbf4b50 2024-04-22T07:23:28.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa7dc0a3d20 0x7fa7dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7ed7fa640 1 -- 172.21.15.113:0/1468905319 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa7e00d06f0 con 0x7fa7f0101080 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbc04280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbc04280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59998/0 (socket says 172.21.15.113:59998) 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 -- 172.21.15.113:0/1961533845 learned_addr learned my addr 172.21.15.113:0/1961533845 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7e7fff640 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc01950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbf4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 -- 172.21.15.113:0/1961533845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 msgr2=0x55d4fbc01950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc01950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 -- 172.21.15.113:0/1961533845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 msgr2=0x55d4fbbf4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbf4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 -- 172.21.15.113:0/1961533845 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4fb9357d0 con 0x55d4fbc01e90 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa7dc0a3d20 0x7fa7dc0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ec60 tx=0x7fa7e8009e30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7effff640 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbc04280 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800cfd0 tx=0x7fa7e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf38d929fd99541 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7e5ffb640 1 -- 172.21.15.113:0/1961533845 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8020020 con 0x55d4fbc01e90 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7e5ffb640 1 -- 172.21.15.113:0/1961533845 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa7e801c430 con 0x55d4fbc01e90 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.869+0000 7fa7e5ffb640 1 -- 172.21.15.113:0/1961533845 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e801a5f0 con 0x55d4fbc01e90 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 -- 172.21.15.113:0/1961533845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 msgr2=0x55d4fbc04280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbc04280 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800cfd0 tx=0x7fa7e8018000 comp rx=0 tx=0).stop 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 -- 172.21.15.113:0/1961533845 shutdown_connections 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbc04280 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc01950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1961533845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbf4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 -- 172.21.15.113:0/1961533845 >> 172.21.15.113:0/1961533845 conn(0x55d4fbbeecb0 msgr2=0x55d4fbbef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 -- 172.21.15.113:0/1961533845 shutdown_connections 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 -- 172.21.15.113:0/1961533845 wait complete. 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.870+0000 7fa7f886e740 1 -- start start 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 Processor -- start 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbfeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbbff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbbff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbc04c70 con 0x55d4fbc01e90 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbb122f0 con 0x55d4fbbf4230 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d4fbc04df0 con 0x55d4fbbf4b50 2024-04-22T07:23:28.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7f886e740 10 client.0 _ll_register_callbacks cb 0x55d4fbb1f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbbff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbbff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60002/0 (socket says 172.21.15.113:60002) 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 -- 172.21.15.113:0/1414010860 learned_addr learned my addr 172.21.15.113:0/1414010860 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7e7fff640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbbff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbfeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 -- 172.21.15.113:0/1414010860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 msgr2=0x55d4fbbff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbbff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 -- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 msgr2=0x55d4fbbfeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbfeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7effff640 1 -- 172.21.15.113:0/1414010860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4fb86b3f0 con 0x55d4fbc01e90 2024-04-22T07:23:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.871+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7effff640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbbff930 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa7e801e040 tx=0x7fa7e801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4222e5717980b1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7ceffd640 1 -- 172.21.15.113:0/1414010860 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e802b020 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e801c430 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7ceffd640 1 -- 172.21.15.113:0/1414010860 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa7e8008d30 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7ceffd640 1 -- 172.21.15.113:0/1414010860 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa7e8036050 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 10 client.17481 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d4fb82b0f0 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7ceffd640 1 -- 172.21.15.113:0/1414010860 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa7e801c430 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7ceffd640 1 -- 172.21.15.113:0/1414010860 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa7e801b070 con 0x55d4fbc01e90 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 10 client.17481 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 20 client.17481 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc11ee0 0x55d4fbc142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 4 client.17481 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.872+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4fbb66c00 con 0x55d4fbc11ee0 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.873+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc11ee0 0x55d4fbc142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.873+0000 7fa7ceffd640 10 client.17481 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.873+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc11ee0 0x55d4fbc142d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcda45aa88c1d5d0 server_cookie=b8ee08abc341509a in_seq=0 out_seq=0 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7ceffd640 1 -- 172.21.15.113:0/1414010860 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4fbb66c00 con 0x55d4fbc11ee0 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7ceffd640 10 client.17481 handle_command_reply: tid=0 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 client.17481 shutdown 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 msgr2=0x55d4fbbff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbbff930 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa7e801e040 tx=0x7fa7e801ce60 comp rx=0 tx=0).stop 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 shutdown_connections 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc11ee0 0x55d4fbc142d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbc01e90 0x55d4fbbff930 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbbff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 --2- 172.21.15.113:0/1414010860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbbfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7ceffd640 0 client.17481 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 >> 172.21.15.113:0/1414010860 conn(0x55d4fbbeecb0 msgr2=0x55d4fbbf0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 shutdown_connections 2024-04-22T07:23:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 1 -- 172.21.15.113:0/1414010860 wait complete. 2024-04-22T07:23:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.874+0000 7fa7f886e740 20 client.17481 trim_cache size 0 max 16384 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.891+0000 7fa7f886e740 1 Processor -- start 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.891+0000 7fa7f886e740 1 -- start start 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 0x55d4fbc015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbc01e90 0x55d4fbbfdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7f886e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01ae6e0 con 0x55d4fbbf4230 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7f886e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01abfb0 con 0x55d4fbc01e90 2024-04-22T07:23:28.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7f886e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01a9e40 con 0x55d4fbbf4b50 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 0x55d4fbc015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 0x55d4fbc015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60012/0 (socket says 172.21.15.113:60012) 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/2412689707 learned_addr learned my addr 172.21.15.113:0/2412689707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/2412689707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 msgr2=0x55d4fbc0dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/2412689707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbc01e90 msgr2=0x55d4fbbfdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbc01e90 0x55d4fbbfdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 -- 172.21.15.113:0/2412689707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4fbbf5010 con 0x55d4fbbf4230 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 0x55d4fbc015e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00ce2a0 tx=0x7fa7e0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e56770fa3ed1267 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7e57fa640 1 -- 172.21.15.113:0/2412689707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001d040 con 0x55d4fbbf4230 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7e57fa640 1 -- 172.21.15.113:0/2412689707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa7e00a5160 con 0x55d4fbbf4230 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.892+0000 7fa7e57fa640 1 -- 172.21.15.113:0/2412689707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00a4cd0 con 0x55d4fbbf4230 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 -- 172.21.15.113:0/2412689707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 msgr2=0x55d4fbc015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 0x55d4fbc015e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00ce2a0 tx=0x7fa7e0004570 comp rx=0 tx=0).stop 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 -- 172.21.15.113:0/2412689707 shutdown_connections 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbc01e90 0x55d4fbbfdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 --2- 172.21.15.113:0/2412689707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4230 0x55d4fbc015e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 -- 172.21.15.113:0/2412689707 >> 172.21.15.113:0/2412689707 conn(0x55d4fbbeecb0 msgr2=0x55d4fbbfd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 -- 172.21.15.113:0/2412689707 shutdown_connections 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 -- 172.21.15.113:0/2412689707 wait complete. 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 -- start start 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.893+0000 7fa7f886e740 1 Processor -- start 2024-04-22T07:23:28.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbc0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbc01e90 0x55d4fbc07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01aa490 con 0x55d4fbbf4b50 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01ab8e0 con 0x55d4fbbf4230 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01aab40 con 0x55d4fbc01e90 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 10 client.0 _ll_register_callbacks cb 0x7fa7d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60026/0 (socket says 172.21.15.113:60026) 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 -- 172.21.15.113:0/4165382635 learned_addr learned my addr 172.21.15.113:0/4165382635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbc0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7effff640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbc01e90 0x55d4fbc07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 -- 172.21.15.113:0/4165382635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbc01e90 msgr2=0x55d4fbc07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbc01e90 0x55d4fbc07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 -- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 msgr2=0x55d4fbc0a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbc0a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 -- 172.21.15.113:0/4165382635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4fbc07740 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7e7fff640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0a670 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa7f006b6f0 tx=0x7fa7d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d79c51bf51206a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7ceffd640 1 -- 172.21.15.113:0/4165382635 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d802c680 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7ceffd640 1 -- 172.21.15.113:0/4165382635 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa7d8005240 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.894+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e00a5160 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7ceffd640 1 -- 172.21.15.113:0/4165382635 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa7d8005440 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 10 client.17487 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55d4fbc78c60 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7ceffd640 1 -- 172.21.15.113:0/4165382635 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa7e00a5160 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7ceffd640 1 -- 172.21.15.113:0/4165382635 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa7d8044070 con 0x55d4fbbf4b50 2024-04-22T07:23:28.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 10 client.17487 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 20 client.17487 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc79410 0x55d4fbc7b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 4 client.17487 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4fbb60a70 con 0x55d4fbc79410 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.895+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc79410 0x55d4fbc7b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7ceffd640 10 client.17487 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7ef7fe640 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc79410 0x55d4fbc7b830 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af872cbbe8f9e395 server_cookie=b12781b96cd02f07 in_seq=0 out_seq=0 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7ceffd640 1 -- 172.21.15.113:0/4165382635 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d4fbb60a70 con 0x55d4fbc79410 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7ceffd640 10 client.17487 handle_command_reply: tid=0 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 client.17487 shutdown 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 msgr2=0x55d4fbc0a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0a670 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa7f006b6f0 tx=0x7fa7d8004270 comp rx=0 tx=0).stop 2024-04-22T07:23:28.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 shutdown_connections 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55d4fbc79410 0x55d4fbc7b830 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d4fbc01e90 0x55d4fbc07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d4fbbf4b50 0x55d4fbc0a670 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7ceffd640 0 client.17487 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 --2- 172.21.15.113:0/4165382635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4fbbf4230 0x55d4fbc0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 >> 172.21.15.113:0/4165382635 conn(0x55d4fbbeecb0 msgr2=0x55d4fbbfdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 shutdown_connections 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 1 -- 172.21.15.113:0/4165382635 wait complete. 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.896+0000 7fa7f886e740 20 client.17487 trim_cache size 0 max 16384 2024-04-22T07:23:28.902 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:23:28.903 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:23:28.903 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:23:28.903 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa7dc0a3d20 msgr2=0x7fa7dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa7dc0a3d20 0x7fa7dc0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800ec60 tx=0x7fa7e8009e30 comp rx=0 tx=0).stop 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 msgr2=0x7fa7f019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 0x7fa7f019e580 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c890 tx=0x7fa7e000cd50 comp rx=0 tx=0).stop 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 shutdown_connections 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa7dc0a3d20 0x7fa7dc0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7f0198640 0x7fa7f0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7f0106150 0x7fa7f019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 --2- 172.21.15.113:0/1468905319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa7f0101080 0x7fa7f019e580 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 >> 172.21.15.113:0/1468905319 conn(0x7fa7f0078000 msgr2=0x7fa7f0069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 shutdown_connections 2024-04-22T07:23:28.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:28.897+0000 7fa7f5c74640 1 -- 172.21.15.113:0/1468905319 wait complete. 2024-04-22T07:23:28.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:23:28.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:23:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 -- 172.21.15.113:0/1848894142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 msgr2=0x7ff894100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 --2- 172.21.15.113:0/1848894142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff894100940 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff890009930 tx=0x7ff89002bc30 comp rx=0 tx=0).stop 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 -- 172.21.15.113:0/1848894142 shutdown_connections 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 --2- 172.21.15.113:0/1848894142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff894100e80 0x7ff89410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 --2- 172.21.15.113:0/1848894142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff894100940 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 --2- 172.21.15.113:0/1848894142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 0x7ff894107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 -- 172.21.15.113:0/1848894142 >> 172.21.15.113:0/1848894142 conn(0x7ff894077450 msgr2=0x7ff8940fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 -- 172.21.15.113:0/1848894142 shutdown_connections 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.005+0000 7ff89b650640 1 -- 172.21.15.113:0/1848894142 wait complete. 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 Processor -- start 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 -- start start 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff89419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff894100e80 0x7ff89419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 0x7ff8941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941123e0 con 0x7ff8941004c0 2024-04-22T07:23:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff894112260 con 0x7ff894100e80 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff89b650640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff894112560 con 0x7ff894106f50 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff89419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff89419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60036/0 (socket says 172.21.15.113:60036) 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 -- 172.21.15.113:0/3443712405 learned_addr learned my addr 172.21.15.113:0/3443712405 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff898beb640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff894100e80 0x7ff89419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff899bed640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 0x7ff8941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 -- 172.21.15.113:0/3443712405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 msgr2=0x7ff8941986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 0x7ff8941986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 -- 172.21.15.113:0/3443712405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff894100e80 msgr2=0x7ff89419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff894100e80 0x7ff89419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff8993ec640 1 -- 172.21.15.113:0/3443712405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff894198f80 con 0x7ff8941004c0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.006+0000 7ff899bed640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 0x7ff8941986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.007+0000 7ff8993ec640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff89419e5b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff88400d950 tx=0x7ff88400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.007+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884017070 con 0x7ff8941004c0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.007+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff884009c30 con 0x7ff8941004c0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.007+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff894199270 con 0x7ff8941004c0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.007+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88400f6b0 con 0x7ff8941004c0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.007+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8941a5530 con 0x7ff8941004c0 2024-04-22T07:23:29.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.008+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8941995b0 con 0x7ff8941004c0 2024-04-22T07:23:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.009+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7ff8840109c0 con 0x7ff8941004c0 2024-04-22T07:23:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.009+0000 7ff88a7fc640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff8800a3d20 0x7ff8800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.009+0000 7ff898beb640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff8800a3d20 0x7ff8800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.010+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7ff8840d05f0 con 0x7ff8941004c0 2024-04-22T07:23:29.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.010+0000 7ff898beb640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff8800a3d20 0x7ff8800a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff890005ab0 tx=0x7ff890000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.013+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff884009de0 con 0x7ff8941004c0 2024-04-22T07:23:29.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.228+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff894199ba0 con 0x7ff8941004c0 2024-04-22T07:23:29.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.229+0000 7ff88a7fc640 1 -- 172.21.15.113:0/3443712405 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7ff88409caf0 con 0x7ff8941004c0 2024-04-22T07:23:29.228 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:23:29.228 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:23:29.228 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff8800a3d20 msgr2=0x7ff8800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff8800a3d20 0x7ff8800a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff890005ab0 tx=0x7ff890000bc0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 msgr2=0x7ff89419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff89419e5b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff88400d950 tx=0x7ff88400de10 comp rx=0 tx=0).stop 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 shutdown_connections 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7ff8800a3d20 0x7ff8800a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff894106f50 0x7ff8941986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff894100e80 0x7ff89419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 --2- 172.21.15.113:0/3443712405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8941004c0 0x7ff89419e5b0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 >> 172.21.15.113:0/3443712405 conn(0x7ff894077450 msgr2=0x7ff894105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 shutdown_connections 2024-04-22T07:23:29.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.231+0000 7ff89b650640 1 -- 172.21.15.113:0/3443712405 wait complete. 2024-04-22T07:23:29.240 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:23:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- 172.21.15.113:0/2659295943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 msgr2=0x7facf0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- 172.21.15.113:0/2659295943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf0101470 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7face0009900 tx=0x7face002bc30 comp rx=0 tx=0).stop 2024-04-22T07:23:29.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- 172.21.15.113:0/2659295943 shutdown_connections 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- 172.21.15.113:0/2659295943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf0106160 0x7facf010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- 172.21.15.113:0/2659295943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf0101a40 0x7facf0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- 172.21.15.113:0/2659295943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf0101470 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- 172.21.15.113:0/2659295943 >> 172.21.15.113:0/2659295943 conn(0x7facf0077f50 msgr2=0x7facf0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- 172.21.15.113:0/2659295943 shutdown_connections 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- 172.21.15.113:0/2659295943 wait complete. 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 Processor -- start 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- start start 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf0106160 0x7facf019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf01985e0 0x7facf0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7facf01111e0 con 0x7facf0101090 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7facf0111060 con 0x7facf0106160 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.342+0000 7facf717e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7facf0111360 con 0x7facf01985e0 2024-04-22T07:23:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60058/0 (socket says 172.21.15.113:60058) 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 -- 172.21.15.113:0/523954248 learned_addr learned my addr 172.21.15.113:0/523954248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7faceffff640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf0106160 0x7facf019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 -- 172.21.15.113:0/523954248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf01985e0 msgr2=0x7facf0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf01985e0 0x7facf0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 -- 172.21.15.113:0/523954248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf0106160 msgr2=0x7facf019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf0106160 0x7facf019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 -- 172.21.15.113:0/523954248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf0199100 con 0x7facf0101090 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf4f1a640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf019e430 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7face0005940 tx=0x7face0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facedffb640 1 -- 172.21.15.113:0/523954248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0041070 con 0x7facf0101090 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facedffb640 1 -- 172.21.15.113:0/523954248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7face00052f0 con 0x7facf0101090 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf0199390 con 0x7facf0101090 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facedffb640 1 -- 172.21.15.113:0/523954248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0034440 con 0x7facf0101090 2024-04-22T07:23:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.343+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01a55e0 con 0x7facf0101090 2024-04-22T07:23:29.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.345+0000 7facedffb640 1 -- 172.21.15.113:0/523954248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7face0044070 con 0x7facf0101090 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 Processor -- start 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 -- start start 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facedffb640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7facd80a3d20 0x7facd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b29ba00 0x56126b29de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56126b1f71f0 con 0x56126b292e80 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56126b1944b0 con 0x56126b2924d0 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf571b640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7facd80a3d20 0x7facd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf9d6a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56126b1f31b0 con 0x56126b29ba00 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facedffb640 1 -- 172.21.15.113:0/523954248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7face00f25e0 con 0x7facf0101090 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60062/0 (socket says 172.21.15.113:60062) 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 -- 172.21.15.113:0/1525922084 learned_addr learned my addr 172.21.15.113:0/1525922084 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf4f1a640 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7facf571b640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7facd80a3d20 0x7facd80a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7face8005fd0 tx=0x7face8007f20 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 -- 172.21.15.113:0/1525922084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b29ba00 msgr2=0x56126b29de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b29ba00 0x56126b29de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 -- 172.21.15.113:0/1525922084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 msgr2=0x56126b2928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.346+0000 7faceffff640 1 -- 172.21.15.113:0/1525922084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126b1861b0 con 0x56126b292e80 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7faceffff640 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29b430 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7face400cbd0 tx=0x7face400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d54adbf6f88957 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facd6ffd640 1 -- 172.21.15.113:0/1525922084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4005220 con 0x56126b292e80 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facd6ffd640 1 -- 172.21.15.113:0/1525922084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7face4004c70 con 0x56126b292e80 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facd6ffd640 1 -- 172.21.15.113:0/1525922084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4010810 con 0x56126b292e80 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 -- 172.21.15.113:0/1525922084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 msgr2=0x56126b29b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29b430 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7face400cbd0 tx=0x7face400e720 comp rx=0 tx=0).stop 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 -- 172.21.15.113:0/1525922084 shutdown_connections 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b29ba00 0x56126b29de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29b430 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1525922084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 -- 172.21.15.113:0/1525922084 >> 172.21.15.113:0/1525922084 conn(0x56126b28cba0 msgr2=0x56126b28cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 -- 172.21.15.113:0/1525922084 shutdown_connections 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.347+0000 7facf9d6a740 1 -- 172.21.15.113:0/1525922084 wait complete. 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 -- start start 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 Processor -- start 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b29b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b298690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b298c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56126b29e970 con 0x56126b29ba00 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56126b1b0500 con 0x56126b2924d0 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56126b29eaf0 con 0x56126b292e80 2024-04-22T07:23:29.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf9d6a740 10 client.0 _ll_register_callbacks cb 0x56126b17eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf571b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b298c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf571b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b298c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60074/0 (socket says 172.21.15.113:60074) 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf571b640 1 -- 172.21.15.113:0/1317863510 learned_addr learned my addr 172.21.15.113:0/1317863510 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.348+0000 7facf4f1a640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b29b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf571b640 1 -- 172.21.15.113:0/1317863510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 msgr2=0x56126b298690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf571b640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b298690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf571b640 1 -- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 msgr2=0x56126b29b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf571b640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b29b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf571b640 1 -- 172.21.15.113:0/1317863510 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56126af093f0 con 0x56126b29ba00 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf4f1a640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b29b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf571b640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b298c10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7face8011a50 tx=0x7face8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9f1e851c9fa0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7faccffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7faccffff640 1 -- 172.21.15.113:0/1317863510 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8010ce0 con 0x56126b29ba00 2024-04-22T07:23:29.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7faccffff640 1 -- 172.21.15.113:0/1317863510 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7face801d460 con 0x56126b29ba00 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face4004c70 con 0x56126b29ba00 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7faccffff640 1 -- 172.21.15.113:0/1317863510 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7face802b050 con 0x56126b29ba00 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 10 client.17505 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56126afd37d0 con 0x56126b29ba00 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7faccffff640 1 -- 172.21.15.113:0/1317863510 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7face4004c70 con 0x56126b29ba00 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7faccffff640 1 -- 172.21.15.113:0/1317863510 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7face800f6e0 con 0x56126b29ba00 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 10 client.17505 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 20 client.17505 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b2abc40 0x56126b2ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 4 client.17505 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.349+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56126b196930 con 0x56126b2abc40 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.350+0000 7facf4f1a640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b2abc40 0x56126b2ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.350+0000 7faccffff640 10 client.17505 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.350+0000 7facf4f1a640 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b2abc40 0x56126b2ae100 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfdcc2e36da985f server_cookie=89faa2a123adbe9d in_seq=0 out_seq=0 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7faccffff640 1 -- 172.21.15.113:0/1317863510 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56126b196930 con 0x56126b2abc40 2024-04-22T07:23:29.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7faccffff640 10 client.17505 handle_command_reply: tid=0 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 client.17505 shutdown 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 msgr2=0x56126b298c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b298c10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7face8011a50 tx=0x7face8004a40 comp rx=0 tx=0).stop 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 shutdown_connections 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b2abc40 0x56126b2ae100 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b298c10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b298690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 --2- 172.21.15.113:0/1317863510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b29b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7faccffff640 0 client.17505 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 >> 172.21.15.113:0/1317863510 conn(0x56126b28cba0 msgr2=0x56126b28e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 shutdown_connections 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 1 -- 172.21.15.113:0/1317863510 wait complete. 2024-04-22T07:23:29.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.351+0000 7facf9d6a740 20 client.17505 trim_cache size 0 max 16384 2024-04-22T07:23:29.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 Processor -- start 2024-04-22T07:23:29.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 -- start start 2024-04-22T07:23:29.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b2924d0 0x56126b2a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b29ba00 0x56126b29b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7facf01b07b0 con 0x56126b292e80 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ae080 con 0x56126b29ba00 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7facf9d6a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7facf01abef0 con 0x56126b2924d0 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7faceffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7faceffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60078/0 (socket says 172.21.15.113:60078) 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.364+0000 7faceffff640 1 -- 172.21.15.113:0/3230143734 learned_addr learned my addr 172.21.15.113:0/3230143734 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7faceffff640 1 -- 172.21.15.113:0/3230143734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b2924d0 msgr2=0x56126b2a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7faceffff640 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b2924d0 0x56126b2a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7faceffff640 1 -- 172.21.15.113:0/3230143734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b29ba00 msgr2=0x56126b29b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7faceffff640 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b29ba00 0x56126b29b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7faceffff640 1 -- 172.21.15.113:0/3230143734 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126b29e1c0 con 0x56126b292e80 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7faceffff640 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29adc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7face400cb20 tx=0x7face400c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8477b9738eab873 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facd67fc640 1 -- 172.21.15.113:0/3230143734 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4010630 con 0x56126b292e80 2024-04-22T07:23:29.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facd67fc640 1 -- 172.21.15.113:0/3230143734 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7face400c7d0 con 0x56126b292e80 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facd67fc640 1 -- 172.21.15.113:0/3230143734 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face401aba0 con 0x56126b292e80 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facf9d6a740 1 -- 172.21.15.113:0/3230143734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 msgr2=0x56126b29adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29adc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7face400cb20 tx=0x7face400c390 comp rx=0 tx=0).stop 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facf9d6a740 1 -- 172.21.15.113:0/3230143734 shutdown_connections 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b29ba00 0x56126b29b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.365+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b292e80 0x56126b29adc0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3230143734 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b2924d0 0x56126b2a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- 172.21.15.113:0/3230143734 >> 172.21.15.113:0/3230143734 conn(0x56126b28cba0 msgr2=0x56126b28e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- 172.21.15.113:0/3230143734 shutdown_connections 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- 172.21.15.113:0/3230143734 wait complete. 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- start start 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 Processor -- start 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b29f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b29fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ac540 con 0x56126b29ba00 2024-04-22T07:23:29.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7facf01ad9b0 con 0x56126b2924d0 2024-04-22T07:23:29.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7facf01acc10 con 0x56126b292e80 2024-04-22T07:23:29.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf9d6a740 10 client.0 _ll_register_callbacks cb 0x7face8011750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:29.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf571b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b29fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf571b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b29fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60090/0 (socket says 172.21.15.113:60090) 2024-04-22T07:23:29.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf571b640 1 -- 172.21.15.113:0/3813184825 learned_addr learned my addr 172.21.15.113:0/3813184825 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7facf4f1a640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.366+0000 7faceffff640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b29f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf571b640 1 -- 172.21.15.113:0/3813184825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 msgr2=0x56126b29f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf571b640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b29f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf571b640 1 -- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 msgr2=0x56126b2a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf571b640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf571b640 1 -- 172.21.15.113:0/3813184825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56126b2a02b0 con 0x56126b29ba00 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf571b640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b29fa80 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7face800fcc0 tx=0x7face8007240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8a9fec39e9024a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7faccffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7faccffff640 1 -- 172.21.15.113:0/3813184825 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face800f430 con 0x56126b29ba00 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7faccffff640 1 -- 172.21.15.113:0/3813184825 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7face800f5d0 con 0x56126b29ba00 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face400c7d0 con 0x56126b29ba00 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7faccffff640 1 -- 172.21.15.113:0/3813184825 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7face800f810 con 0x56126b29ba00 2024-04-22T07:23:29.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf9d6a740 10 client.17511 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56126b3122e0 con 0x56126b29ba00 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7faccffff640 1 -- 172.21.15.113:0/3813184825 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7face400c7d0 con 0x56126b29ba00 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7faccffff640 1 -- 172.21.15.113:0/3813184825 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7face80046a0 con 0x56126b29ba00 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf9d6a740 10 client.17511 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf9d6a740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.367+0000 7facf9d6a740 20 client.17511 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b3129a0 0x56126b314d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 4 client.17511 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56126b204c10 con 0x56126b3129a0 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf4f1a640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b3129a0 0x56126b314d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf4f1a640 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b3129a0 0x56126b314d90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ffb489c14e3b10 server_cookie=98896d2eea9b0498 in_seq=0 out_seq=0 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7faccffff640 10 client.17511 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7faccffff640 1 -- 172.21.15.113:0/3813184825 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56126b204c10 con 0x56126b3129a0 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7faccffff640 10 client.17511 handle_command_reply: tid=0 2024-04-22T07:23:29.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 client.17511 shutdown 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 msgr2=0x56126b29fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b29fa80 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7face800fcc0 tx=0x7face8007240 comp rx=0 tx=0).stop 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 shutdown_connections 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56126b3129a0 0x56126b314d90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56126b29ba00 0x56126b29fa80 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56126b292e80 0x56126b29f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7faccffff640 0 client.17511 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 --2- 172.21.15.113:0/3813184825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56126b2924d0 0x56126b2a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.368+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 >> 172.21.15.113:0/3813184825 conn(0x56126b28cba0 msgr2=0x56126b28b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 shutdown_connections 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf9d6a740 1 -- 172.21.15.113:0/3813184825 wait complete. 2024-04-22T07:23:29.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf9d6a740 20 client.17511 trim_cache size 0 max 16384 2024-04-22T07:23:29.378 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:23:29.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7facd80a3d20 msgr2=0x7facd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7facd80a3d20 0x7facd80a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7face8005fd0 tx=0x7face8007f20 comp rx=0 tx=0).stop 2024-04-22T07:23:29.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 msgr2=0x7facf019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf019e430 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7face0005940 tx=0x7face0004040 comp rx=0 tx=0).stop 2024-04-22T07:23:29.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 shutdown_connections 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7facd80a3d20 0x7facd80a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf01985e0 0x7facf0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf0106160 0x7facf019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 --2- 172.21.15.113:0/523954248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facf0101090 0x7facf019e430 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 >> 172.21.15.113:0/523954248 conn(0x7facf0077f50 msgr2=0x7facf0069440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 shutdown_connections 2024-04-22T07:23:29.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.369+0000 7facf717e640 1 -- 172.21.15.113:0/523954248 wait complete. 2024-04-22T07:23:29.380 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:29 smithi113 ceph-mon[27933]: pgmap v272: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:23:29.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:23:29.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/1125400359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4102b40 msgr2=0x7f6ea4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/1125400359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4102b40 0x7f6ea4106b80 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009900 tx=0x7f6e9802bc30 comp rx=0 tx=0).stop 2024-04-22T07:23:29.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/1125400359 shutdown_connections 2024-04-22T07:23:29.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/1125400359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ea4107260 0x7f6ea410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/1125400359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4102b40 0x7f6ea4106b80 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/1125400359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4102190 0x7f6ea4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/1125400359 >> 172.21.15.113:0/1125400359 conn(0x7f6ea40793a0 msgr2=0x7f6ea40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/1125400359 shutdown_connections 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.483+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/1125400359 wait complete. 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 Processor -- start 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 -- start start 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ea4102190 0x7f6ea4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 0x7f6ea4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4074cd0 0x7f6ea41a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41122e0 con 0x7f6ea4107260 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4112160 con 0x7f6ea4074cd0 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea9ab2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea4112460 con 0x7f6ea4102190 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 0x7f6ea4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 0x7f6ea4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60108/0 (socket says 172.21.15.113:60108) 2024-04-22T07:23:29.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/2372960283 learned_addr learned my addr 172.21.15.113:0/2372960283 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4074cd0 0x7f6ea41a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ea4102190 0x7f6ea4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/2372960283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ea4102190 msgr2=0x7f6ea4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ea4102190 0x7f6ea4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/2372960283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4074cd0 msgr2=0x7f6ea41a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4074cd0 0x7f6ea41a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/2372960283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea41a7090 con 0x7f6ea4107260 2024-04-22T07:23:29.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.484+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4074cd0 0x7f6ea41a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:29.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.485+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 0x7f6ea4074790 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6e98000c00 tx=0x7f6e9802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.485+0000 7f6ea0ff9640 1 -- 172.21.15.113:0/2372960283 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98041070 con 0x7f6ea4107260 2024-04-22T07:23:29.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.485+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea41a7320 con 0x7f6ea4107260 2024-04-22T07:23:29.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.485+0000 7f6ea0ff9640 1 -- 172.21.15.113:0/2372960283 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6e980044d0 con 0x7f6ea4107260 2024-04-22T07:23:29.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.485+0000 7f6ea0ff9640 1 -- 172.21.15.113:0/2372960283 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98003d50 con 0x7f6ea4107260 2024-04-22T07:23:29.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.485+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea41a78c0 con 0x7f6ea4107260 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6ea0ff9640 1 -- 172.21.15.113:0/2372960283 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6e98044070 con 0x7f6ea4107260 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 Processor -- start 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 -- start start 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1357430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1359e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559de12b31f0 con 0x559de134ee80 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559de12504b0 con 0x559de134e4d0 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6eac777740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559de12af1b0 con 0x559de1357a00 2024-04-22T07:23:29.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6ea0ff9640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6e8c0a3d50 0x7f6e8c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1357430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1357430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60112/0 (socket says 172.21.15.113:60112) 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.487+0000 7f6ea37fe640 1 -- 172.21.15.113:0/2199443940 learned_addr learned my addr 172.21.15.113:0/2199443940 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6e8c0a3d50 0x7f6e8c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea0ff9640 1 -- 172.21.15.113:0/2372960283 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f6e980f23a0 con 0x7f6ea4107260 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1359e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea37fe640 1 -- 172.21.15.113:0/2199443940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 msgr2=0x559de1359e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1359e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea37fe640 1 -- 172.21.15.113:0/2199443940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 msgr2=0x559de134e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea37fe640 1 -- 172.21.15.113:0/2199443940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de12421b0 con 0x559de134ee80 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1359e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1357430 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400bcb0 tx=0x7f6e9400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563b345eb8eb1f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6e8c0a3d50 0x7f6e8c0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c007310 tx=0x7f6e9c007450 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6e89ffb640 1 -- 172.21.15.113:0/2199443940 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94015070 con 0x559de134ee80 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6e89ffb640 1 -- 172.21.15.113:0/2199443940 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6e94011910 con 0x559de134ee80 2024-04-22T07:23:29.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.488+0000 7f6e89ffb640 1 -- 172.21.15.113:0/2199443940 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e940109c0 con 0x559de134ee80 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 -- 172.21.15.113:0/2199443940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 msgr2=0x559de1357430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1357430 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400bcb0 tx=0x7f6e9400f730 comp rx=0 tx=0).stop 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 -- 172.21.15.113:0/2199443940 shutdown_connections 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1359e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1357430 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 --2- 172.21.15.113:0/2199443940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 -- 172.21.15.113:0/2199443940 >> 172.21.15.113:0/2199443940 conn(0x559de1348ba0 msgr2=0x559de1348f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 -- 172.21.15.113:0/2199443940 shutdown_connections 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 -- 172.21.15.113:0/2199443940 wait complete. 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 -- start start 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.489+0000 7f6eac777740 1 Processor -- start 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de1357240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de13545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1354b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559de135a970 con 0x559de134ee80 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559de126c500 con 0x559de134e4d0 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559de135aaf0 con 0x559de1357a00 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 10 client.0 _ll_register_callbacks cb 0x559de123ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de13545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de13545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60124/0 (socket says 172.21.15.113:60124) 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/858953514 learned_addr learned my addr 172.21.15.113:0/858953514 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de1357240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1354b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 msgr2=0x559de1354b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1354b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/858953514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 msgr2=0x559de1357240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de1357240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 -- 172.21.15.113:0/858953514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de0fc53f0 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1354b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de13545d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6e980be040 tx=0x7f6e980be1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3ed81da6c5a99a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6e82ffd640 1 -- 172.21.15.113:0/858953514 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9804b040 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6e82ffd640 1 -- 172.21.15.113:0/858953514 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f6e980c6a90 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e94011910 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6e82ffd640 1 -- 172.21.15.113:0/858953514 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6e980f21c0 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 10 client.17523 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.490+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559de108f7d0 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6e82ffd640 1 -- 172.21.15.113:0/858953514 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6e94011910 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6e82ffd640 1 -- 172.21.15.113:0/858953514 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f6e980c2170 con 0x559de134ee80 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6eac777740 10 client.17523 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:29.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6eac777740 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6eac777740 20 client.17523 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6eac777740 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1367aa0 0x559de1369ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6eac777740 4 client.17523 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559de1252930 con 0x559de1367aa0 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.491+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1367aa0 0x559de1369ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6e82ffd640 10 client.17523 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1367aa0 0x559de1369ed0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5503e25d8ff6a554 server_cookie=a0884a9e25971d87 in_seq=0 out_seq=0 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6e82ffd640 1 -- 172.21.15.113:0/858953514 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559de1252930 con 0x559de1367aa0 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6e82ffd640 10 client.17523 handle_command_reply: tid=0 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 client.17523 shutdown 2024-04-22T07:23:29.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 msgr2=0x559de13545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de13545d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6e980be040 tx=0x7f6e980be1d0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 shutdown_connections 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1367aa0 0x559de1369ed0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1357a00 0x559de1354b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de13545d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6e82ffd640 0 client.17523 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 --2- 172.21.15.113:0/858953514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de1357240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.492+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 >> 172.21.15.113:0/858953514 conn(0x559de1348ba0 msgr2=0x559de134a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.493+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 shutdown_connections 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.493+0000 7f6eac777740 1 -- 172.21.15.113:0/858953514 wait complete. 2024-04-22T07:23:29.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.493+0000 7f6eac777740 20 client.17523 trim_cache size 0 max 16384 2024-04-22T07:23:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 Processor -- start 2024-04-22T07:23:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 -- start start 2024-04-22T07:23:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1348ba0 0x559de1363830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1356d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41b2a60 con 0x559de134ee80 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41b0330 con 0x559de134e4d0 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6eac777740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ae1a0 con 0x559de1348ba0 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6ea3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1356d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6ea3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1356d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60136/0 (socket says 172.21.15.113:60136) 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.506+0000 7f6ea3fff640 1 -- 172.21.15.113:0/1122385764 learned_addr learned my addr 172.21.15.113:0/1122385764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea3fff640 1 -- 172.21.15.113:0/1122385764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1348ba0 msgr2=0x559de1363830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1348ba0 0x559de1363830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea3fff640 1 -- 172.21.15.113:0/1122385764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 msgr2=0x559de134f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea3fff640 1 -- 172.21.15.113:0/1122385764 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559de13489d0 con 0x559de134ee80 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1356d00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c011790 tx=0x7f6e9c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1795c1e073b519e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6e897fa640 1 -- 172.21.15.113:0/1122385764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c004d20 con 0x559de134ee80 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6e897fa640 1 -- 172.21.15.113:0/1122385764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6e9c021070 con 0x559de134ee80 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6e897fa640 1 -- 172.21.15.113:0/1122385764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c005380 con 0x559de134ee80 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6eac777740 1 -- 172.21.15.113:0/1122385764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 msgr2=0x559de1356d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6eac777740 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1356d00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c011790 tx=0x7f6e9c014070 comp rx=0 tx=0).stop 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6eac777740 1 -- 172.21.15.113:0/1122385764 shutdown_connections 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6eac777740 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de134ee80 0x559de1356d00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6eac777740 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134e4d0 0x559de134f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.507+0000 7f6eac777740 1 --2- 172.21.15.113:0/1122385764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de1348ba0 0x559de1363830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- 172.21.15.113:0/1122385764 >> 172.21.15.113:0/1122385764 conn(0x559de1354c70 msgr2=0x559de134f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- 172.21.15.113:0/1122385764 shutdown_connections 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- 172.21.15.113:0/1122385764 wait complete. 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- start start 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 Processor -- start 2024-04-22T07:23:29.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 0x559de13514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de134e4d0 0x559de13519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134ee80 0x559de1351f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41ae7f0 con 0x559de1348ba0 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41afc60 con 0x559de134ee80 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea41aeec0 con 0x559de134e4d0 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6eac777740 10 client.0 _ll_register_callbacks cb 0x7f6e980ba6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 0x559de13514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6ea37fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 0x559de13514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60146/0 (socket says 172.21.15.113:60146) 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.508+0000 7f6ea37fe640 1 -- 172.21.15.113:0/24630227 learned_addr learned my addr 172.21.15.113:0/24630227 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea37fe640 1 -- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de134e4d0 msgr2=0x559de13519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de134e4d0 0x559de13519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea3fff640 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134ee80 0x559de1351f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea37fe640 1 -- 172.21.15.113:0/24630227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134ee80 msgr2=0x559de1351f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134ee80 0x559de1351f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea37fe640 1 -- 172.21.15.113:0/24630227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de13cd250 con 0x559de1348ba0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6ea37fe640 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 0x559de13514a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400bbd0 tx=0x7f6e940104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca8949c6cc85ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6e82ffd640 1 -- 172.21.15.113:0/24630227 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94015050 con 0x559de1348ba0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6e82ffd640 1 -- 172.21.15.113:0/24630227 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f6e94004240 con 0x559de1348ba0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9c021070 con 0x559de1348ba0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6e82ffd640 1 -- 172.21.15.113:0/24630227 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6e94004460 con 0x559de1348ba0 2024-04-22T07:23:29.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 10 client.17529 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559de13ce4b0 con 0x559de1348ba0 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6e82ffd640 1 -- 172.21.15.113:0/24630227 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6e9c021070 con 0x559de1348ba0 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6e82ffd640 1 -- 172.21.15.113:0/24630227 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f6e94028600 con 0x559de1348ba0 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 10 client.17529 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 10 client.17529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 20 client.17529 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1357a00 0x559de13d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 4 client.17529 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.509+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559de12c0c10 con 0x559de1357a00 2024-04-22T07:23:29.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1357a00 0x559de13d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6e82ffd640 10 client.17529 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6ea2ffd640 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1357a00 0x559de13d0b60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8df9875bba67501f server_cookie=b8c6aee914f29513 in_seq=0 out_seq=0 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6e82ffd640 1 -- 172.21.15.113:0/24630227 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559de12c0c10 con 0x559de1357a00 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6e82ffd640 10 client.17529 handle_command_reply: tid=0 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 client.17529 shutdown 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 msgr2=0x559de13514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 0x559de13514a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400bbd0 tx=0x7f6e940104d0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 shutdown_connections 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x559de1357a00 0x559de13d0b60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559de134ee80 0x559de1351f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559de134e4d0 0x559de13519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6e82ffd640 0 client.17529 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:23:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 --2- 172.21.15.113:0/24630227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559de1348ba0 0x559de13514a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.510+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 >> 172.21.15.113:0/24630227 conn(0x559de1354c70 msgr2=0x559de134a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 shutdown_connections 2024-04-22T07:23:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6eac777740 1 -- 172.21.15.113:0/24630227 wait complete. 2024-04-22T07:23:29.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6eac777740 20 client.17529 trim_cache size 0 max 16384 2024-04-22T07:23:29.522 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:23:29.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6e8c0a3d50 msgr2=0x7f6e8c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6e8c0a3d50 0x7f6e8c0a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c007310 tx=0x7f6e9c007450 comp rx=0 tx=0).stop 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 msgr2=0x7f6ea4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 0x7f6ea4074790 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6e98000c00 tx=0x7f6e9802cdf0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 shutdown_connections 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f6e8c0a3d50 0x7f6e8c0a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea4074cd0 0x7f6ea41a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ea4107260 0x7f6ea4074790 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 --2- 172.21.15.113:0/2372960283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ea4102190 0x7f6ea4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 >> 172.21.15.113:0/2372960283 conn(0x7f6ea40793a0 msgr2=0x7f6ea40fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 shutdown_connections 2024-04-22T07:23:29.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.511+0000 7f6ea9ab2640 1 -- 172.21.15.113:0/2372960283 wait complete. 2024-04-22T07:23:29.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:23:29.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:29 smithi133 ceph-mon[31198]: pgmap v272: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:23:29.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:29 smithi154 ceph-mon[31862]: pgmap v272: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:23:29.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.627+0000 7fead343a640 1 -- 172.21.15.113:0/3891901353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 msgr2=0x7feacc1025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.627+0000 7fead343a640 1 --2- 172.21.15.113:0/3891901353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc1025c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7feac4009ca0 tx=0x7feac402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- 172.21.15.113:0/3891901353 shutdown_connections 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 --2- 172.21.15.113:0/3891901353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 0x7feacc10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 --2- 172.21.15.113:0/3891901353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feacc102b90 0x7feacc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 --2- 172.21.15.113:0/3891901353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc1025c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- 172.21.15.113:0/3891901353 >> 172.21.15.113:0/3891901353 conn(0x7feacc0fda60 msgr2=0x7feacc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- 172.21.15.113:0/3891901353 shutdown_connections 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- 172.21.15.113:0/3891901353 wait complete. 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 Processor -- start 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- start start 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 0x7feacc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feacc1986f0 0x7feacc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feacc112330 con 0x7feacc1021e0 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1121b0 con 0x7feacc1072b0 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead343a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1124b0 con 0x7feacc1986f0 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.628+0000 7fead11d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60158/0 (socket says 172.21.15.113:60158) 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 -- 172.21.15.113:0/1276073042 learned_addr learned my addr 172.21.15.113:0/1276073042 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead19d7640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feacc1986f0 0x7feacc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead09d5640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 0x7feacc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 -- 172.21.15.113:0/1276073042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feacc1986f0 msgr2=0x7feacc198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feacc1986f0 0x7feacc198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 -- 172.21.15.113:0/1276073042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 msgr2=0x7feacc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 0x7feacc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 -- 172.21.15.113:0/1276073042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc1992f0 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead09d5640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 0x7feacc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead11d6640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc19e630 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7feac4005e30 tx=0x7feac4035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7feac27fc640 1 -- 172.21.15.113:0/1276073042 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4039040 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc199580 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7feac27fc640 1 -- 172.21.15.113:0/1276073042 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feac4044070 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7feac27fc640 1 -- 172.21.15.113:0/1276073042 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40044b0 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.629+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc1a56c0 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.631+0000 7feac27fc640 1 -- 172.21.15.113:0/1276073042 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7feac40327a0 con 0x7feacc1021e0 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 Processor -- start 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 -- start start 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d77c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d785430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 0x558a9d787e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558a9d6e11f0 con 0x558a9d785a00 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a9d67e4b0 con 0x558a9d77c4d0 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead6022740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558a9d6dd1b0 con 0x558a9d77ce80 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7feac27fc640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7feab80a3d20 0x7feab80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 0x558a9d787e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead19d7640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7feab80a3d20 0x7feab80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 0x558a9d787e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60172/0 (socket says 172.21.15.113:60172) 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 -- 172.21.15.113:0/328878928 learned_addr learned my addr 172.21.15.113:0/328878928 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead19d7640 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d77c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7feac27fc640 1 -- 172.21.15.113:0/1276073042 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7feac40f2fe0 con 0x7feacc1021e0 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead11d6640 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d785430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 -- 172.21.15.113:0/328878928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 msgr2=0x558a9d785430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d785430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 -- 172.21.15.113:0/328878928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 msgr2=0x558a9d77c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d77c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead09d5640 1 -- 172.21.15.113:0/328878928 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9d6701b0 con 0x558a9d785a00 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead11d6640 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d785430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.632+0000 7fead19d7640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7feab80a3d20 0x7feab80a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feac8005d80 tx=0x7feac800ca70 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead09d5640 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 0x558a9d787e50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551321e261e010d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7feab77fe640 1 -- 172.21.15.113:0/328878928 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc016070 con 0x558a9d785a00 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7feab77fe640 1 -- 172.21.15.113:0/328878928 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feabc0054f0 con 0x558a9d785a00 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7feab77fe640 1 -- 172.21.15.113:0/328878928 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004da0 con 0x558a9d785a00 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 -- 172.21.15.113:0/328878928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 msgr2=0x558a9d787e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 0x558a9d787e50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc00e6c0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 -- 172.21.15.113:0/328878928 shutdown_connections 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d785a00 0x558a9d787e50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d785430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 --2- 172.21.15.113:0/328878928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d77c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 -- 172.21.15.113:0/328878928 >> 172.21.15.113:0/328878928 conn(0x558a9d776ba0 msgr2=0x558a9d776f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 -- 172.21.15.113:0/328878928 shutdown_connections 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.633+0000 7fead6022740 1 -- 172.21.15.113:0/328878928 wait complete. 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 -- start start 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 Processor -- start 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d7852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77ce80 0x558a9d782650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d782bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x558a9d788970 con 0x558a9d77c4d0 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a9d69a500 con 0x558a9d77ce80 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558a9d788af0 con 0x558a9d785a00 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead6022740 10 client.0 _ll_register_callbacks cb 0x558a9d668b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:29.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead11d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d7852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead11d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d7852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60184/0 (socket says 172.21.15.113:60184) 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead11d6640 1 -- 172.21.15.113:0/3814194293 learned_addr learned my addr 172.21.15.113:0/3814194293 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead09d5640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77ce80 0x558a9d782650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.634+0000 7fead19d7640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d782bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead11d6640 1 -- 172.21.15.113:0/3814194293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 msgr2=0x558a9d782bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead11d6640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d782bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead11d6640 1 -- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77ce80 msgr2=0x558a9d782650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead11d6640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77ce80 0x558a9d782650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead11d6640 1 -- 172.21.15.113:0/3814194293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9d3f33f0 con 0x558a9d77c4d0 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead19d7640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d782bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead11d6640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d7852c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7feac4005cc0 tx=0x7feac402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f484cde923c58d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7feab4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:29.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7feab4ff9640 1 -- 172.21.15.113:0/3814194293 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40beef0 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7feab4ff9640 1 -- 172.21.15.113:0/3814194293 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7feac4009890 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc0054f0 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7feab4ff9640 1 -- 172.21.15.113:0/3814194293 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7feac40c32a0 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 10 client.17541 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558a9d4bd7d0 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7feab4ff9640 1 -- 172.21.15.113:0/3814194293 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feabc0054f0 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7feab4ff9640 1 -- 172.21.15.113:0/3814194293 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7feac403c030 con 0x558a9d77c4d0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 10 client.17541 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 10 client.17541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 20 client.17541 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d795b70 0x558a9d798000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 4 client.17541 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.635+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a9d680930 con 0x558a9d795b70 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.636+0000 7fead09d5640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d795b70 0x558a9d798000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.636+0000 7feab4ff9640 10 client.17541 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.636+0000 7fead09d5640 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d795b70 0x558a9d798000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3543ed4d284a23e server_cookie=2596b1077a26ac86 in_seq=0 out_seq=0 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7feab4ff9640 1 -- 172.21.15.113:0/3814194293 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a9d680930 con 0x558a9d795b70 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7feab4ff9640 10 client.17541 handle_command_reply: tid=0 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 client.17541 shutdown 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 msgr2=0x558a9d7852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d7852c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7feac4005cc0 tx=0x7feac402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 shutdown_connections 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d795b70 0x558a9d798000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d782bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77ce80 0x558a9d782650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7feab4ff9640 0 client.17541 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 --2- 172.21.15.113:0/3814194293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d7852c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 >> 172.21.15.113:0/3814194293 conn(0x558a9d776ba0 msgr2=0x558a9d7788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 shutdown_connections 2024-04-22T07:23:29.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 1 -- 172.21.15.113:0/3814194293 wait complete. 2024-04-22T07:23:29.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.637+0000 7fead6022740 20 client.17541 trim_cache size 0 max 16384 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.650+0000 7fead6022740 1 Processor -- start 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.650+0000 7fead6022740 1 -- start start 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead6022740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d791830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead6022740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 0x558a9d784d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead6022740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d7852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead6022740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1b0840 con 0x558a9d77ce80 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead6022740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1ae110 con 0x558a9d77c4d0 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead6022740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1abf80 con 0x558a9d785a00 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 0x558a9d784d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 0x558a9d784d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60190/0 (socket says 172.21.15.113:60190) 2024-04-22T07:23:29.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 -- 172.21.15.113:0/3027750093 learned_addr learned my addr 172.21.15.113:0/3027750093 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead19d7640 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d7852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 -- 172.21.15.113:0/3027750093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 msgr2=0x558a9d7852c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d7852c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 -- 172.21.15.113:0/3027750093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 msgr2=0x558a9d791830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d791830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 -- 172.21.15.113:0/3027750093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9d782f20 con 0x558a9d77ce80 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7fead09d5640 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 0x558a9d784d80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7feabc000c00 tx=0x7feabc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a39588eece6297b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7feab6ffd640 1 -- 172.21.15.113:0/3027750093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc016040 con 0x558a9d77ce80 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7feab6ffd640 1 -- 172.21.15.113:0/3027750093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feabc009b70 con 0x558a9d77ce80 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.651+0000 7feab6ffd640 1 -- 172.21.15.113:0/3027750093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc011760 con 0x558a9d77ce80 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 -- 172.21.15.113:0/3027750093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 msgr2=0x558a9d784d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 0x558a9d784d80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7feabc000c00 tx=0x7feabc009780 comp rx=0 tx=0).stop 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 -- 172.21.15.113:0/3027750093 shutdown_connections 2024-04-22T07:23:29.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d785a00 0x558a9d7852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77ce80 0x558a9d784d80 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 --2- 172.21.15.113:0/3027750093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d77c4d0 0x558a9d791830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 -- 172.21.15.113:0/3027750093 >> 172.21.15.113:0/3027750093 conn(0x558a9d776ba0 msgr2=0x558a9d77dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 -- 172.21.15.113:0/3027750093 shutdown_connections 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 -- 172.21.15.113:0/3027750093 wait complete. 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 -- start start 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.652+0000 7fead6022740 1 Processor -- start 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d77f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d77f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d785a00 0x558a9d77fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1ac5d0 con 0x558a9d77c4d0 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1ada40 con 0x558a9d785a00 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feacc1acca0 con 0x558a9d77ce80 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 10 client.0 _ll_register_callbacks cb 0x7feac40bb260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:29.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d77f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d77f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60192/0 (socket says 172.21.15.113:60192) 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 -- 172.21.15.113:0/2653534475 learned_addr learned my addr 172.21.15.113:0/2653534475 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead09d5640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d77f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 -- 172.21.15.113:0/2653534475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 msgr2=0x558a9d77f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d77f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 -- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d785a00 msgr2=0x558a9d77fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead19d7640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d785a00 0x558a9d77fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d785a00 0x558a9d77fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 -- 172.21.15.113:0/2653534475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9d7fb200 con 0x558a9d77c4d0 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead19d7640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d785a00 0x558a9d77fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead11d6640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d77f3e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7feac4009430 tx=0x7feac403e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8258f82b34b80ead server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:29.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7feab4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7feab4ff9640 1 -- 172.21.15.113:0/2653534475 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac40be850 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7feab4ff9640 1 -- 172.21.15.113:0/2653534475 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7feac40c76d0 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc009b70 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7feab4ff9640 1 -- 172.21.15.113:0/2653534475 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7feac40d8270 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 10 client.17547 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.653+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558a9d7fc4e0 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7feab4ff9640 1 -- 172.21.15.113:0/2653534475 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feabc009b70 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7feab4ff9640 1 -- 172.21.15.113:0/2653534475 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7feac40bf150 con 0x558a9d77c4d0 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead6022740 10 client.17547 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead6022740 10 client.17547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead6022740 20 client.17547 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead6022740 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d7fca80 0x558a9d7feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead6022740 4 client.17547 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:23:29.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a9d6eec10 con 0x558a9d7fca80 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead09d5640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d7fca80 0x558a9d7feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7feab4ff9640 10 client.17547 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.654+0000 7fead09d5640 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d7fca80 0x558a9d7feea0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75f7f1fffa83b540 server_cookie=75283b2af3e9dbd9 in_seq=0 out_seq=0 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7feab4ff9640 1 -- 172.21.15.113:0/2653534475 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a9d6eec10 con 0x558a9d7fca80 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7feab4ff9640 10 client.17547 handle_command_reply: tid=0 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 client.17547 shutdown 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 msgr2=0x558a9d77f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d77f3e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7feac4009430 tx=0x7feac403e620 comp rx=0 tx=0).stop 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 shutdown_connections 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x558a9d7fca80 0x558a9d7feea0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a9d785a00 0x558a9d77fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558a9d77ce80 0x558a9d77f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 --2- 172.21.15.113:0/2653534475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558a9d77c4d0 0x558a9d77f3e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7feab4ff9640 0 client.17547 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:23:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 >> 172.21.15.113:0/2653534475 conn(0x558a9d776ba0 msgr2=0x558a9d77d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 shutdown_connections 2024-04-22T07:23:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 1 -- 172.21.15.113:0/2653534475 wait complete. 2024-04-22T07:23:29.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead6022740 20 client.17547 trim_cache size 0 max 16384 2024-04-22T07:23:29.661 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7feab80a3d20 msgr2=0x7feab80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7feab80a3d20 0x7feab80a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feac8005d80 tx=0x7feac800ca70 comp rx=0 tx=0).stop 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 msgr2=0x7feacc19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc19e630 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7feac4005e30 tx=0x7feac4035d80 comp rx=0 tx=0).stop 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 shutdown_connections 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7feab80a3d20 0x7feab80a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feacc1986f0 0x7feacc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feacc1072b0 0x7feacc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 --2- 172.21.15.113:0/1276073042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feacc1021e0 0x7feacc19e630 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:29.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.655+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 >> 172.21.15.113:0/1276073042 conn(0x7feacc0fda60 msgr2=0x7feacc0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:29.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.656+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 shutdown_connections 2024-04-22T07:23:29.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:29.656+0000 7fead343a640 1 -- 172.21.15.113:0/1276073042 wait complete. 2024-04-22T07:23:30.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:30 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3443712405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:23:30.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:30 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3443712405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:23:30.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:30 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3443712405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:23:30.666 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:23:30.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:23:30.666 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:23:30.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 -- 172.21.15.113:0/2394639193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 msgr2=0x7effec10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 --2- 172.21.15.113:0/2394639193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec10e9b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7effe4009ca0 tx=0x7effe402c230 comp rx=0 tx=0).stop 2024-04-22T07:23:30.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 -- 172.21.15.113:0/2394639193 shutdown_connections 2024-04-22T07:23:30.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 --2- 172.21.15.113:0/2394639193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec10e9b0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 --2- 172.21.15.113:0/2394639193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effec103af0 0x7effec10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 --2- 172.21.15.113:0/2394639193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec1031d0 0x7effec1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 -- 172.21.15.113:0/2394639193 >> 172.21.15.113:0/2394639193 conn(0x7effec077f50 msgr2=0x7effec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 -- 172.21.15.113:0/2394639193 shutdown_connections 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.783+0000 7efff0f54640 1 -- 172.21.15.113:0/2394639193 wait complete. 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 Processor -- start 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 -- start start 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec1031d0 0x7effec19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effec103af0 0x7effec19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7effec112340 con 0x7effec10c510 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7effec1121c0 con 0x7effec103af0 2024-04-22T07:23:30.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7efff0f54640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7effec1124c0 con 0x7effec1031d0 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60204/0 (socket says 172.21.15.113:60204) 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 -- 172.21.15.113:0/931980907 learned_addr learned my addr 172.21.15.113:0/931980907 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effe3fff640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effec103af0 0x7effec19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effea59c640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec1031d0 0x7effec19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 -- 172.21.15.113:0/931980907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec1031d0 msgr2=0x7effec19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec1031d0 0x7effec19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 -- 172.21.15.113:0/931980907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effec103af0 msgr2=0x7effec19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effec103af0 0x7effec19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.784+0000 7effead9d640 1 -- 172.21.15.113:0/931980907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec198f00 con 0x7effec10c510 2024-04-22T07:23:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.785+0000 7effead9d640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec198640 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7effe402c840 tx=0x7effe4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.785+0000 7effe37fe640 1 -- 172.21.15.113:0/931980907 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4039040 con 0x7effec10c510 2024-04-22T07:23:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.785+0000 7effe37fe640 1 -- 172.21.15.113:0/931980907 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7effe4044070 con 0x7effec10c510 2024-04-22T07:23:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.785+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec199190 con 0x7effec10c510 2024-04-22T07:23:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.785+0000 7effe37fe640 1 -- 172.21.15.113:0/931980907 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4004570 con 0x7effec10c510 2024-04-22T07:23:30.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.785+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec1a5320 con 0x7effec10c510 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.787+0000 7effe37fe640 1 -- 172.21.15.113:0/931980907 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7effe4004710 con 0x7effec10c510 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.787+0000 7efff3c16740 1 Processor -- start 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 -- start start 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd8acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 0x55646dd93840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effe37fe640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7effd80a3d20 0x7effd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55646dceef00 con 0x55646dd8b290 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55646dc8c590 con 0x55646dd8a8e0 2024-04-22T07:23:30.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7efff3c16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55646dceaec0 con 0x55646dd93e10 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 0x55646dd93840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 0x55646dd93840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60214/0 (socket says 172.21.15.113:60214) 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 -- 172.21.15.113:0/3034713780 learned_addr learned my addr 172.21.15.113:0/3034713780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effead9d640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7effd80a3d20 0x7effd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effe3fff640 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd8acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effe37fe640 1 -- 172.21.15.113:0/931980907 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7effe40f30b0 con 0x7effec10c510 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effe3fff640 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 -- 172.21.15.113:0/3034713780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 msgr2=0x55646dd96260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd96260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 -- 172.21.15.113:0/3034713780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 msgr2=0x55646dd8acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd8acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effea59c640 1 -- 172.21.15.113:0/3034713780 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646dac97e0 con 0x55646dd8b290 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.788+0000 7effe3fff640 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7effead9d640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7effd80a3d20 0x7effd80a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7effe40becc0 tx=0x7effe4090a00 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7effea59c640 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 0x55646dd93840 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7effdc00b820 tx=0x7effdc00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5071af2f6d834b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7effe0ff9640 1 -- 172.21.15.113:0/3034713780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc00d950 con 0x55646dd8b290 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7effe0ff9640 1 -- 172.21.15.113:0/3034713780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7effdc019070 con 0x55646dd8b290 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7effe0ff9640 1 -- 172.21.15.113:0/3034713780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc0123e0 con 0x55646dd8b290 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7efff3c16740 1 -- 172.21.15.113:0/3034713780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 msgr2=0x55646dd93840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.789+0000 7efff3c16740 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 0x55646dd93840 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7effdc00b820 tx=0x7effdc00bce0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- 172.21.15.113:0/3034713780 shutdown_connections 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8b290 0x55646dd93840 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 --2- 172.21.15.113:0/3034713780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd8acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- 172.21.15.113:0/3034713780 >> 172.21.15.113:0/3034713780 conn(0x55646dd84fb0 msgr2=0x55646dd85390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- 172.21.15.113:0/3034713780 shutdown_connections 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- 172.21.15.113:0/3034713780 wait complete. 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- start start 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 Processor -- start 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd90dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd8b290 0x55646dd91300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 0x55646dd91840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55646dd96d80 con 0x55646dd93e10 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55646dca8580 con 0x55646dd8a8e0 2024-04-22T07:23:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55646dd96f00 con 0x55646dd8b290 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7efff3c16740 10 client.0 _ll_register_callbacks cb 0x55646dcae650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7effe3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 0x55646dd91840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7effe3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 0x55646dd91840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60222/0 (socket says 172.21.15.113:60222) 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.790+0000 7effe3fff640 1 -- 172.21.15.113:0/3804482402 learned_addr learned my addr 172.21.15.113:0/3804482402 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd90dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effea59c640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd8b290 0x55646dd91300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 -- 172.21.15.113:0/3804482402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd8b290 msgr2=0x55646dd91300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd8b290 0x55646dd91300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 -- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 msgr2=0x55646dd90dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd90dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 -- 172.21.15.113:0/3804482402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646db29f70 con 0x55646dd93e10 2024-04-22T07:23:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effe3fff640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 0x55646dd91840 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7effd4010fa0 tx=0x7effd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139d6f623b81efbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effd1ffb640 1 -- 172.21.15.113:0/3804482402 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4004020 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effd1ffb640 1 -- 172.21.15.113:0/3804482402 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7effd40041c0 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effdc019070 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effd1ffb640 1 -- 172.21.15.113:0/3804482402 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7effd40043c0 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7efff3c16740 10 client.17559 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55646d9c1170 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.791+0000 7effd1ffb640 1 -- 172.21.15.113:0/3804482402 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7effdc019070 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7effd1ffb640 1 -- 172.21.15.113:0/3804482402 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7effd401f6d0 con 0x55646dd93e10 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7efff3c16740 10 client.17559 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7efff3c16740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:30.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7efff3c16740 20 client.17559 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7efff3c16740 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646dda43c0 0x55646dda6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7efff3c16740 4 client.17559 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55646dcfc920 con 0x55646dda43c0 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7effea59c640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646dda43c0 0x55646dda6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7effea59c640 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646dda43c0 0x55646dda6820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3df581948be1c5 server_cookie=1436f229f5cdd487 in_seq=0 out_seq=0 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.792+0000 7effd1ffb640 10 client.17559 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7effd1ffb640 1 -- 172.21.15.113:0/3804482402 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55646dcfc920 con 0x55646dda43c0 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7effd1ffb640 10 client.17559 handle_command_reply: tid=0 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 client.17559 shutdown 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 msgr2=0x55646dd91840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 0x55646dd91840 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7effd4010fa0 tx=0x7effd4004730 comp rx=0 tx=0).stop 2024-04-22T07:23:30.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 shutdown_connections 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646dda43c0 0x55646dda6820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd93e10 0x55646dd91840 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd8b290 0x55646dd91300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 --2- 172.21.15.113:0/3804482402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8a8e0 0x55646dd90dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7effd1ffb640 0 client.17559 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 >> 172.21.15.113:0/3804482402 conn(0x55646dd84fb0 msgr2=0x55646dd86d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 shutdown_connections 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 1 -- 172.21.15.113:0/3804482402 wait complete. 2024-04-22T07:23:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.793+0000 7efff3c16740 20 client.17559 trim_cache size 0 max 16384 2024-04-22T07:23:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.810+0000 7efff3c16740 1 Processor -- start 2024-04-22T07:23:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.810+0000 7efff3c16740 1 -- start start 2024-04-22T07:23:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7efff3c16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7efff3c16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd9fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7efff3c16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7efff3c16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7effec1b08c0 con 0x55646dd8a8e0 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7efff3c16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7effec1ae190 con 0x55646dd8b290 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7efff3c16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7effec1ac000 con 0x55646dd93e10 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd934f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60234/0 (socket says 172.21.15.113:60234) 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 -- 172.21.15.113:0/1797197682 learned_addr learned my addr 172.21.15.113:0/1797197682 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 -- 172.21.15.113:0/1797197682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 msgr2=0x55646dd86b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 -- 172.21.15.113:0/1797197682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 msgr2=0x55646dd9fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd9fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 -- 172.21.15.113:0/1797197682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646dd847e0 con 0x55646dd8a8e0 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effe3fff640 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd934f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7effd400a6e0 tx=0x7effd400a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d876d096df18ff2d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effd3fff640 1 -- 172.21.15.113:0/1797197682 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd400a7f0 con 0x55646dd8a8e0 2024-04-22T07:23:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effd3fff640 1 -- 172.21.15.113:0/1797197682 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7effd401fce0 con 0x55646dd8a8e0 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.811+0000 7effd3fff640 1 -- 172.21.15.113:0/1797197682 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40047a0 con 0x55646dd8a8e0 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 -- 172.21.15.113:0/1797197682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 msgr2=0x55646dd934f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd934f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7effd400a6e0 tx=0x7effd400a350 comp rx=0 tx=0).stop 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 -- 172.21.15.113:0/1797197682 shutdown_connections 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646dd86b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd9fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 --2- 172.21.15.113:0/1797197682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd934f0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 -- 172.21.15.113:0/1797197682 >> 172.21.15.113:0/1797197682 conn(0x55646dd84fb0 msgr2=0x55646dd86400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 -- 172.21.15.113:0/1797197682 shutdown_connections 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 -- 172.21.15.113:0/1797197682 wait complete. 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 -- start start 2024-04-22T07:23:30.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.812+0000 7efff3c16740 1 Processor -- start 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd97eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646de09960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7effec1ac650 con 0x55646dd8a8e0 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7effec1adac0 con 0x55646dd8b290 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7effec1acd20 con 0x55646dd93e10 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 10 client.0 _ll_register_callbacks cb 0x7effd400ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd97eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd97eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60236/0 (socket says 172.21.15.113:60236) 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 -- 172.21.15.113:0/3721390913 learned_addr learned my addr 172.21.15.113:0/3721390913 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effea59c640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646de09960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 -- 172.21.15.113:0/3721390913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 msgr2=0x55646de09960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646de09960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 -- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 msgr2=0x55646dd983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 -- 172.21.15.113:0/3721390913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646dd98c80 con 0x55646dd8a8e0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effea59c640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effe3fff640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd97eb0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7effd4010c90 tx=0x7effd4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8567096c4d6b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effd1ffb640 1 -- 172.21.15.113:0/3721390913 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd401f400 con 0x55646dd8a8e0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effd1ffb640 1 -- 172.21.15.113:0/3721390913 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7effd401f5a0 con 0x55646dd8a8e0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effd401fce0 con 0x55646dd8a8e0 2024-04-22T07:23:30.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7effd1ffb640 1 -- 172.21.15.113:0/3721390913 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7effd401f7a0 con 0x55646dd8a8e0 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 10 client.17565 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.813+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55646de0afb0 con 0x55646dd8a8e0 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7effd1ffb640 1 -- 172.21.15.113:0/3721390913 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7effd401fce0 con 0x55646dd8a8e0 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7effd1ffb640 1 -- 172.21.15.113:0/3721390913 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7effd403b6e0 con 0x55646dd8a8e0 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7efff3c16740 10 client.17565 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7efff3c16740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7efff3c16740 20 client.17565 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7efff3c16740 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646de0b620 0x55646de0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7efff3c16740 4 client.17565 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55646dcf6790 con 0x55646de0b620 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7effea59c640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646de0b620 0x55646de0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7effea59c640 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646de0b620 0x55646de0da40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed863fca5b7bf6d server_cookie=68ef8b7d69a13ea9 in_seq=0 out_seq=0 2024-04-22T07:23:30.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.814+0000 7effd1ffb640 10 client.17565 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7effd1ffb640 1 -- 172.21.15.113:0/3721390913 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55646dcf6790 con 0x55646de0b620 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7effd1ffb640 10 client.17565 handle_command_reply: tid=0 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 client.17565 shutdown 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 msgr2=0x55646dd97eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd97eb0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7effd4010c90 tx=0x7effd4010c20 comp rx=0 tx=0).stop 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 shutdown_connections 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55646de0b620 0x55646de0da40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55646dd93e10 0x55646de09960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55646dd8b290 0x55646dd983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7effd1ffb640 0 client.17565 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 --2- 172.21.15.113:0/3721390913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55646dd8a8e0 0x55646dd97eb0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 >> 172.21.15.113:0/3721390913 conn(0x55646dd84fb0 msgr2=0x55646dd91610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 shutdown_connections 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 1 -- 172.21.15.113:0/3721390913 wait complete. 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff3c16740 20 client.17565 trim_cache size 0 max 16384 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "7c6270b8-b300-411a-ba3b-0a4f5b8ac7c6", 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:23:30.823 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:23:30.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7effd80a3d20 msgr2=0x7effd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7effd80a3d20 0x7effd80a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7effe40becc0 tx=0x7effe4090a00 comp rx=0 tx=0).stop 2024-04-22T07:23:30.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 msgr2=0x7effec198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec198640 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7effe402c840 tx=0x7effe4003c20 comp rx=0 tx=0).stop 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 shutdown_connections 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7effd80a3d20 0x7effd80a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effec10c510 0x7effec198640 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effec103af0 0x7effec19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 --2- 172.21.15.113:0/931980907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec1031d0 0x7effec19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.816+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 >> 172.21.15.113:0/931980907 conn(0x7effec077f50 msgr2=0x7effec0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.817+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 shutdown_connections 2024-04-22T07:23:30.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.817+0000 7efff0f54640 1 -- 172.21.15.113:0/931980907 wait complete. 2024-04-22T07:23:30.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:23:30.826 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:23:30.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 -- 172.21.15.113:0/3312480661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c1031c0 msgr2=0x7fc34c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3312480661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c1031c0 0x7fc34c1035a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009900 tx=0x7fc33c02bd60 comp rx=0 tx=0).stop 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 -- 172.21.15.113:0/3312480661 shutdown_connections 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3312480661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc34c10c500 0x7fc34c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3312480661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c103ae0 0x7fc34c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3312480661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c1031c0 0x7fc34c1035a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 -- 172.21.15.113:0/3312480661 >> 172.21.15.113:0/3312480661 conn(0x7fc34c078000 msgr2=0x7fc34c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 -- 172.21.15.113:0/3312480661 shutdown_connections 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.927+0000 7fc3520a5640 1 -- 172.21.15.113:0/3312480661 wait complete. 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 Processor -- start 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 -- start start 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c1031c0 0x7fc34c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc34c103ae0 0x7fc34c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 0x7fc34c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c112320 con 0x7fc34c10c500 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1121a0 con 0x7fc34c1031c0 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc3520a5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1124a0 con 0x7fc34c103ae0 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc34bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 0x7fc34c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.928+0000 7fc34bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 0x7fc34c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60254/0 (socket says 172.21.15.113:60254) 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 -- 172.21.15.113:0/3930935279 learned_addr learned my addr 172.21.15.113:0/3930935279 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc342ffd640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc34c103ae0 0x7fc34c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c1031c0 0x7fc34c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 -- 172.21.15.113:0/3930935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc34c103ae0 msgr2=0x7fc34c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc34c103ae0 0x7fc34c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 -- 172.21.15.113:0/3930935279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c1031c0 msgr2=0x7fc34c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c1031c0 0x7fc34c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 -- 172.21.15.113:0/3930935279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c198da0 con 0x7fc34c10c500 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c1031c0 0x7fc34c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc34bfff640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 0x7fc34c198620 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc34400eaa0 tx=0x7fc34400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc3497fa640 1 -- 172.21.15.113:0/3930935279 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34400cdb0 con 0x7fc34c10c500 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc3497fa640 1 -- 172.21.15.113:0/3930935279 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc344005020 con 0x7fc34c10c500 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c199090 con 0x7fc34c10c500 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc3497fa640 1 -- 172.21.15.113:0/3930935279 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440116d0 con 0x7fc34c10c500 2024-04-22T07:23:30.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.929+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c1a52e0 con 0x7fc34c10c500 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.931+0000 7fc3497fa640 1 -- 172.21.15.113:0/3930935279 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fc3440051d0 con 0x7fc34c10c500 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.931+0000 7fc354d74740 1 Processor -- start 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 -- start start 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc3497fa640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc3380a3d20 0x7fc3380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070abe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5643070051e0 con 0x5643070a0580 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564306fa2390 con 0x5643070a0ea0 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc354d74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5643070011a0 con 0x5643070a9a80 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc3380a3d20 0x7fc3380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc3497fa640 1 -- 172.21.15.113:0/3930935279 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fc3440d0bd0 con 0x7fc34c10c500 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60260/0 (socket says 172.21.15.113:60260) 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 -- 172.21.15.113:0/165133903 learned_addr learned my addr 172.21.15.113:0/165133903 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc34bfff640 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 -- 172.21.15.113:0/165133903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 msgr2=0x5643070abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 -- 172.21.15.113:0/165133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 msgr2=0x5643070a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 -- 172.21.15.113:0/165133903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564306de17d0 con 0x5643070a0580 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc3380a3d20 0x7fc3380a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc33c006010 tx=0x7fc33c005a20 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.932+0000 7fc342ffd640 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a0960 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc33400bfd0 tx=0x7fc33400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca20182cd9e3dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc341ffb640 1 -- 172.21.15.113:0/165133903 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334004030 con 0x5643070a0580 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc341ffb640 1 -- 172.21.15.113:0/165133903 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc3340054f0 con 0x5643070a0580 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc341ffb640 1 -- 172.21.15.113:0/165133903 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334004bc0 con 0x5643070a0580 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 -- 172.21.15.113:0/165133903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 msgr2=0x5643070a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a0960 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc33400bfd0 tx=0x7fc33400b820 comp rx=0 tx=0).stop 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 -- 172.21.15.113:0/165133903 shutdown_connections 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 --2- 172.21.15.113:0/165133903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a0960 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 -- 172.21.15.113:0/165133903 >> 172.21.15.113:0/165133903 conn(0x56430709acb0 msgr2=0x56430709b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 -- 172.21.15.113:0/165133903 shutdown_connections 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.933+0000 7fc354d74740 1 -- 172.21.15.113:0/165133903 wait complete. 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 -- start start 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 Processor -- start 2024-04-22T07:23:30.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0580 0x5643070a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a0ea0 0x5643070a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 0x5643070a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5643070ac870 con 0x5643070a9a80 2024-04-22T07:23:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564306fbe2f0 con 0x5643070a0580 2024-04-22T07:23:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5643070ac9f0 con 0x5643070a0ea0 2024-04-22T07:23:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc354d74740 10 client.0 _ll_register_callbacks cb 0x564306fcb580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 0x5643070a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 0x5643070a7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60276/0 (socket says 172.21.15.113:60276) 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 -- 172.21.15.113:0/3061420788 learned_addr learned my addr 172.21.15.113:0/3061420788 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0580 0x5643070a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc342ffd640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a0ea0 0x5643070a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 -- 172.21.15.113:0/3061420788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a0ea0 msgr2=0x5643070a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a0ea0 0x5643070a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 -- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0580 msgr2=0x5643070a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0580 0x5643070a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc34bfff640 1 -- 172.21.15.113:0/3061420788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564306d173f0 con 0x5643070a9a80 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.934+0000 7fc342ffd640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a0ea0 0x5643070a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc34bfff640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 0x5643070a7560 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc3440ce780 tx=0x7fc34401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591d3b7db7fb98f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc32affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc32affd640 1 -- 172.21.15.113:0/3061420788 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34409c9d0 con 0x5643070a9a80 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc32affd640 1 -- 172.21.15.113:0/3061420788 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fc3440b71c0 con 0x5643070a9a80 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3340054f0 con 0x5643070a9a80 2024-04-22T07:23:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc32affd640 1 -- 172.21.15.113:0/3061420788 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc3440a0d80 con 0x5643070a9a80 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 10 client.17577 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564306cd70f0 con 0x5643070a9a80 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc32affd640 1 -- 172.21.15.113:0/3061420788 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc3340054f0 con 0x5643070a9a80 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc32affd640 1 -- 172.21.15.113:0/3061420788 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fc34400f040 con 0x5643070a9a80 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 10 client.17577 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 20 client.17577 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5643070b9b20 0x5643070bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 4 client.17577 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.935+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564307012c00 con 0x5643070b9b20 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.936+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5643070b9b20 0x5643070bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.936+0000 7fc32affd640 10 client.17577 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.936+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5643070b9b20 0x5643070bbf10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cebf7d2fc07f17e server_cookie=31c55b3f60be9535 in_seq=0 out_seq=0 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.936+0000 7fc32affd640 1 -- 172.21.15.113:0/3061420788 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564307012c00 con 0x5643070b9b20 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.936+0000 7fc32affd640 10 client.17577 handle_command_reply: tid=0 2024-04-22T07:23:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 client.17577 shutdown 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 msgr2=0x5643070a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 0x5643070a7560 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc3440ce780 tx=0x7fc34401f3f0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 shutdown_connections 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5643070b9b20 0x5643070bbf10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a9a80 0x5643070a7560 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc32affd640 0 client.17577 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a0ea0 0x5643070a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 --2- 172.21.15.113:0/3061420788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0580 0x5643070a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 >> 172.21.15.113:0/3061420788 conn(0x56430709acb0 msgr2=0x56430709ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 shutdown_connections 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 1 -- 172.21.15.113:0/3061420788 wait complete. 2024-04-22T07:23:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.937+0000 7fc354d74740 20 client.17577 trim_cache size 0 max 16384 2024-04-22T07:23:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 Processor -- start 2024-04-22T07:23:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 -- start start 2024-04-22T07:23:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1b07d0 con 0x5643070a0580 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ae0a0 con 0x5643070a0ea0 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc354d74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1abf10 con 0x5643070a9a80 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60282/0 (socket says 172.21.15.113:60282) 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1109837325 learned_addr learned my addr 172.21.15.113:0/1109837325 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1109837325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 msgr2=0x5643070b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.954+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1109837325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 msgr2=0x5643070a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1109837325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643070bc280 con 0x5643070a0580 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a9210 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc33c036040 tx=0x7fc33c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53abd20f1c65c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc3417fa640 1 -- 172.21.15.113:0/1109837325 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c046040 con 0x5643070a0580 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc3417fa640 1 -- 172.21.15.113:0/1109837325 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc33c056070 con 0x5643070a0580 2024-04-22T07:23:30.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc3417fa640 1 -- 172.21.15.113:0/1109837325 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c004920 con 0x5643070a0580 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 -- 172.21.15.113:0/1109837325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 msgr2=0x5643070a9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a9210 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc33c036040 tx=0x7fc33c03d000 comp rx=0 tx=0).stop 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 -- 172.21.15.113:0/1109837325 shutdown_connections 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 --2- 172.21.15.113:0/1109837325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a9210 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 -- 172.21.15.113:0/1109837325 >> 172.21.15.113:0/1109837325 conn(0x56430709acb0 msgr2=0x56430709c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 -- 172.21.15.113:0/1109837325 shutdown_connections 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.955+0000 7fc354d74740 1 -- 172.21.15.113:0/1109837325 wait complete. 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 -- start start 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 Processor -- start 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ac560 con 0x5643070a0580 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ad9d0 con 0x5643070a0ea0 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1acc30 con 0x5643070a9a80 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc354d74740 10 client.0 _ll_register_callbacks cb 0x7fc344098c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60288/0 (socket says 172.21.15.113:60288) 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1184610687 learned_addr learned my addr 172.21.15.113:0/1184610687 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc342ffd640 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1184610687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 msgr2=0x5643070a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 msgr2=0x5643070a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.956+0000 7fc34b7fe640 1 -- 172.21.15.113:0/1184610687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56430711f310 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc34b7fe640 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a3a50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc33c02c2a0 tx=0x7fc33c0516f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d8cc000911b369 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc32affd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc32affd640 1 -- 172.21.15.113:0/1184610687 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c046040 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc32affd640 1 -- 172.21.15.113:0/1184610687 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fc33c0512a0 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc33c056070 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc32affd640 1 -- 172.21.15.113:0/1184610687 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc33c0514a0 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 10 client.17583 fetch_fsmap learned FSMap version 52 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564307120600 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc32affd640 1 -- 172.21.15.113:0/1184610687 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc33c056070 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc32affd640 1 -- 172.21.15.113:0/1184610687 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fc33c040810 con 0x5643070a0580 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 10 client.17583 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 20 client.17583 populate_metadata read hostname 'smithi113' 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564307120d40 0x564307123160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:23:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 4 client.17583 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.957+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56430700ca70 con 0x564307120d40 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc342ffd640 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564307120d40 0x564307123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc32affd640 10 client.17583 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc342ffd640 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564307120d40 0x564307123160 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9bb92fe0c74394f server_cookie=da509f745cea259a in_seq=0 out_seq=0 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc32affd640 1 -- 172.21.15.113:0/1184610687 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56430700ca70 con 0x564307120d40 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc32affd640 10 client.17583 handle_command_reply: tid=0 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 client.17583 shutdown 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 msgr2=0x5643070a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a3a50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc33c02c2a0 tx=0x7fc33c0516f0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 shutdown_connections 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564307120d40 0x564307123160 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643070a9a80 0x5643070a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc32affd640 0 client.17583 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:23:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643070a0ea0 0x5643070a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 --2- 172.21.15.113:0/1184610687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643070a0580 0x5643070a3a50 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 >> 172.21.15.113:0/1184610687 conn(0x56430709acb0 msgr2=0x56430709c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 shutdown_connections 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.958+0000 7fc354d74740 1 -- 172.21.15.113:0/1184610687 wait complete. 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc354d74740 20 client.17583 trim_cache size 0 max 16384 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "7c6270b8-b300-411a-ba3b-0a4f5b8ac7c6": { 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:23:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "7c6270b8-b300-411a-ba3b-0a4f5b8ac7c6", 2024-04-22T07:23:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:23:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:23:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:23:30.965 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc3380a3d20 msgr2=0x7fc3380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc3380a3d20 0x7fc3380a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc33c006010 tx=0x7fc33c005a20 comp rx=0 tx=0).stop 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 msgr2=0x7fc34c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 0x7fc34c198620 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc34400eaa0 tx=0x7fc34400ef60 comp rx=0 tx=0).stop 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 shutdown_connections 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fc3380a3d20 0x7fc3380a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc34c10c500 0x7fc34c198620 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc34c103ae0 0x7fc34c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 --2- 172.21.15.113:0/3930935279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc34c1031c0 0x7fc34c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 >> 172.21.15.113:0/3930935279 conn(0x7fc34c078000 msgr2=0x7fc34c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 shutdown_connections 2024-04-22T07:23:30.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:23:30.959+0000 7fc3520a5640 1 -- 172.21.15.113:0/3930935279 wait complete. 2024-04-22T07:23:30.968 INFO:tasks.cephfs.filesystem:scrub status for tag:7c6270b8-b300-411a-ba3b-0a4f5b8ac7c6 - {'path': '/', 'tag': '7c6270b8-b300-411a-ba3b-0a4f5b8ac7c6', 'options': 'recursive,force'} 2024-04-22T07:23:31.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:31 smithi113 ceph-mon[27933]: pgmap v273: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:23:31.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:31 smithi133 ceph-mon[31198]: pgmap v273: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:23:31.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:31 smithi154 ceph-mon[31862]: pgmap v273: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 6 op/s 2024-04-22T07:23:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:32 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:23:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:32 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:32 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:23:32.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:32 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:32.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:32 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:23:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:32 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:32 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:23:32.566 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:32 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:32 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:23:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:32 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:32 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:23:32.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:32 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:23:33.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:33 smithi133 ceph-mon[31198]: pgmap v274: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:23:33.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:33 smithi154 ceph-mon[31862]: pgmap v274: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:23:33.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:33 smithi113 ceph-mon[27933]: pgmap v274: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:23:35.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:35 smithi133 ceph-mon[31198]: pgmap v275: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:35.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:35 smithi154 ceph-mon[31862]: pgmap v275: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:35.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:35 smithi113 ceph-mon[27933]: pgmap v275: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:36.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:36 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:23:36.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:36 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:23:36.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:36 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:23:37.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:37 smithi133 ceph-mon[31198]: pgmap v276: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:37.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:37 smithi154 ceph-mon[31862]: pgmap v276: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:37.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:37 smithi113 ceph-mon[27933]: pgmap v276: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:39.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:39 smithi133 ceph-mon[31198]: pgmap v277: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:39.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:39 smithi154 ceph-mon[31862]: pgmap v277: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:39.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:39 smithi113 ceph-mon[27933]: pgmap v277: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:41.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:41 smithi133 ceph-mon[31198]: pgmap v278: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:41.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:41 smithi154 ceph-mon[31862]: pgmap v278: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:41.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:41 smithi113 ceph-mon[27933]: pgmap v278: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:43.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:43 smithi133 ceph-mon[31198]: pgmap v279: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:43.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:43 smithi154 ceph-mon[31862]: pgmap v279: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:43.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:43 smithi113 ceph-mon[27933]: pgmap v279: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:45.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:45 smithi133 ceph-mon[31198]: pgmap v280: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:45.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:45 smithi154 ceph-mon[31862]: pgmap v280: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:45.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:45 smithi113 ceph-mon[27933]: pgmap v280: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:23:47.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:47 smithi133 ceph-mon[31198]: pgmap v281: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:47.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:47 smithi154 ceph-mon[31862]: pgmap v281: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:47.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:47 smithi113 ceph-mon[27933]: pgmap v281: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:23:49.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:49 smithi133 ceph-mon[31198]: pgmap v282: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:49.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:49 smithi154 ceph-mon[31862]: pgmap v282: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:49.583 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:49 smithi113 ceph-mon[27933]: pgmap v282: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:51.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:51 smithi133 ceph-mon[31198]: pgmap v283: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:51.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:51 smithi154 ceph-mon[31862]: pgmap v283: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:51.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:51 smithi113 ceph-mon[27933]: pgmap v283: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:53.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:53 smithi133 ceph-mon[31198]: pgmap v284: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:23:53.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:53 smithi154 ceph-mon[31862]: pgmap v284: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:23:53.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:53 smithi113 ceph-mon[27933]: pgmap v284: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:23:55.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:55 smithi133 ceph-mon[31198]: pgmap v285: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:55.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:55 smithi154 ceph-mon[31862]: pgmap v285: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:55.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:55 smithi113 ceph-mon[27933]: pgmap v285: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:57.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:57 smithi133 ceph-mon[31198]: pgmap v286: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:23:57.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:57 smithi154 ceph-mon[31862]: pgmap v286: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:23:57.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:57 smithi113 ceph-mon[27933]: pgmap v286: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:23:59.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:23:59 smithi133 ceph-mon[31198]: pgmap v287: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:59.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:23:59 smithi154 ceph-mon[31862]: pgmap v287: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:23:59.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:23:59 smithi113 ceph-mon[27933]: pgmap v287: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:00.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:24:00.970 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:24:01.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.084+0000 7fa4d07f4640 1 -- 172.21.15.113:0/397586077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8102b50 msgr2=0x7fa4c8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.084+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/397586077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8102b50 0x7fa4c8106b90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009900 tx=0x7fa4b802bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.084+0000 7fa4d07f4640 1 -- 172.21.15.113:0/397586077 shutdown_connections 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.084+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/397586077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c8107270 0x7fa4c810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.084+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/397586077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8102b50 0x7fa4c8106b90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.084+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/397586077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c81021a0 0x7fa4c8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- 172.21.15.113:0/397586077 >> 172.21.15.113:0/397586077 conn(0x7fa4c80793e0 msgr2=0x7fa4c80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- 172.21.15.113:0/397586077 shutdown_connections 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- 172.21.15.113:0/397586077 wait complete. 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 Processor -- start 2024-04-22T07:24:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- start start 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c81021a0 0x7fa4c8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c8107270 0x7fa4c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 0x7fa4c81a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81122f0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8112170 con 0x7fa4c81021a0 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4d07f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8112470 con 0x7fa4c8107270 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.085+0000 7fa4ced91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 0x7fa4c81a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 0x7fa4c81a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42556/0 (socket says 172.21.15.113:42556) 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 -- 172.21.15.113:0/645605840 learned_addr learned my addr 172.21.15.113:0/645605840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ce590640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c81021a0 0x7fa4c8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c8107270 0x7fa4c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 -- 172.21.15.113:0/645605840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c8107270 msgr2=0x7fa4c8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c8107270 0x7fa4c8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 -- 172.21.15.113:0/645605840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c81021a0 msgr2=0x7fa4c8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c81021a0 0x7fa4c8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 -- 172.21.15.113:0/645605840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c81a7100 con 0x7fa4c8074cd0 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4ced91640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 0x7fa4c81a6a00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000eaa0 tx=0x7fa4c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4bf7fe640 1 -- 172.21.15.113:0/645605840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000cdb0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c81a73f0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4bf7fe640 1 -- 172.21.15.113:0/645605840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4c0005020 con 0x7fa4c8074cd0 2024-04-22T07:24:01.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4bf7fe640 1 -- 172.21.15.113:0/645605840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00116d0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.086+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c81a78c0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.088+0000 7fa4bf7fe640 1 -- 172.21.15.113:0/645605840 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa4c00051d0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 Processor -- start 2024-04-22T07:24:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 -- start start 2024-04-22T07:24:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4bf7fe640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d50 0x7fa4b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x561126334960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 0x56112633fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5611262991e0 con 0x56112633da80 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561126236390 con 0x561126334ea0 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5611262951a0 con 0x561126334580 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d50 0x7fa4b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4bf7fe640 1 -- 172.21.15.113:0/645605840 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa4c00d0ab0 con 0x7fa4c8074cd0 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4ced91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 0x56112633fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4ced91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 0x56112633fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42568/0 (socket says 172.21.15.113:42568) 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4ced91640 1 -- 172.21.15.113:0/1488029851 learned_addr learned my addr 172.21.15.113:0/1488029851 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.089+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ce590640 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x561126334960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ced91640 1 -- 172.21.15.113:0/1488029851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 msgr2=0x561126334960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ced91640 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x561126334960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ced91640 1 -- 172.21.15.113:0/1488029851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 msgr2=0x56112633d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ced91640 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ced91640 1 -- 172.21.15.113:0/1488029851 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611260757d0 con 0x56112633da80 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d50 0x7fa4b40a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005ab0 tx=0x7fa4b802ca30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4ced91640 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 0x56112633fe70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00ce660 tx=0x7fa4c0060370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d27ac891af7e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4bcff9640 1 -- 172.21.15.113:0/1488029851 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c001f070 con 0x56112633da80 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4bcff9640 1 -- 172.21.15.113:0/1488029851 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4c00a5520 con 0x56112633da80 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.090+0000 7fa4bcff9640 1 -- 172.21.15.113:0/1488029851 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00986b0 con 0x56112633da80 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- 172.21.15.113:0/1488029851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 msgr2=0x56112633fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 0x56112633fe70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00ce660 tx=0x7fa4c0060370 comp rx=0 tx=0).stop 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- 172.21.15.113:0/1488029851 shutdown_connections 2024-04-22T07:24:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56112633da80 0x56112633fe70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/1488029851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x561126334960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- 172.21.15.113:0/1488029851 >> 172.21.15.113:0/1488029851 conn(0x56112632ecb0 msgr2=0x56112632f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- 172.21.15.113:0/1488029851 shutdown_connections 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- 172.21.15.113:0/1488029851 wait complete. 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- start start 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 Processor -- start 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x56112633aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x56112633b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x561126340870 con 0x561126334580 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5611262522f0 con 0x561126334ea0 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5611263409f0 con 0x56112633da80 2024-04-22T07:24:01.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.091+0000 7fa4d33dd740 10 client.0 _ll_register_callbacks cb 0x56112625f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x56112633aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x56112633aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42576/0 (socket says 172.21.15.113:42576) 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 -- 172.21.15.113:0/3150466645 learned_addr learned my addr 172.21.15.113:0/3150466645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ced91640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x56112633b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 -- 172.21.15.113:0/3150466645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 msgr2=0x56112633b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x56112633b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 -- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 msgr2=0x56112633aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 -- 172.21.15.113:0/3150466645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561125fab3f0 con 0x561126334580 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4ce590640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x56112633aab0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b950 tx=0x7fa4c400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff1cdd8983e0bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/3150466645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4003ea0 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/3150466645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa4c4011440 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c00a5520 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/3150466645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa4c401b640 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4d33dd740 10 client.17595 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561125f6b0f0 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.092+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/3150466645 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4c00a5520 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/3150466645 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa4c4019020 con 0x561126334580 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4d33dd740 10 client.17595 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4d33dd740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4d33dd740 20 client.17595 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56112634dad0 0x56112634ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4d33dd740 4 client.17595 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611262a6c00 con 0x56112634dad0 2024-04-22T07:24:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56112634dad0 0x56112634ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56112634dad0 0x56112634ff40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f21be9b444c13a1 server_cookie=6e4d0da8d045273d in_seq=0 out_seq=0 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.093+0000 7fa4b1ffb640 10 client.17595 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/3150466645 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611262a6c00 con 0x56112634dad0 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4b1ffb640 10 client.17595 handle_command_reply: tid=0 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4d33dd740 1 client.17595 shutdown 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 msgr2=0x56112633aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x56112633aab0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b950 tx=0x7fa4c400be10 comp rx=0 tx=0).stop 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 shutdown_connections 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.094+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56112634dad0 0x56112634ff40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x56112633b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x56112633aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4b1ffb640 0 client.17595 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/3150466645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x56112633aab0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 >> 172.21.15.113:0/3150466645 conn(0x56112632ecb0 msgr2=0x561126330a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 shutdown_connections 2024-04-22T07:24:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 1 -- 172.21.15.113:0/3150466645 wait complete. 2024-04-22T07:24:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.095+0000 7fa4d33dd740 20 client.17595 trim_cache size 0 max 16384 2024-04-22T07:24:01.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 Processor -- start 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 -- start start 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x56112633d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 0x5611263399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56112633da80 0x561126349730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81b2ae0 con 0x561126334ea0 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81b03b0 con 0x56112633da80 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81ae220 con 0x561126334580 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 0x5611263399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 0x5611263399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42580/0 (socket says 172.21.15.113:42580) 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 -- 172.21.15.113:0/2347445302 learned_addr learned my addr 172.21.15.113:0/2347445302 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4ced91640 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56112633da80 0x561126349730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4ce590640 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x56112633d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 -- 172.21.15.113:0/2347445302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 msgr2=0x56112633d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x56112633d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 -- 172.21.15.113:0/2347445302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56112633da80 msgr2=0x561126349730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56112633da80 0x561126349730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4cdd8f640 1 -- 172.21.15.113:0/2347445302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112632f880 con 0x561126334ea0 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.112+0000 7fa4ced91640 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56112633da80 0x561126349730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 0x5611263399f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005c40 tx=0x7fa4b8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa70f9be34b2f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4b3fff640 1 -- 172.21.15.113:0/2347445302 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8054070 con 0x561126334ea0 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4b3fff640 1 -- 172.21.15.113:0/2347445302 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4b8004380 con 0x561126334ea0 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4b3fff640 1 -- 172.21.15.113:0/2347445302 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8004b50 con 0x561126334ea0 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 -- 172.21.15.113:0/2347445302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 msgr2=0x5611263399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 0x5611263399f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005c40 tx=0x7fa4b8045000 comp rx=0 tx=0).stop 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 -- 172.21.15.113:0/2347445302 shutdown_connections 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56112633da80 0x561126349730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334ea0 0x5611263399f0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/2347445302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561126334580 0x56112633d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 -- 172.21.15.113:0/2347445302 >> 172.21.15.113:0/2347445302 conn(0x56112632ecb0 msgr2=0x561126330990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 -- 172.21.15.113:0/2347445302 shutdown_connections 2024-04-22T07:24:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.113+0000 7fa4d33dd740 1 -- 172.21.15.113:0/2347445302 wait complete. 2024-04-22T07:24:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 -- start start 2024-04-22T07:24:01.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 Processor -- start 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x561126341d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x561126342260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x5611263b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81ae870 con 0x561126334580 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81afce0 con 0x561126334ea0 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c81aef40 con 0x56112633da80 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4d33dd740 10 client.0 _ll_register_callbacks cb 0x7fa4c4004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x561126341d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x561126341d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42586/0 (socket says 172.21.15.113:42586) 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 -- 172.21.15.113:0/4132646666 learned_addr learned my addr 172.21.15.113:0/4132646666 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x561126342260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ced91640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x5611263b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 -- 172.21.15.113:0/4132646666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 msgr2=0x5611263b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x5611263b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 -- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 msgr2=0x561126342260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x561126342260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.114+0000 7fa4ce590640 1 -- 172.21.15.113:0/4132646666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611263b35c0 con 0x561126334580 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4ce590640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x561126341d20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b950 tx=0x7fa4c4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e7ea5b41f74b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/4132646666 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4003ea0 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/4132646666 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa4c401b9f0 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b8004380 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/4132646666 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa4c4037050 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 10 client.17601 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5611263b4890 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/4132646666 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4b8004380 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/4132646666 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa4c4032030 con 0x561126334580 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 10 client.17601 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 20 client.17601 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5611263b5000 0x5611263b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 4 client.17601 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:24:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5611262a0a70 con 0x5611263b5000 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.115+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5611263b5000 0x5611263b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4b1ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4cdd8f640 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5611263b5000 0x5611263b73f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd96f95c8f428224 server_cookie=fe7dd074720db65b in_seq=0 out_seq=0 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4b1ffb640 1 -- 172.21.15.113:0/4132646666 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5611262a0a70 con 0x5611263b5000 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4b1ffb640 10 client.17601 handle_command_reply: tid=0 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 client.17601 shutdown 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 msgr2=0x561126341d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x561126341d20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b950 tx=0x7fa4c4005970 comp rx=0 tx=0).stop 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 shutdown_connections 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5611263b5000 0x5611263b73f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56112633da80 0x5611263b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4b1ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561126334ea0 0x561126342260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 --2- 172.21.15.113:0/4132646666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561126334580 0x561126341d20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 >> 172.21.15.113:0/4132646666 conn(0x56112632ecb0 msgr2=0x56112632fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 shutdown_connections 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.116+0000 7fa4d33dd740 1 -- 172.21.15.113:0/4132646666 wait complete. 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d33dd740 20 client.17601 trim_cache size 0 max 16384 2024-04-22T07:24:01.122 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:24:01.123 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:24:01.123 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:24:01.123 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:24:01.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d50 msgr2=0x7fa4b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d50 0x7fa4b40a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005ab0 tx=0x7fa4b802ca30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 msgr2=0x7fa4c81a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 0x7fa4c81a6a00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000eaa0 tx=0x7fa4c000ef60 comp rx=0 tx=0).stop 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 shutdown_connections 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa4b40a3d50 0x7fa4b40a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4c8074cd0 0x7fa4c81a6a00 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4c8107270 0x7fa4c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 --2- 172.21.15.113:0/645605840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4c81021a0 0x7fa4c8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 >> 172.21.15.113:0/645605840 conn(0x7fa4c80793e0 msgr2=0x7fa4c80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 shutdown_connections 2024-04-22T07:24:01.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.117+0000 7fa4d07f4640 1 -- 172.21.15.113:0/645605840 wait complete. 2024-04-22T07:24:01.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:24:01.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:24:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/166520202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 msgr2=0x7f1ac0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/166520202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac0101490 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009900 tx=0x7f1abc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/166520202 shutdown_connections 2024-04-22T07:24:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/166520202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac0106180 0x7f1ac010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/166520202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac0101a60 0x7f1ac0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/166520202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac0101490 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/166520202 >> 172.21.15.113:0/166520202 conn(0x7f1ac00771d0 msgr2=0x7f1ac00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/166520202 shutdown_connections 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.226+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/166520202 wait complete. 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 Processor -- start 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 -- start start 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac0106180 0x7f1ac0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac019a120 0x7f1ac01a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0111220 con 0x7f1ac01010b0 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01110a0 con 0x7f1ac0106180 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac7d7e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac01113a0 con 0x7f1ac019a120 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac01996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42596/0 (socket says 172.21.15.113:42596) 2024-04-22T07:24:01.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 -- 172.21.15.113:0/143628625 learned_addr learned my addr 172.21.15.113:0/143628625 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac631b640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac019a120 0x7f1ac01a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5319640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac0106180 0x7f1ac0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 -- 172.21.15.113:0/143628625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac019a120 msgr2=0x7f1ac01a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac019a120 0x7f1ac01a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 -- 172.21.15.113:0/143628625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac0106180 msgr2=0x7f1ac0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac0106180 0x7f1ac0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.227+0000 7f1ac5b1a640 1 -- 172.21.15.113:0/143628625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac01a4ea0 con 0x7f1ac01010b0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.228+0000 7f1ac5b1a640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac01996a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1abc005940 tx=0x7f1abc02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.228+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc041070 con 0x7f1ac01010b0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.228+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1abc0052f0 con 0x7f1ac01010b0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.228+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac01a5130 con 0x7f1ac01010b0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.228+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc003d60 con 0x7f1ac01010b0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.228+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac01a5680 con 0x7f1ac01010b0 2024-04-22T07:24:01.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.229+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac0069230 con 0x7f1ac01010b0 2024-04-22T07:24:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.230+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1abc044070 con 0x7f1ac01010b0 2024-04-22T07:24:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.231+0000 7f1ab6ffd640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.231+0000 7f1ac5319640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.231+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f1abc0f2780 con 0x7f1ac01010b0 2024-04-22T07:24:01.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.231+0000 7f1ac5319640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0005fd0 tx=0x7f1ab0005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.235+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1abc0bed90 con 0x7f1ac01010b0 2024-04-22T07:24:01.334 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:01 smithi113 ceph-mon[27933]: pgmap v288: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:01.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.450+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac01093b0 con 0x7f1ac01010b0 2024-04-22T07:24:01.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.451+0000 7f1ab6ffd640 1 -- 172.21.15.113:0/143628625 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7f1abc0be690 con 0x7f1ac01010b0 2024-04-22T07:24:01.448 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:24:01.449 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:24:01.449 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:24:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1aac0a3d20 msgr2=0x7f1aac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0005fd0 tx=0x7f1ab0005850 comp rx=0 tx=0).stop 2024-04-22T07:24:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 msgr2=0x7f1ac01996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac01996a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1abc005940 tx=0x7f1abc02c6c0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 shutdown_connections 2024-04-22T07:24:01.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1aac0a3d20 0x7f1aac0a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac019a120 0x7f1ac01a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac0106180 0x7f1ac0199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 --2- 172.21.15.113:0/143628625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac01010b0 0x7f1ac01996a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 >> 172.21.15.113:0/143628625 conn(0x7f1ac00771d0 msgr2=0x7f1ac00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 shutdown_connections 2024-04-22T07:24:01.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.453+0000 7f1ac7d7e640 1 -- 172.21.15.113:0/143628625 wait complete. 2024-04-22T07:24:01.462 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:24:01.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 -- 172.21.15.113:0/4150430199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c102b40 msgr2=0x7fa35c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 --2- 172.21.15.113:0/4150430199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c102b40 0x7fa35c106b80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa35002bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 -- 172.21.15.113:0/4150430199 shutdown_connections 2024-04-22T07:24:01.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 --2- 172.21.15.113:0/4150430199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa35c107260 0x7fa35c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 --2- 172.21.15.113:0/4150430199 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c102b40 0x7fa35c106b80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 --2- 172.21.15.113:0/4150430199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c102190 0x7fa35c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 -- 172.21.15.113:0/4150430199 >> 172.21.15.113:0/4150430199 conn(0x7fa35c0793a0 msgr2=0x7fa35c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 -- 172.21.15.113:0/4150430199 shutdown_connections 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.565+0000 7fa3609ce640 1 -- 172.21.15.113:0/4150430199 wait complete. 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 Processor -- start 2024-04-22T07:24:01.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 -- start start 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c102190 0x7fa35c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa35c107260 0x7fa35c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 0x7fa35c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1122e0 con 0x7fa35c198660 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c112160 con 0x7fa35c102190 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa3609ce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c112460 con 0x7fa35c107260 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa35ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 0x7fa35c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa35a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c102190 0x7fa35c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa35ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 0x7fa35c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42608/0 (socket says 172.21.15.113:42608) 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.566+0000 7fa35ad9d640 1 -- 172.21.15.113:0/714747272 learned_addr learned my addr 172.21.15.113:0/714747272 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa35ad9d640 1 -- 172.21.15.113:0/714747272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa35c107260 msgr2=0x7fa35c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa35ad9d640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa35c107260 0x7fa35c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa35ad9d640 1 -- 172.21.15.113:0/714747272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c102190 msgr2=0x7fa35c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa35ad9d640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c102190 0x7fa35c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa35ad9d640 1 -- 172.21.15.113:0/714747272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c199260 con 0x7fa35c198660 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa35ad9d640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 0x7fa35c198ae0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa35400eaa0 tx=0x7fa35400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa34b7fe640 1 -- 172.21.15.113:0/714747272 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35400cdb0 con 0x7fa35c198660 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c1a5340 con 0x7fa35c198660 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa34b7fe640 1 -- 172.21.15.113:0/714747272 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa354005020 con 0x7fa35c198660 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa34b7fe640 1 -- 172.21.15.113:0/714747272 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540116b0 con 0x7fa35c198660 2024-04-22T07:24:01.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.567+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1a57a0 con 0x7fa35c198660 2024-04-22T07:24:01.569 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:01 smithi154 ceph-mon[31862]: pgmap v288: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:01.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:01 smithi133 ceph-mon[31198]: pgmap v288: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.569+0000 7fa34b7fe640 1 -- 172.21.15.113:0/714747272 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa3540051d0 con 0x7fa35c198660 2024-04-22T07:24:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.569+0000 7fa3635c0740 1 Processor -- start 2024-04-22T07:24:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 -- start start 2024-04-22T07:24:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x564205840430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x564205842e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56420579c1f0 con 0x5642058374d0 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5642057394b0 con 0x564205840a00 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa3635c0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5642057981b0 con 0x564205837e80 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa34b7fe640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3440a3d20 0x7fa3440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42624/0 (socket says 172.21.15.113:42624) 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 -- 172.21.15.113:0/3382409744 learned_addr learned my addr 172.21.15.113:0/3382409744 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa35ad9d640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3440a3d20 0x7fa3440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa35a59c640 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x564205840430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x564205842e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa34b7fe640 1 -- 172.21.15.113:0/714747272 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa3540d0b50 con 0x7fa35c198660 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 -- 172.21.15.113:0/3382409744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 msgr2=0x564205840430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x564205840430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 -- 172.21.15.113:0/3382409744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 msgr2=0x564205842e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x564205842e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 -- 172.21.15.113:0/3382409744 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56420572b1b0 con 0x5642058374d0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa359d9b640 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058378b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa350005ab0 tx=0x7fa350005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f18810bc751fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa35ad9d640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3440a3d20 0x7fa3440a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ce1b0 tx=0x7fa3540bd040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.570+0000 7fa348ff9640 1 -- 172.21.15.113:0/3382409744 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350046070 con 0x5642058374d0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa348ff9640 1 -- 172.21.15.113:0/3382409744 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa35002cad0 con 0x5642058374d0 2024-04-22T07:24:01.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa348ff9640 1 -- 172.21.15.113:0/3382409744 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500046c0 con 0x5642058374d0 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 -- 172.21.15.113:0/3382409744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 msgr2=0x5642058378b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058378b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa350005ab0 tx=0x7fa350005a20 comp rx=0 tx=0).stop 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 -- 172.21.15.113:0/3382409744 shutdown_connections 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x564205842e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x564205840430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 --2- 172.21.15.113:0/3382409744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058378b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 -- 172.21.15.113:0/3382409744 >> 172.21.15.113:0/3382409744 conn(0x564205831ba0 msgr2=0x564205831f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 -- 172.21.15.113:0/3382409744 shutdown_connections 2024-04-22T07:24:01.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.571+0000 7fa3635c0740 1 -- 172.21.15.113:0/3382409744 wait complete. 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 -- start start 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 Processor -- start 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x56420583d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x56420583de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564205843910 con 0x5642058374d0 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564205755500 con 0x564205840a00 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564205843a90 con 0x564205837e80 2024-04-22T07:24:01.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa3635c0740 10 client.0 _ll_register_callbacks cb 0x564205723b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42632/0 (socket says 172.21.15.113:42632) 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 -- 172.21.15.113:0/2885092732 learned_addr learned my addr 172.21.15.113:0/2885092732 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa35a59c640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x56420583d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x56420583de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 -- 172.21.15.113:0/2885092732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 msgr2=0x56420583d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x56420583d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 -- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 msgr2=0x56420583de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x56420583de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 -- 172.21.15.113:0/2885092732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642054ae3f0 con 0x5642058374d0 2024-04-22T07:24:01.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa359d9b640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058403a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa350000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fcf3c99fe05811 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.572+0000 7fa341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa341ffb640 1 -- 172.21.15.113:0/2885092732 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350046030 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa341ffb640 1 -- 172.21.15.113:0/2885092732 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa3500043d0 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35002cad0 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa341ffb640 1 -- 172.21.15.113:0/2885092732 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa3500035f0 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 10 client.17619 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5642055787d0 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa341ffb640 1 -- 172.21.15.113:0/2885092732 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa35002cad0 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa341ffb640 1 -- 172.21.15.113:0/2885092732 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa350040040 con 0x5642058374d0 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 10 client.17619 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 20 client.17619 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564205850ae0 0x564205852ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 4 client.17619 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56420573b930 con 0x564205850ae0 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.573+0000 7fa35a59c640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564205850ae0 0x564205852ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.574+0000 7fa35a59c640 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564205850ae0 0x564205852ed0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bf0e92b7b28a4 server_cookie=88393b4b29e4dbf4 in_seq=0 out_seq=0 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.574+0000 7fa341ffb640 10 client.17619 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa341ffb640 1 -- 172.21.15.113:0/2885092732 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56420573b930 con 0x564205850ae0 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa341ffb640 10 client.17619 handle_command_reply: tid=0 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 client.17619 shutdown 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 msgr2=0x5642058403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058403a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa350000bc0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 shutdown_connections 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x564205850ae0 0x564205852ed0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205840a00 0x56420583de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205837e80 0x56420583d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa341ffb640 0 client.17619 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 --2- 172.21.15.113:0/2885092732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x5642058403a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 >> 172.21.15.113:0/2885092732 conn(0x564205831ba0 msgr2=0x5642058338f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 shutdown_connections 2024-04-22T07:24:01.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 1 -- 172.21.15.113:0/2885092732 wait complete. 2024-04-22T07:24:01.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.575+0000 7fa3635c0740 20 client.17619 trim_cache size 0 max 16384 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 Processor -- start 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 -- start start 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205855ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x564205856430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 0x56420583fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1b0810 con 0x5642058374d0 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ae0e0 con 0x564205837e80 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa3635c0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1abf50 con 0x564205840a00 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205855ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205855ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42636/0 (socket says 172.21.15.113:42636) 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.588+0000 7fa359d9b640 1 -- 172.21.15.113:0/1604028861 learned_addr learned my addr 172.21.15.113:0/1604028861 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa359d9b640 1 -- 172.21.15.113:0/1604028861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 msgr2=0x56420583fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa359d9b640 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 0x56420583fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa359d9b640 1 -- 172.21.15.113:0/1604028861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 msgr2=0x564205856430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa359d9b640 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x564205856430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa359d9b640 1 -- 172.21.15.113:0/1604028861 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564205858d10 con 0x5642058374d0 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa359d9b640 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205855ef0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa350005ab0 tx=0x7fa350050b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8647f1a22174141c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa343fff640 1 -- 172.21.15.113:0/1604028861 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350046070 con 0x5642058374d0 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa343fff640 1 -- 172.21.15.113:0/1604028861 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa350005850 con 0x5642058374d0 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa343fff640 1 -- 172.21.15.113:0/1604028861 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350050e60 con 0x5642058374d0 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa3635c0740 1 -- 172.21.15.113:0/1604028861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 msgr2=0x564205855ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.589+0000 7fa3635c0740 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205855ef0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa350005ab0 tx=0x7fa350050b80 comp rx=0 tx=0).stop 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- 172.21.15.113:0/1604028861 shutdown_connections 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 0x56420583fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x564205856430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 --2- 172.21.15.113:0/1604028861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205855ef0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- 172.21.15.113:0/1604028861 >> 172.21.15.113:0/1604028861 conn(0x564205831ba0 msgr2=0x564205830c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- 172.21.15.113:0/1604028861 shutdown_connections 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- 172.21.15.113:0/1604028861 wait complete. 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- start start 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 Processor -- start 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205856060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x5642058533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 0x564205853930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ac5a0 con 0x5642058374d0 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ada10 con 0x564205837e80 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1acc70 con 0x564205840a00 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa3635c0740 10 client.0 _ll_register_callbacks cb 0x7fa35002c5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205856060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205856060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42638/0 (socket says 172.21.15.113:42638) 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 -- 172.21.15.113:0/309063217 learned_addr learned my addr 172.21.15.113:0/309063217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa35a59c640 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x5642058533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 -- 172.21.15.113:0/309063217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 msgr2=0x564205853930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 0x564205853930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 -- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 msgr2=0x5642058533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x5642058533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.590+0000 7fa359d9b640 1 -- 172.21.15.113:0/309063217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564205853f20 con 0x5642058374d0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa359d9b640 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205856060 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa3500044a0 tx=0x7fa35002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d43a6077386311d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa341ffb640 1 -- 172.21.15.113:0/309063217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350062070 con 0x5642058374d0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa341ffb640 1 -- 172.21.15.113:0/309063217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa350003ba0 con 0x5642058374d0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa350005850 con 0x5642058374d0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa341ffb640 1 -- 172.21.15.113:0/309063217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa350050e60 con 0x5642058374d0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 10 client.17625 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5642058b6ac0 con 0x5642058374d0 2024-04-22T07:24:01.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa341ffb640 1 -- 172.21.15.113:0/309063217 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa350005850 con 0x5642058374d0 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa341ffb640 1 -- 172.21.15.113:0/309063217 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa350034a30 con 0x5642058374d0 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 10 client.17625 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 20 client.17625 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5642058b7070 0x5642058b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 4 client.17625 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.591+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642057a9c10 con 0x5642058b7070 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa35a59c640 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5642058b7070 0x5642058b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa35a59c640 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5642058b7070 0x5642058b9490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cc1d900e22f792 server_cookie=2867e9cbdaad6411 in_seq=0 out_seq=0 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa341ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa341ffb640 1 -- 172.21.15.113:0/309063217 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642057a9c10 con 0x5642058b7070 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa341ffb640 10 client.17625 handle_command_reply: tid=0 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 client.17625 shutdown 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 msgr2=0x564205856060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205856060 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa3500044a0 tx=0x7fa35002c850 comp rx=0 tx=0).stop 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 shutdown_connections 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5642058b7070 0x5642058b9490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564205840a00 0x564205853930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564205837e80 0x5642058533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa341ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 --2- 172.21.15.113:0/309063217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5642058374d0 0x564205856060 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 >> 172.21.15.113:0/309063217 conn(0x564205831ba0 msgr2=0x564205830370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 shutdown_connections 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.592+0000 7fa3635c0740 1 -- 172.21.15.113:0/309063217 wait complete. 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3635c0740 20 client.17625 trim_cache size 0 max 16384 2024-04-22T07:24:01.596 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:24:01.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3440a3d20 msgr2=0x7fa3440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3440a3d20 0x7fa3440a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ce1b0 tx=0x7fa3540bd040 comp rx=0 tx=0).stop 2024-04-22T07:24:01.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 msgr2=0x7fa35c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 0x7fa35c198ae0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa35400eaa0 tx=0x7fa35400ef60 comp rx=0 tx=0).stop 2024-04-22T07:24:01.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 shutdown_connections 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa3440a3d20 0x7fa3440a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa35c198660 0x7fa35c198ae0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa35c107260 0x7fa35c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 --2- 172.21.15.113:0/714747272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c102190 0x7fa35c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 >> 172.21.15.113:0/714747272 conn(0x7fa35c0793a0 msgr2=0x7fa35c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 shutdown_connections 2024-04-22T07:24:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.593+0000 7fa3609ce640 1 -- 172.21.15.113:0/714747272 wait complete. 2024-04-22T07:24:01.601 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:24:01.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 -- 172.21.15.113:0/2764403607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 msgr2=0x7fa46010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 --2- 172.21.15.113:0/2764403607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46010dd30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa458009ca0 tx=0x7fa45802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 -- 172.21.15.113:0/2764403607 shutdown_connections 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 --2- 172.21.15.113:0/2764403607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46010dd30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 --2- 172.21.15.113:0/2764403607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa460101a40 0x7fa460105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 --2- 172.21.15.113:0/2764403607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 0x7fa460101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 -- 172.21.15.113:0/2764403607 >> 172.21.15.113:0/2764403607 conn(0x7fa460077f50 msgr2=0x7fa460078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 -- 172.21.15.113:0/2764403607 shutdown_connections 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.704+0000 7fa468432640 1 -- 172.21.15.113:0/2764403607 wait complete. 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 Processor -- start 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 -- start start 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 0x7fa46019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4601986b0 0x7fa460198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa460111200 con 0x7fa460106160 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa460111080 con 0x7fa4601986b0 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa468432640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa460111380 con 0x7fa460101090 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42662/0 (socket says 172.21.15.113:42662) 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 -- 172.21.15.113:0/165534806 learned_addr learned my addr 172.21.15.113:0/165534806 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4669cf640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4601986b0 0x7fa460198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4661ce640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 0x7fa46019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 -- 172.21.15.113:0/165534806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 msgr2=0x7fa46019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 0x7fa46019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 -- 172.21.15.113:0/165534806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4601986b0 msgr2=0x7fa460198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4601986b0 0x7fa460198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 -- 172.21.15.113:0/165534806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4601992b0 con 0x7fa460106160 2024-04-22T07:24:01.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4661ce640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 0x7fa46019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4659cd640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46019eae0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d770 tx=0x7fa45000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.705+0000 7fa4577fe640 1 -- 172.21.15.113:0/165534806 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450017070 con 0x7fa460106160 2024-04-22T07:24:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.706+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4601a5340 con 0x7fa460106160 2024-04-22T07:24:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.706+0000 7fa4577fe640 1 -- 172.21.15.113:0/165534806 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa450009c30 con 0x7fa460106160 2024-04-22T07:24:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.706+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4601a57a0 con 0x7fa460106160 2024-04-22T07:24:01.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.706+0000 7fa4577fe640 1 -- 172.21.15.113:0/165534806 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450021060 con 0x7fa460106160 2024-04-22T07:24:01.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.708+0000 7fa4577fe640 1 -- 172.21.15.113:0/165534806 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fa45001a070 con 0x7fa460106160 2024-04-22T07:24:01.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.708+0000 7fa46b01c740 1 Processor -- start 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.708+0000 7fa46b01c740 1 -- start start 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a654d0 0x55cec9a658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a6ea00 0x55cec9a70e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa46b01c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cec99ca1f0 con 0x55cec9a65e80 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa46b01c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cec99674b0 con 0x55cec9a6ea00 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa46b01c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cec99c61b0 con 0x55cec9a654d0 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4577fe640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa44c0a3d20 0x7fa44c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42672/0 (socket says 172.21.15.113:42672) 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 -- 172.21.15.113:0/2840898609 learned_addr learned my addr 172.21.15.113:0/2840898609 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4669cf640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa44c0a3d20 0x7fa44c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4669cf640 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a654d0 0x55cec9a658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4577fe640 1 -- 172.21.15.113:0/165534806 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa4500d07d0 con 0x7fa460106160 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4659cd640 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a6ea00 0x55cec9a70e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 -- 172.21.15.113:0/2840898609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a654d0 msgr2=0x55cec9a658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a654d0 0x55cec9a658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 -- 172.21.15.113:0/2840898609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a6ea00 msgr2=0x55cec9a70e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a6ea00 0x55cec9a70e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 -- 172.21.15.113:0/2840898609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec99591b0 con 0x55cec9a65e80 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4669cf640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa44c0a3d20 0x7fa44c0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa458009430 tx=0x7fa458032340 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6e430 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00bb70 tx=0x7fa45c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6088bb31df54295 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.709+0000 7fa454bfe640 1 -- 172.21.15.113:0/2840898609 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c017620 con 0x55cec9a65e80 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa454bfe640 1 -- 172.21.15.113:0/2840898609 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa45c017c90 con 0x55cec9a65e80 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa454bfe640 1 -- 172.21.15.113:0/2840898609 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00fb20 con 0x55cec9a65e80 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 -- 172.21.15.113:0/2840898609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 msgr2=0x55cec9a6e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6e430 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00bb70 tx=0x7fa45c00d710 comp rx=0 tx=0).stop 2024-04-22T07:24:01.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 -- 172.21.15.113:0/2840898609 shutdown_connections 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a6ea00 0x55cec9a70e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6e430 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2840898609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a654d0 0x55cec9a658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 -- 172.21.15.113:0/2840898609 >> 172.21.15.113:0/2840898609 conn(0x55cec9a5fba0 msgr2=0x55cec9a5ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 -- 172.21.15.113:0/2840898609 shutdown_connections 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.710+0000 7fa46b01c740 1 -- 172.21.15.113:0/2840898609 wait complete. 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 -- start start 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 Processor -- start 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a6b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a6bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cec9a71970 con 0x55cec9a6ea00 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cec9983500 con 0x55cec9a654d0 2024-04-22T07:24:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cec9a71af0 con 0x55cec9a65e80 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa46b01c740 10 client.0 _ll_register_callbacks cb 0x55cec9951b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a6bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a6bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42680/0 (socket says 172.21.15.113:42680) 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 -- 172.21.15.113:0/2914848121 learned_addr learned my addr 172.21.15.113:0/2914848121 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 -- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 msgr2=0x55cec9a6b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a6b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 -- 172.21.15.113:0/2914848121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 msgr2=0x55cec9a6e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a6e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 -- 172.21.15.113:0/2914848121 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec96dc3f0 con 0x55cec9a6ea00 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa4669cf640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a6bb10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa458036000 tx=0x7fa458044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223216f084385965 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.711+0000 7fa449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa449ffb640 1 -- 172.21.15.113:0/2914848121 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458049040 con 0x55cec9a6ea00 2024-04-22T07:24:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa449ffb640 1 -- 172.21.15.113:0/2914848121 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa458004040 con 0x55cec9a6ea00 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa45c017c90 con 0x55cec9a6ea00 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa449ffb640 1 -- 172.21.15.113:0/2914848121 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa45805b050 con 0x55cec9a6ea00 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 10 client.17637 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cec97a67d0 con 0x55cec9a6ea00 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa449ffb640 1 -- 172.21.15.113:0/2914848121 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa45c017c90 con 0x55cec9a6ea00 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa449ffb640 1 -- 172.21.15.113:0/2914848121 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa458034880 con 0x55cec9a6ea00 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 10 client.17637 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 10 client.17637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 20 client.17637 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9a7ebf0 0x55cec9a81080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 4 client.17637 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.712+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec9969930 con 0x55cec9a7ebf0 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.713+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9a7ebf0 0x55cec9a81080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.713+0000 7fa4661ce640 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9a7ebf0 0x55cec9a81080 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8cfa9646fb5821ad server_cookie=54faaba7564db2af in_seq=0 out_seq=0 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.713+0000 7fa449ffb640 10 client.17637 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:01.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa449ffb640 1 -- 172.21.15.113:0/2914848121 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cec9969930 con 0x55cec9a7ebf0 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa449ffb640 10 client.17637 handle_command_reply: tid=0 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 client.17637 shutdown 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 msgr2=0x55cec9a6bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a6bb10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa458036000 tx=0x7fa458044000 comp rx=0 tx=0).stop 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 shutdown_connections 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9a7ebf0 0x55cec9a81080 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a6bb10 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa449ffb640 0 client.17637 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 --2- 172.21.15.113:0/2914848121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 >> 172.21.15.113:0/2914848121 conn(0x55cec9a5fba0 msgr2=0x55cec9a618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 shutdown_connections 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 1 -- 172.21.15.113:0/2914848121 wait complete. 2024-04-22T07:24:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.714+0000 7fa46b01c740 20 client.17637 trim_cache size 0 max 16384 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 Processor -- start 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 -- start start 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a7a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a6ea00 0x55cec9a6e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601b0890 con 0x55cec9a65e80 2024-04-22T07:24:01.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601ae160 con 0x55cec9a654d0 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa46b01c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601abfd0 con 0x55cec9a6ea00 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa4659cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa4659cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42688/0 (socket says 172.21.15.113:42688) 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.727+0000 7fa4659cd640 1 -- 172.21.15.113:0/132583450 learned_addr learned my addr 172.21.15.113:0/132583450 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4669cf640 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a6ea00 0x55cec9a6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4659cd640 1 -- 172.21.15.113:0/132583450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a6ea00 msgr2=0x55cec9a6e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4659cd640 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a6ea00 0x55cec9a6e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4659cd640 1 -- 172.21.15.113:0/132583450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 msgr2=0x55cec9a7a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4659cd640 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a7a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4659cd640 1 -- 172.21.15.113:0/132583450 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec9a70df0 con 0x55cec9a65e80 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4669cf640 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a6ea00 0x55cec9a6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa4659cd640 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6dd00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d740 tx=0x7fa45000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf0b894d60d0e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa44bfff640 1 -- 172.21.15.113:0/132583450 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450009de0 con 0x55cec9a65e80 2024-04-22T07:24:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa44bfff640 1 -- 172.21.15.113:0/132583450 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa450015020 con 0x55cec9a65e80 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa44bfff640 1 -- 172.21.15.113:0/132583450 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4500a4db0 con 0x55cec9a65e80 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 -- 172.21.15.113:0/132583450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 msgr2=0x55cec9a6dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6dd00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d740 tx=0x7fa45000de80 comp rx=0 tx=0).stop 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 -- 172.21.15.113:0/132583450 shutdown_connections 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a6ea00 0x55cec9a6e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a65e80 0x55cec9a6dd00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 --2- 172.21.15.113:0/132583450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a7a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 -- 172.21.15.113:0/132583450 >> 172.21.15.113:0/132583450 conn(0x55cec9a5fba0 msgr2=0x55cec9a66c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 -- 172.21.15.113:0/132583450 shutdown_connections 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.728+0000 7fa46b01c740 1 -- 172.21.15.113:0/132583450 wait complete. 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 -- start start 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 Processor -- start 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a72940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a72e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601ac620 con 0x55cec9a6ea00 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601ada90 con 0x55cec9a654d0 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601accf0 con 0x55cec9a65e80 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa46b01c740 10 client.0 _ll_register_callbacks cb 0x7fa458009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42700/0 (socket says 172.21.15.113:42700) 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 -- 172.21.15.113:0/1405595015 learned_addr learned my addr 172.21.15.113:0/1405595015 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4661ce640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a72940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4659cd640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a72e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 -- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 msgr2=0x55cec9a72e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a72e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 -- 172.21.15.113:0/1405595015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 msgr2=0x55cec9a72940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a72940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 -- 172.21.15.113:0/1405595015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec9ae4370 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4661ce640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a72940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4659cd640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a72e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa4669cf640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a733c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa458034e40 tx=0x7fa4580346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d15d7b995bd5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.729+0000 7fa449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa449ffb640 1 -- 172.21.15.113:0/1405595015 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458049070 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa450015020 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa449ffb640 1 -- 172.21.15.113:0/1405595015 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7fa45804d620 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa449ffb640 1 -- 172.21.15.113:0/1405595015 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa45804d820 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 10 client.17643 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55cec9ae55a0 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa449ffb640 1 -- 172.21.15.113:0/1405595015 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa450015020 con 0x55cec9a6ea00 2024-04-22T07:24:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa449ffb640 1 -- 172.21.15.113:0/1405595015 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fa458041800 con 0x55cec9a6ea00 2024-04-22T07:24:01.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 10 client.17643 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 10 client.17643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:24:01.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 20 client.17643 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9ae5c40 0x55cec9ae8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 4 client.17643 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:24:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.730+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cec99d7c10 con 0x55cec9ae5c40 2024-04-22T07:24:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa4661ce640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9ae5c40 0x55cec9ae8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa4661ce640 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9ae5c40 0x55cec9ae8060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a72b307c6c4e7227 server_cookie=fa31d687d3336e16 in_seq=0 out_seq=0 2024-04-22T07:24:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa449ffb640 10 client.17643 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa449ffb640 1 -- 172.21.15.113:0/1405595015 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cec99d7c10 con 0x55cec9ae5c40 2024-04-22T07:24:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa449ffb640 10 client.17643 handle_command_reply: tid=0 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 client.17643 shutdown 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 msgr2=0x55cec9a733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a733c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa458034e40 tx=0x7fa4580346f0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 shutdown_connections 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55cec9ae5c40 0x55cec9ae8060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cec9a6ea00 0x55cec9a733c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cec9a65e80 0x55cec9a72e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 --2- 172.21.15.113:0/1405595015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cec9a654d0 0x55cec9a72940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa449ffb640 0 client.17643 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 >> 172.21.15.113:0/1405595015 conn(0x55cec9a5fba0 msgr2=0x55cec9a5e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 shutdown_connections 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 1 -- 172.21.15.113:0/1405595015 wait complete. 2024-04-22T07:24:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.731+0000 7fa46b01c740 20 client.17643 trim_cache size 0 max 16384 2024-04-22T07:24:01.739 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:24:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa44c0a3d20 msgr2=0x7fa44c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa44c0a3d20 0x7fa44c0a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa458009430 tx=0x7fa458032340 comp rx=0 tx=0).stop 2024-04-22T07:24:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 msgr2=0x7fa46019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46019eae0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa45000d770 tx=0x7fa45000dc30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 shutdown_connections 2024-04-22T07:24:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fa44c0a3d20 0x7fa44c0a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4601986b0 0x7fa460198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa460106160 0x7fa46019eae0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 --2- 172.21.15.113:0/165534806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa460101090 0x7fa46019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 >> 172.21.15.113:0/165534806 conn(0x7fa460077f50 msgr2=0x7fa460104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 shutdown_connections 2024-04-22T07:24:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.732+0000 7fa468432640 1 -- 172.21.15.113:0/165534806 wait complete. 2024-04-22T07:24:01.741 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 -- 172.21.15.113:0/3178199815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 msgr2=0x7f354c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 --2- 172.21.15.113:0/3178199815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c101480 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f353c009900 tx=0x7f353c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 -- 172.21.15.113:0/3178199815 shutdown_connections 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 --2- 172.21.15.113:0/3178199815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c106170 0x7f354c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 --2- 172.21.15.113:0/3178199815 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c101a50 0x7f354c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 --2- 172.21.15.113:0/3178199815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c101480 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 -- 172.21.15.113:0/3178199815 >> 172.21.15.113:0/3178199815 conn(0x7f354c077eb0 msgr2=0x7f354c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 -- 172.21.15.113:0/3178199815 shutdown_connections 2024-04-22T07:24:01.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.844+0000 7f3550ea5640 1 -- 172.21.15.113:0/3178199815 wait complete. 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 Processor -- start 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 -- start start 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c106170 0x7f354c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c1986f0 0x7f354c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f354c112310 con 0x7f354c1010a0 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f354c112190 con 0x7f354c1986f0 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3550ea5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f354c112490 con 0x7f354c106170 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42726/0 (socket says 172.21.15.113:42726) 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 -- 172.21.15.113:0/1507660979 learned_addr learned my addr 172.21.15.113:0/1507660979 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354ad9d640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c1986f0 0x7f354c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f3543fff640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c106170 0x7f354c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 -- 172.21.15.113:0/1507660979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c106170 msgr2=0x7f354c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c106170 0x7f354c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 -- 172.21.15.113:0/1507660979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c1986f0 msgr2=0x7f354c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c1986f0 0x7f354c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354a59c640 1 -- 172.21.15.113:0/1507660979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c1992d0 con 0x7f354c1010a0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.845+0000 7f354ad9d640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c1986f0 0x7f354c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.846+0000 7f354a59c640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c19e5a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f353c006010 tx=0x7f353c02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.846+0000 7f35437fe640 1 -- 172.21.15.113:0/1507660979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c041070 con 0x7f354c1010a0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.846+0000 7f35437fe640 1 -- 172.21.15.113:0/1507660979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f353c0052f0 con 0x7f354c1010a0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.846+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c1a53b0 con 0x7f354c1010a0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.846+0000 7f35437fe640 1 -- 172.21.15.113:0/1507660979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c003db0 con 0x7f354c1010a0 2024-04-22T07:24:01.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.846+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c1a5760 con 0x7f354c1010a0 2024-04-22T07:24:01.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f35437fe640 1 -- 172.21.15.113:0/1507660979 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f353c044070 con 0x7f354c1010a0 2024-04-22T07:24:01.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 Processor -- start 2024-04-22T07:24:01.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 -- start start 2024-04-22T07:24:01.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee64d0 0x55c3adee68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adeef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adef1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c3ade4b1f0 con 0x55c3adee6e80 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c3adde84b0 con 0x55c3adeefa00 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.848+0000 7f3553b71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c3ade471b0 con 0x55c3adee64d0 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adeef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adeef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42738/0 (socket says 172.21.15.113:42738) 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 -- 172.21.15.113:0/992672944 learned_addr learned my addr 172.21.15.113:0/992672944 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f35437fe640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f35380a3d20 0x7f35380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354ad9d640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee64d0 0x55c3adee68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f3543fff640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adef1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354ad9d640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f35380a3d20 0x7f35380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f35437fe640 1 -- 172.21.15.113:0/1507660979 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f353c0f27e0 con 0x7f354c1010a0 2024-04-22T07:24:01.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 -- 172.21.15.113:0/992672944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee64d0 msgr2=0x55c3adee68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee64d0 0x55c3adee68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 -- 172.21.15.113:0/992672944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 msgr2=0x55c3adef1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adef1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 -- 172.21.15.113:0/992672944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3addda1b0 con 0x55c3adee6e80 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f3543fff640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adef1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354ad9d640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee64d0 0x55c3adee68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354a59c640 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adeef430 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f353c0be3f0 tx=0x7f353c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bdce87818dbffd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f3540ff9640 1 -- 172.21.15.113:0/992672944 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c04b070 con 0x55c3adee6e80 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f354ad9d640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f35380a3d20 0x7f35380a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3544013600 tx=0x7f3544005c60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f3540ff9640 1 -- 172.21.15.113:0/992672944 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f353c0bedf0 con 0x55c3adee6e80 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.849+0000 7f3540ff9640 1 -- 172.21.15.113:0/992672944 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0bc9f0 con 0x55c3adee6e80 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 -- 172.21.15.113:0/992672944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 msgr2=0x55c3adeef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adeef430 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f353c0be3f0 tx=0x7f353c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:01.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 -- 172.21.15.113:0/992672944 shutdown_connections 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adef1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adeef430 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 --2- 172.21.15.113:0/992672944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee64d0 0x55c3adee68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 -- 172.21.15.113:0/992672944 >> 172.21.15.113:0/992672944 conn(0x55c3adee0ba0 msgr2=0x55c3adee0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 -- 172.21.15.113:0/992672944 shutdown_connections 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 -- 172.21.15.113:0/992672944 wait complete. 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.850+0000 7f3553b71740 1 -- start start 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 Processor -- start 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 0x55c3adeef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee6e80 0x55c3adeec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adeecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c3ade4b1f0 con 0x55c3adee64d0 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c3ade04500 con 0x55c3adeefa00 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c3adef2730 con 0x55c3adee6e80 2024-04-22T07:24:01.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3553b71740 10 client.0 _ll_register_callbacks cb 0x55c3addd2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 0x55c3adeef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 0x55c3adeef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42742/0 (socket says 172.21.15.113:42742) 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 -- 172.21.15.113:0/2377130540 learned_addr learned my addr 172.21.15.113:0/2377130540 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3543fff640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee6e80 0x55c3adeec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354ad9d640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adeecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 -- 172.21.15.113:0/2377130540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee6e80 msgr2=0x55c3adeec6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee6e80 0x55c3adeec6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 -- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 msgr2=0x55c3adeecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adeecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 -- 172.21.15.113:0/2377130540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3adb5d3f0 con 0x55c3adee64d0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354ad9d640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adeecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f354a59c640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 0x55c3adeef320 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f353c0bb0c0 tx=0x7f353c0be2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e40c6ba446bf6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3531ffb640 1 -- 172.21.15.113:0/2377130540 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c04b070 con 0x55c3adee64d0 2024-04-22T07:24:01.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.851+0000 7f3531ffb640 1 -- 172.21.15.113:0/2377130540 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f353c0054a0 con 0x55c3adee64d0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353c0bedf0 con 0x55c3adee64d0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3531ffb640 1 -- 172.21.15.113:0/2377130540 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f353c0c7360 con 0x55c3adee64d0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 10 client.17655 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c3adc277d0 con 0x55c3adee64d0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3531ffb640 1 -- 172.21.15.113:0/2377130540 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f353c0bedf0 con 0x55c3adee64d0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3531ffb640 1 -- 172.21.15.113:0/2377130540 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f353c0c6ed0 con 0x55c3adee64d0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 10 client.17655 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 10 client.17655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 20 client.17655 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adeff6e0 0x55c3adf01ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 4 client.17655 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3addea930 con 0x55c3adeff6e0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.852+0000 7f3543fff640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adeff6e0 0x55c3adf01ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.853+0000 7f3531ffb640 10 client.17655 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.853+0000 7f3543fff640 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adeff6e0 0x55c3adf01ad0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f05f56cbc4fd43ab server_cookie=ce985ad3ae879e7 in_seq=0 out_seq=0 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.853+0000 7f3531ffb640 1 -- 172.21.15.113:0/2377130540 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3addea930 con 0x55c3adeff6e0 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.853+0000 7f3531ffb640 10 client.17655 handle_command_reply: tid=0 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.853+0000 7f3553b71740 1 client.17655 shutdown 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 msgr2=0x55c3adeef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 0x55c3adeef320 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f353c0bb0c0 tx=0x7f353c0be2c0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 shutdown_connections 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adeff6e0 0x55c3adf01ad0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adeefa00 0x55c3adeecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3531ffb640 0 client.17655 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adee6e80 0x55c3adeec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 --2- 172.21.15.113:0/2377130540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee64d0 0x55c3adeef320 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 >> 172.21.15.113:0/2377130540 conn(0x55c3adee0ba0 msgr2=0x55c3adee28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 shutdown_connections 2024-04-22T07:24:01.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 1 -- 172.21.15.113:0/2377130540 wait complete. 2024-04-22T07:24:01.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.854+0000 7f3553b71740 20 client.17655 trim_cache size 0 max 16384 2024-04-22T07:24:01.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 Processor -- start 2024-04-22T07:24:01.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 -- start start 2024-04-22T07:24:01.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 0x55c3adefb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adeeede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1b08a0 con 0x55c3adee6e80 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ae170 con 0x55c3adee64d0 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3553b71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1abfe0 con 0x55c3adeefa00 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42748/0 (socket says 172.21.15.113:42748) 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 -- 172.21.15.113:0/2239227143 learned_addr learned my addr 172.21.15.113:0/2239227143 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 -- 172.21.15.113:0/2239227143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 msgr2=0x55c3adeeede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adeeede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 -- 172.21.15.113:0/2239227143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 msgr2=0x55c3adefb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 0x55c3adefb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 -- 172.21.15.113:0/2239227143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3adee5820 con 0x55c3adee6e80 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3543fff640 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee2700 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f354c06a400 tx=0x7f3534005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463b2d05952622ba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.867+0000 7f3533fff640 1 -- 172.21.15.113:0/2239227143 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353401f020 con 0x55c3adee6e80 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3533fff640 1 -- 172.21.15.113:0/2239227143 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f353401bdc0 con 0x55c3adee6e80 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3533fff640 1 -- 172.21.15.113:0/2239227143 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534003850 con 0x55c3adee6e80 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 -- 172.21.15.113:0/2239227143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 msgr2=0x55c3adee2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee2700 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f354c06a400 tx=0x7f3534005850 comp rx=0 tx=0).stop 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 -- 172.21.15.113:0/2239227143 shutdown_connections 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adeeede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee2700 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 --2- 172.21.15.113:0/2239227143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 0x55c3adefb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 -- 172.21.15.113:0/2239227143 >> 172.21.15.113:0/2239227143 conn(0x55c3adee0ba0 msgr2=0x55c3adee1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 -- 172.21.15.113:0/2239227143 shutdown_connections 2024-04-22T07:24:01.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 -- 172.21.15.113:0/2239227143 wait complete. 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 -- start start 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.868+0000 7f3553b71740 1 Processor -- start 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 0x55c3adee9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adee9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ac630 con 0x55c3adee6e80 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1adaa0 con 0x55c3adee64d0 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1acd00 con 0x55c3adeefa00 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 10 client.0 _ll_register_callbacks cb 0x7f353c0be6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42752/0 (socket says 172.21.15.113:42752) 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 -- 172.21.15.113:0/4129494749 learned_addr learned my addr 172.21.15.113:0/4129494749 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f354ad9d640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adee9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 -- 172.21.15.113:0/4129494749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 msgr2=0x55c3adee9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adee9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 -- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 msgr2=0x55c3adee9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 0x55c3adee9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 -- 172.21.15.113:0/4129494749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3adf64a80 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f354ad9d640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adee9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3543fff640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee9850 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f353401bb70 tx=0x7f353401bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaede86e46e39d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3531ffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3531ffb640 1 -- 172.21.15.113:0/4129494749 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353401b790 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3531ffb640 1 -- 172.21.15.113:0/4129494749 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f353402d440 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353401bdc0 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3531ffb640 1 -- 172.21.15.113:0/4129494749 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f353402d640 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 10 client.17661 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.869+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c3adf65d60 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3531ffb640 1 -- 172.21.15.113:0/4129494749 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f353401bdc0 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3531ffb640 1 -- 172.21.15.113:0/4129494749 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f353402b020 con 0x55c3adee6e80 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3553b71740 10 client.17661 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:01.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3553b71740 10 client.17661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3553b71740 20 client.17661 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3553b71740 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adf66410 0x55c3adf68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3553b71740 4 client.17661 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3ade58c10 con 0x55c3adf66410 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f354a59c640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adf66410 0x55c3adf68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f3531ffb640 10 client.17661 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.870+0000 7f354a59c640 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adf66410 0x55c3adf68830 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25f3a4b025b4551a server_cookie=c08b0d9a502b1281 in_seq=0 out_seq=0 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3531ffb640 1 -- 172.21.15.113:0/4129494749 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3ade58c10 con 0x55c3adf66410 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3531ffb640 10 client.17661 handle_command_reply: tid=0 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 client.17661 shutdown 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 msgr2=0x55c3adee9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee9850 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f353401bb70 tx=0x7f353401bba0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 shutdown_connections 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x55c3adf66410 0x55c3adf68830 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c3adeefa00 0x55c3adee9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c3adee6e80 0x55c3adee9850 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3531ffb640 0 client.17661 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 --2- 172.21.15.113:0/4129494749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c3adee64d0 0x55c3adee9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 >> 172.21.15.113:0/4129494749 conn(0x55c3adee0ba0 msgr2=0x55c3adee2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 shutdown_connections 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 1 -- 172.21.15.113:0/4129494749 wait complete. 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3553b71740 20 client.17661 trim_cache size 0 max 16384 2024-04-22T07:24:01.875 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:24:01.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f35380a3d20 msgr2=0x7f35380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f35380a3d20 0x7f35380a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3544013600 tx=0x7f3544005c60 comp rx=0 tx=0).stop 2024-04-22T07:24:01.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 msgr2=0x7f354c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:01.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c19e5a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f353c006010 tx=0x7f353c02c710 comp rx=0 tx=0).stop 2024-04-22T07:24:01.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 shutdown_connections 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f35380a3d20 0x7f35380a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f354c1986f0 0x7f354c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f354c106170 0x7f354c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 --2- 172.21.15.113:0/1507660979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c1010a0 0x7f354c19e5a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.871+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 >> 172.21.15.113:0/1507660979 conn(0x7f354c077eb0 msgr2=0x7f354c107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.872+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 shutdown_connections 2024-04-22T07:24:01.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:01.872+0000 7f3550ea5640 1 -- 172.21.15.113:0/1507660979 wait complete. 2024-04-22T07:24:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:02 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/143628625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:24:02.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:02 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:24:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:02 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/143628625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:24:02.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:02 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:24:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:02 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/143628625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:24:02.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:02 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:24:02.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:24:02.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:24:02.881 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:24:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.987+0000 7f1f27025640 1 -- 172.21.15.113:0/1434939289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2010c520 msgr2=0x7f1f2010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.987+0000 7f1f27025640 1 --2- 172.21.15.113:0/1434939289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2010c520 0x7f1f2010e9c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009900 tx=0x7f1f0802bd60 comp rx=0 tx=0).stop 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- 172.21.15.113:0/1434939289 shutdown_connections 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 --2- 172.21.15.113:0/1434939289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f2010c520 0x7f1f2010e9c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 --2- 172.21.15.113:0/1434939289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f20103b00 0x7f1f2010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 --2- 172.21.15.113:0/1434939289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f201031e0 0x7f1f201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- 172.21.15.113:0/1434939289 >> 172.21.15.113:0/1434939289 conn(0x7f1f200771d0 msgr2=0x7f1f200775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- 172.21.15.113:0/1434939289 shutdown_connections 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- 172.21.15.113:0/1434939289 wait complete. 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 Processor -- start 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- start start 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f201031e0 0x7f1f2019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 0x7f1f2019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f2010c520 0x7f1f20198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20112340 con 0x7f1f20103b00 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201121c0 con 0x7f1f2010c520 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f27025640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201124c0 con 0x7f1f201031e0 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 0x7f1f2019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 0x7f1f2019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42762/0 (socket says 172.21.15.113:42762) 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.988+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1134603517 learned_addr learned my addr 172.21.15.113:0/1134603517 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f201031e0 0x7f1f2019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f255c2640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f2010c520 0x7f1f20198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1134603517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f201031e0 msgr2=0x7f1f2019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f201031e0 0x7f1f2019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1134603517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f2010c520 msgr2=0x7f1f20198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f2010c520 0x7f1f20198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1134603517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f20198da0 con 0x7f1f20103b00 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 0x7f1f2019e960 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000eab0 tx=0x7f1f1000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1dffb640 1 -- 172.21.15.113:0/1134603517 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000ce40 con 0x7f1f20103b00 2024-04-22T07:24:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1dffb640 1 -- 172.21.15.113:0/1134603517 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f10011940 con 0x7f1f20103b00 2024-04-22T07:24:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f20199090 con 0x7f1f20103b00 2024-04-22T07:24:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f1dffb640 1 -- 172.21.15.113:0/1134603517 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10010aa0 con 0x7f1f20103b00 2024-04-22T07:24:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.989+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f201a52e0 con 0x7f1f20103b00 2024-04-22T07:24:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.991+0000 7f1f1dffb640 1 -- 172.21.15.113:0/1134603517 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1f10011af0 con 0x7f1f20103b00 2024-04-22T07:24:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 Processor -- start 2024-04-22T07:24:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 -- start start 2024-04-22T07:24:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f1dffb640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f0c0a3d20 0x7f1f0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f083b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f0840f0 0x56011f08cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f08d250 0x56011f08d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56011efe7f00 con 0x56011f083740 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56011ef85590 con 0x56011f08d250 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f29c12740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56011efe3ec0 con 0x56011f0840f0 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f255c2640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f0c0a3d20 0x7f1f0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f083b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f083b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42770/0 (socket says 172.21.15.113:42770) 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 -- 172.21.15.113:0/661277363 learned_addr learned my addr 172.21.15.113:0/661277363 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f1dffb640 1 -- 172.21.15.113:0/1134603517 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f1f100d0eb0 con 0x7f1f20103b00 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f255c2640 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f08d250 0x56011f08d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f0840f0 0x56011f08cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 -- 172.21.15.113:0/661277363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f0840f0 msgr2=0x56011f08cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f0840f0 0x56011f08cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 -- 172.21.15.113:0/661277363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f08d250 msgr2=0x56011f08d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f08d250 0x56011f08d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 -- 172.21.15.113:0/661277363 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56011edc27e0 con 0x56011f083740 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f255c2640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f0c0a3d20 0x7f1f0c0a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1f08006010 tx=0x7f1f08004270 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.992+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f083b20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bd10 tx=0x7f1f180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f8a1c27ca6087c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f16ffd640 1 -- 172.21.15.113:0/661277363 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18004030 con 0x56011f083740 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f16ffd640 1 -- 172.21.15.113:0/661277363 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f180041d0 con 0x56011f083740 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f16ffd640 1 -- 172.21.15.113:0/661277363 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180117b0 con 0x56011f083740 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 -- 172.21.15.113:0/661277363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 msgr2=0x56011f083b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f083b20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bd10 tx=0x7f1f180107a0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 -- 172.21.15.113:0/661277363 shutdown_connections 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f08d250 0x56011f08d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f0840f0 0x56011f08cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 --2- 172.21.15.113:0/661277363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f083b20 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 -- 172.21.15.113:0/661277363 >> 172.21.15.113:0/661277363 conn(0x56011f07dfb0 msgr2=0x56011f07e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 -- 172.21.15.113:0/661277363 shutdown_connections 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.993+0000 7f1f29c12740 1 -- 172.21.15.113:0/661277363 wait complete. 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 -- start start 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 Processor -- start 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f090d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f091260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56011f0941d0 con 0x56011f083740 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56011efa1580 con 0x56011f0840f0 2024-04-22T07:24:02.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56011f094350 con 0x56011f08d250 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f090d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f29c12740 10 client.0 _ll_register_callbacks cb 0x56011efa7650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f090d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42774/0 (socket says 172.21.15.113:42774) 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2473622711 learned_addr learned my addr 172.21.15.113:0/2473622711 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f091260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2473622711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 msgr2=0x56011f0917a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 msgr2=0x56011f091260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.994+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f091260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2473622711 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56011ee22f70 con 0x56011f083740 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f090d20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800b620 tx=0x7f1f1800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4559d4b307cc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1efffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1efffff640 1 -- 172.21.15.113:0/2473622711 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18029060 con 0x56011f083740 2024-04-22T07:24:03.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1efffff640 1 -- 172.21.15.113:0/2473622711 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f1f18005040 con 0x56011f083740 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f180041d0 con 0x56011f083740 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1efffff640 1 -- 172.21.15.113:0/2473622711 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1f18030050 con 0x56011f083740 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 10 client.17673 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56011ecba170 con 0x56011f083740 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1efffff640 1 -- 172.21.15.113:0/2473622711 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f180041d0 con 0x56011f083740 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1efffff640 1 -- 172.21.15.113:0/2473622711 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f1f18004750 con 0x56011f083740 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 10 client.17673 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 20 client.17673 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f0a1760 0x56011f0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 4 client.17673 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.995+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56011eff5920 con 0x56011f0a1760 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.996+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f0a1760 0x56011f0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.996+0000 7f1efffff640 10 client.17673 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.996+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f0a1760 0x56011f0a3b50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05b0b2bd10c24ee server_cookie=40e1fed066b96146 in_seq=0 out_seq=0 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1efffff640 1 -- 172.21.15.113:0/2473622711 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56011eff5920 con 0x56011f0a1760 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1efffff640 10 client.17673 handle_command_reply: tid=0 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 client.17673 shutdown 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 msgr2=0x56011f090d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f090d20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800b620 tx=0x7f1f1800bf20 comp rx=0 tx=0).stop 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 shutdown_connections 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f0a1760 0x56011f0a3b50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f091260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1efffff640 0 client.17673 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2473622711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f090d20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 >> 172.21.15.113:0/2473622711 conn(0x56011f07dfb0 msgr2=0x56011f07fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 shutdown_connections 2024-04-22T07:24:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 1 -- 172.21.15.113:0/2473622711 wait complete. 2024-04-22T07:24:03.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:02.997+0000 7f1f29c12740 20 client.17673 trim_cache size 0 max 16384 2024-04-22T07:24:03.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.014+0000 7f1f29c12740 1 Processor -- start 2024-04-22T07:24:03.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.014+0000 7f1f29c12740 1 -- start start 2024-04-22T07:24:03.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.014+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f093450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f09d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f08e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f29c12740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201ae600 con 0x56011f083740 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f29c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201abed0 con 0x56011f0840f0 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f29c12740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201a9d40 con 0x56011f08d250 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f093450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f093450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42788/0 (socket says 172.21.15.113:42788) 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2673336482 learned_addr learned my addr 172.21.15.113:0/2673336482 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2673336482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 msgr2=0x56011f08e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f08e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2673336482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 msgr2=0x56011f09d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f09d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 -- 172.21.15.113:0/2673336482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56011f08f290 con 0x56011f083740 2024-04-22T07:24:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f093450 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bc30 tx=0x7f1f180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5c1f800ac15472 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f167fc640 1 -- 172.21.15.113:0/2673336482 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18029030 con 0x56011f083740 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f167fc640 1 -- 172.21.15.113:0/2673336482 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f18004ec0 con 0x56011f083740 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.015+0000 7f1f167fc640 1 -- 172.21.15.113:0/2673336482 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18003850 con 0x56011f083740 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 -- 172.21.15.113:0/2673336482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 msgr2=0x56011f093450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f093450 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bc30 tx=0x7f1f180107a0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 -- 172.21.15.113:0/2673336482 shutdown_connections 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f08e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f0840f0 0x56011f09d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 --2- 172.21.15.113:0/2673336482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f083740 0x56011f093450 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 -- 172.21.15.113:0/2673336482 >> 172.21.15.113:0/2673336482 conn(0x56011f07dfb0 msgr2=0x56011f07c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 -- 172.21.15.113:0/2673336482 shutdown_connections 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 -- 172.21.15.113:0/2673336482 wait complete. 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 -- start start 2024-04-22T07:24:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.016+0000 7f1f29c12740 1 Processor -- start 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f083740 0x56011f0995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 0x56011f099b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201aa390 con 0x56011f0840f0 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201ab800 con 0x56011f083740 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1f201aaa60 con 0x56011f08d250 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 10 client.0 _ll_register_callbacks cb 0x7f1f1800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 0x56011f099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 0x56011f099b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42790/0 (socket says 172.21.15.113:42790) 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1182255645 learned_addr learned my addr 172.21.15.113:0/1182255645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f083740 0x56011f0995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f255c2640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1182255645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 msgr2=0x56011f0965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f083740 msgr2=0x56011f0995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f083740 0x56011f0995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 -- 172.21.15.113:0/1182255645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56011f096bf0 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f1ffff640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 0x56011f099b20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f10099620 tx=0x7f1f1009ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb2c1c824d7b7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1efffff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1efffff640 1 -- 172.21.15.113:0/1182255645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f100679f0 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1efffff640 1 -- 172.21.15.113:0/1182255645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f1f100ce680 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f18004ec0 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1efffff640 1 -- 172.21.15.113:0/1182255645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1f100ce880 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 10 client.17679 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.017+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56011f1083e0 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1efffff640 1 -- 172.21.15.113:0/1182255645 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1f18004ec0 con 0x56011f0840f0 2024-04-22T07:24:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1efffff640 1 -- 172.21.15.113:0/1182255645 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f1f1000f040 con 0x56011f0840f0 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f29c12740 10 client.17679 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f29c12740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f29c12740 20 client.17679 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f29c12740 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f108b10 0x56011f10af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f29c12740 4 client.17679 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56011efef790 con 0x56011f108b10 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f108b10 0x56011f10af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1f24dc1640 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f108b10 0x56011f10af30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4606cec939c9ef87 server_cookie=70aec78902a2846e in_seq=0 out_seq=0 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.018+0000 7f1efffff640 10 client.17679 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1efffff640 1 -- 172.21.15.113:0/1182255645 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56011efef790 con 0x56011f108b10 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1efffff640 10 client.17679 handle_command_reply: tid=0 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 client.17679 shutdown 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 msgr2=0x56011f099b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 0x56011f099b20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f10099620 tx=0x7f1f1009ca90 comp rx=0 tx=0).stop 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 shutdown_connections 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x56011f108b10 0x56011f10af30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56011f08d250 0x56011f0965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56011f0840f0 0x56011f099b20 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 --2- 172.21.15.113:0/1182255645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56011f083740 0x56011f0995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 >> 172.21.15.113:0/1182255645 conn(0x56011f07dfb0 msgr2=0x56011f091550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1efffff640 0 client.17679 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 shutdown_connections 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.019+0000 7f1f29c12740 1 -- 172.21.15.113:0/1182255645 wait complete. 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f29c12740 20 client.17679 trim_cache size 0 max 16384 2024-04-22T07:24:03.023 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:24:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:24:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "4d5042c3-a55d-488b-85d0-e0d6f91f85fb", 2024-04-22T07:24:03.024 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:24:03.024 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f0c0a3d20 msgr2=0x7f1f0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f0c0a3d20 0x7f1f0c0a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1f08006010 tx=0x7f1f08004270 comp rx=0 tx=0).stop 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 msgr2=0x7f1f2019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 0x7f1f2019e960 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000eab0 tx=0x7f1f1000ef70 comp rx=0 tx=0).stop 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 shutdown_connections 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1f0c0a3d20 0x7f1f0c0a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f2010c520 0x7f1f20198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f20103b00 0x7f1f2019e960 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 --2- 172.21.15.113:0/1134603517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f201031e0 0x7f1f2019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 >> 172.21.15.113:0/1134603517 conn(0x7f1f200771d0 msgr2=0x7f1f200fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 shutdown_connections 2024-04-22T07:24:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.020+0000 7f1f27025640 1 -- 172.21.15.113:0/1134603517 wait complete. 2024-04-22T07:24:03.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:24:03.028 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:24:03.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 -- 172.21.15.113:0/2306699572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4102b10 msgr2=0x7f83e4106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 --2- 172.21.15.113:0/2306699572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4102b10 0x7f83e4106b50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009900 tx=0x7f83d402bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 -- 172.21.15.113:0/2306699572 shutdown_connections 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 --2- 172.21.15.113:0/2306699572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e4107230 0x7f83e410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 --2- 172.21.15.113:0/2306699572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4102b10 0x7f83e4106b50 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 --2- 172.21.15.113:0/2306699572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83e4102160 0x7f83e4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 -- 172.21.15.113:0/2306699572 >> 172.21.15.113:0/2306699572 conn(0x7f83e40fda20 msgr2=0x7f83e40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 -- 172.21.15.113:0/2306699572 shutdown_connections 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.130+0000 7f83ec759640 1 -- 172.21.15.113:0/2306699572 wait complete. 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 Processor -- start 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 -- start start 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83e4102160 0x7f83e419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 0x7f83e419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e41985f0 0x7f83e4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4112290 con 0x7f83e4107230 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4112110 con 0x7f83e41985f0 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83ec759640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83e4112410 con 0x7f83e4102160 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83e9cf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 0x7f83e419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83e9cf4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 0x7f83e419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42808/0 (socket says 172.21.15.113:42808) 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83e9cf4640 1 -- 172.21.15.113:0/3762717833 learned_addr learned my addr 172.21.15.113:0/3762717833 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.131+0000 7f83eacf6640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e41985f0 0x7f83e4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83e9cf4640 1 -- 172.21.15.113:0/3762717833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83e4102160 msgr2=0x7f83e419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83e4102160 0x7f83e419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83e9cf4640 1 -- 172.21.15.113:0/3762717833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e41985f0 msgr2=0x7f83e4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e41985f0 0x7f83e4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83e9cf4640 1 -- 172.21.15.113:0/3762717833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e4199110 con 0x7f83e4107230 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 0x7f83e419e930 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000c00 tx=0x7f83d402cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83db7fe640 1 -- 172.21.15.113:0/3762717833 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4041070 con 0x7f83e4107230 2024-04-22T07:24:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83db7fe640 1 -- 172.21.15.113:0/3762717833 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f83d40044d0 con 0x7f83e4107230 2024-04-22T07:24:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e41993a0 con 0x7f83e4107230 2024-04-22T07:24:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83db7fe640 1 -- 172.21.15.113:0/3762717833 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4003d50 con 0x7f83e4107230 2024-04-22T07:24:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.132+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e41a55f0 con 0x7f83e4107230 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.134+0000 7f83db7fe640 1 -- 172.21.15.113:0/3762717833 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f83d4044070 con 0x7f83e4107230 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.134+0000 7f83ef42c740 1 Processor -- start 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.134+0000 7f83ef42c740 1 -- start start 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eae9fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 0x563eaea06540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea08e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ef42c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563eae9621e0 con 0x563eae9fdea0 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ef42c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563eae8ff390 con 0x563eae9fd580 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ef42c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563eae95e1a0 con 0x563eaea06a80 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 0x563eaea06540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 0x563eaea06540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42820/0 (socket says 172.21.15.113:42820) 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 -- 172.21.15.113:0/3212395111 learned_addr learned my addr 172.21.15.113:0/3212395111 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83eacf6640 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eae9fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83db7fe640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f83d00a3da0 0x7f83d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea08e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 -- 172.21.15.113:0/3212395111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 msgr2=0x563eaea08e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea08e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 -- 172.21.15.113:0/3212395111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 msgr2=0x563eae9fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eae9fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 -- 172.21.15.113:0/3212395111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eae73e7d0 con 0x563eae9fdea0 2024-04-22T07:24:03.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83eacf6640 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eae9fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83eacf6640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f83d00a3da0 0x7f83d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83db7fe640 1 -- 172.21.15.113:0/3762717833 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f83d40f22c0 con 0x7f83e4107230 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 0x563eaea06540 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f83e000aa90 tx=0x7f83e000af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8ea243c8447619 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83d8ff9640 1 -- 172.21.15.113:0/3212395111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e000f450 con 0x563eae9fdea0 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83d8ff9640 1 -- 172.21.15.113:0/3212395111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f83e000fac0 con 0x563eae9fdea0 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.135+0000 7f83d8ff9640 1 -- 172.21.15.113:0/3212395111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0018b00 con 0x563eae9fdea0 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 -- 172.21.15.113:0/3212395111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 msgr2=0x563eaea06540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83eacf6640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f83d00a3da0 0x7f83d00a6260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005fd0 tx=0x7f83dc004a40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 0x563eaea06540 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f83e000aa90 tx=0x7f83e000af50 comp rx=0 tx=0).stop 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 -- 172.21.15.113:0/3212395111 shutdown_connections 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea08e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fdea0 0x563eaea06540 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 --2- 172.21.15.113:0/3212395111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eae9fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 -- 172.21.15.113:0/3212395111 >> 172.21.15.113:0/3212395111 conn(0x563eae9f7cb0 msgr2=0x563eae9f8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 -- 172.21.15.113:0/3212395111 shutdown_connections 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 -- 172.21.15.113:0/3212395111 wait complete. 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.136+0000 7f83ef42c740 1 -- start start 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 Processor -- start 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eaea03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fdea0 0x563eaea04060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eaea045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563eaea09870 con 0x563eaea06a80 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563eae91b2f0 con 0x563eae9fd580 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563eaea099f0 con 0x563eae9fdea0 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ef42c740 10 client.0 _ll_register_callbacks cb 0x563eae928580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eaea045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eaea045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42828/0 (socket says 172.21.15.113:42828) 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 -- 172.21.15.113:0/1236236064 learned_addr learned my addr 172.21.15.113:0/1236236064 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fdea0 0x563eaea04060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eaea03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 -- 172.21.15.113:0/1236236064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fdea0 msgr2=0x563eaea04060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fdea0 0x563eaea04060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 -- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 msgr2=0x563eaea03b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eaea03b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 -- 172.21.15.113:0/1236236064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eae6743f0 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eaea03b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83eacf6640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eaea045a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009ed0 tx=0x7f83dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583dff943bb63f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.137+0000 7f83cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83cdffb640 1 -- 172.21.15.113:0/1236236064 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc01eb60 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83cdffb640 1 -- 172.21.15.113:0/1236236064 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f83dc01ed00 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e000fac0 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83cdffb640 1 -- 172.21.15.113:0/1236236064 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f83dc028470 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 10 client.17691 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563eae6340f0 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83cdffb640 1 -- 172.21.15.113:0/1236236064 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f83e000fac0 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83cdffb640 1 -- 172.21.15.113:0/1236236064 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f83dc00f040 con 0x563eaea06a80 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 10 client.17691 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:03.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 20 client.17691 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea16b30 0x563eaea18fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 4 client.17691 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eae96fc00 con 0x563eaea16b30 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.138+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea16b30 0x563eaea18fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.139+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea16b30 0x563eaea18fa0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c4373c40fc3fd7 server_cookie=3c1626f234561a88 in_seq=0 out_seq=0 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.139+0000 7f83cdffb640 10 client.17691 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.139+0000 7f83cdffb640 1 -- 172.21.15.113:0/1236236064 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563eae96fc00 con 0x563eaea16b30 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.139+0000 7f83cdffb640 10 client.17691 handle_command_reply: tid=0 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.139+0000 7f83ef42c740 1 client.17691 shutdown 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 msgr2=0x563eaea045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eaea045a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009ed0 tx=0x7f83dc004ab0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 shutdown_connections 2024-04-22T07:24:03.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea16b30 0x563eaea18fa0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eaea045a0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fdea0 0x563eaea04060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83cdffb640 0 client.17691 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 --2- 172.21.15.113:0/1236236064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fd580 0x563eaea03b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 >> 172.21.15.113:0/1236236064 conn(0x563eae9f7cb0 msgr2=0x563eae9f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 shutdown_connections 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 1 -- 172.21.15.113:0/1236236064 wait complete. 2024-04-22T07:24:03.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.140+0000 7f83ef42c740 20 client.17691 trim_cache size 0 max 16384 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 Processor -- start 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 -- start start 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fd580 0x563eaea06250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea12730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eae9f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41b08d0 con 0x563eaea06a80 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41ae1a0 con 0x563eae9fdea0 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ef42c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41ac010 con 0x563eae9fd580 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eae9f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eae9f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42840/0 (socket says 172.21.15.113:42840) 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 -- 172.21.15.113:0/2439930500 learned_addr learned my addr 172.21.15.113:0/2439930500 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fd580 0x563eaea06250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea12730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 -- 172.21.15.113:0/2439930500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fd580 msgr2=0x563eaea06250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fd580 0x563eaea06250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 -- 172.21.15.113:0/2439930500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 msgr2=0x563eaea12730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea12730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 -- 172.21.15.113:0/2439930500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eae9fdcd0 con 0x563eaea06a80 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fd580 0x563eaea06250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83eacf6640 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eae9f97f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005cc0 tx=0x7f83dc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a807ef5621c6ea68 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83cffff640 1 -- 172.21.15.113:0/2439930500 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc01f450 con 0x563eaea06a80 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.157+0000 7f83cffff640 1 -- 172.21.15.113:0/2439930500 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f83dc023070 con 0x563eaea06a80 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83cffff640 1 -- 172.21.15.113:0/2439930500 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc028c40 con 0x563eaea06a80 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 -- 172.21.15.113:0/2439930500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 msgr2=0x563eae9f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eae9f97f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005cc0 tx=0x7f83dc009770 comp rx=0 tx=0).stop 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 -- 172.21.15.113:0/2439930500 shutdown_connections 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eaea06a80 0x563eae9f97f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea12730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2439930500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eae9fd580 0x563eaea06250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 -- 172.21.15.113:0/2439930500 >> 172.21.15.113:0/2439930500 conn(0x563eae9f7cb0 msgr2=0x563eae9f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 -- 172.21.15.113:0/2439930500 shutdown_connections 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 -- 172.21.15.113:0/2439930500 wait complete. 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 -- start start 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.158+0000 7f83ef42c740 1 Processor -- start 2024-04-22T07:24:03.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 0x563eaea0a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea0ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea0b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41ac660 con 0x563eae9fd580 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41adad0 con 0x563eae9fdea0 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f83e41acd30 con 0x563eaea06a80 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 10 client.0 _ll_register_callbacks cb 0x7f83dc005bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 0x563eaea0a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 0x563eaea0a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42844/0 (socket says 172.21.15.113:42844) 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 -- 172.21.15.113:0/2736602797 learned_addr learned my addr 172.21.15.113:0/2736602797 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83eacf6640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea0b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea0ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 -- 172.21.15.113:0/2736602797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 msgr2=0x563eaea0b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea0b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 -- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 msgr2=0x563eaea0ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea0ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 -- 172.21.15.113:0/2736602797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eaea7c380 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ea4f5640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 0x563eaea0a920 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f83e0005b50 tx=0x7f83e0018760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a178a5a96c4b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83cdffb640 1 -- 172.21.15.113:0/2736602797 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0005220 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83cdffb640 1 -- 172.21.15.113:0/2736602797 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8943+0+0 (secure 0 0 0) 0x7f83e00053c0 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83dc023070 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83cdffb640 1 -- 172.21.15.113:0/2736602797 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f83e000a5a0 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 10 client.17697 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.159+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563eaea7d670 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83cdffb640 1 -- 172.21.15.113:0/2736602797 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f83dc023070 con 0x563eae9fd580 2024-04-22T07:24:03.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83cdffb640 1 -- 172.21.15.113:0/2736602797 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f83e0018aa0 con 0x563eae9fd580 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 10 client.17697 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 20 client.17697 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea7db10 0x563eaea7ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 4 client.17697 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563eae969a70 con 0x563eaea7db10 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea7db10 0x563eaea7ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83cdffb640 10 client.17697 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83e9cf4640 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea7db10 0x563eaea7ff00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a509a039a1d5046 server_cookie=a669f468d2b4d798 in_seq=0 out_seq=0 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83cdffb640 1 -- 172.21.15.113:0/2736602797 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563eae969a70 con 0x563eaea7db10 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83cdffb640 10 client.17697 handle_command_reply: tid=0 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.160+0000 7f83ef42c740 1 client.17697 shutdown 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 msgr2=0x563eaea0a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 0x563eaea0a920 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f83e0005b50 tx=0x7f83e0018760 comp rx=0 tx=0).stop 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 shutdown_connections 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563eaea7db10 0x563eaea7ff00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563eaea06a80 0x563eaea0b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83cdffb640 0 client.17697 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563eae9fdea0 0x563eaea0ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 --2- 172.21.15.113:0/2736602797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563eae9fd580 0x563eaea0a920 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 >> 172.21.15.113:0/2736602797 conn(0x563eae9f7cb0 msgr2=0x563eae9f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 shutdown_connections 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 1 -- 172.21.15.113:0/2736602797 wait complete. 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ef42c740 20 client.17697 trim_cache size 0 max 16384 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stdout: "4d5042c3-a55d-488b-85d0-e0d6f91f85fb": { 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:24:03.170 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "4d5042c3-a55d-488b-85d0-e0d6f91f85fb", 2024-04-22T07:24:03.171 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:24:03.171 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:24:03.171 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:24:03.171 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:24:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f83d00a3da0 msgr2=0x7f83d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f83d00a3da0 0x7f83d00a6260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f83dc005fd0 tx=0x7f83dc004a40 comp rx=0 tx=0).stop 2024-04-22T07:24:03.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 msgr2=0x7f83e419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 0x7f83e419e930 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000c00 tx=0x7f83d402cdf0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 shutdown_connections 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f83d00a3da0 0x7f83d00a6260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e41985f0 0x7f83e4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83e4107230 0x7f83e419e930 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 --2- 172.21.15.113:0/3762717833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83e4102160 0x7f83e419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 >> 172.21.15.113:0/3762717833 conn(0x7f83e40fda20 msgr2=0x7f83e40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 shutdown_connections 2024-04-22T07:24:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:03.161+0000 7f83ec759640 1 -- 172.21.15.113:0/3762717833 wait complete. 2024-04-22T07:24:03.173 INFO:tasks.cephfs.filesystem:scrub status for tag:4d5042c3-a55d-488b-85d0-e0d6f91f85fb - {'path': '/', 'tag': '4d5042c3-a55d-488b-85d0-e0d6f91f85fb', 'options': 'recursive,force'} 2024-04-22T07:24:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:03 smithi133 ceph-mon[31198]: pgmap v289: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T07:24:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:03 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:24:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:03 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:24:03.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:03 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:24:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:03 smithi154 ceph-mon[31862]: pgmap v289: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T07:24:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:03 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:24:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:03 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:24:03.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:03 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:24:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:03 smithi113 ceph-mon[27933]: pgmap v289: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T07:24:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:03 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:24:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:03 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:24:03.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:03 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:24:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:04 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:24:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:04 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:04 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:24:04.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:04 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:04 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:24:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:04 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:04 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:24:04.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:04 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:04.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:04 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:24:04.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:04 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:04.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:04 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:24:04.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:04 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:05.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:05 smithi133 ceph-mon[31198]: pgmap v290: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:24:05.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:05 smithi154 ceph-mon[31862]: pgmap v290: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:24:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:05 smithi113 ceph-mon[27933]: pgmap v290: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-22T07:24:06.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:06 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:24:06.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:06 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:24:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:06 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:24:07.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:07 smithi133 ceph-mon[31198]: pgmap v291: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:24:07.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:07 smithi133 ceph-mon[31198]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T07:24:07.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:07 smithi154 ceph-mon[31862]: pgmap v291: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:24:07.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:07 smithi154 ceph-mon[31862]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T07:24:07.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:07 smithi113 ceph-mon[27933]: pgmap v291: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T07:24:07.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:07 smithi113 ceph-mon[27933]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T07:24:09.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:09 smithi133 ceph-mon[31198]: pgmap v293: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:24:09.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:09 smithi154 ceph-mon[31862]: pgmap v293: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:24:09.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:09 smithi113 ceph-mon[27933]: pgmap v293: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:24:11.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:11 smithi133 ceph-mon[31198]: pgmap v294: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:24:11.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:11 smithi154 ceph-mon[31862]: pgmap v294: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:24:11.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:11 smithi113 ceph-mon[27933]: pgmap v294: 97 pgs: 97 active+clean; 262 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-22T07:24:13.564 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:13 smithi133 ceph-mon[31198]: pgmap v295: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:13.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:13 smithi154 ceph-mon[31862]: pgmap v295: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:13.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:13 smithi113 ceph-mon[27933]: pgmap v295: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:14 smithi133 ceph-mon[31198]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T07:24:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:14 smithi133 ceph-mon[31198]: pgmap v297: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:24:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:14 smithi154 ceph-mon[31862]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T07:24:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:14 smithi154 ceph-mon[31862]: pgmap v297: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:24:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:14 smithi113 ceph-mon[27933]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T07:24:14.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:14 smithi113 ceph-mon[27933]: pgmap v297: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:24:16.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:16 smithi133 ceph-mon[31198]: pgmap v298: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-22T07:24:16.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:16 smithi154 ceph-mon[31862]: pgmap v298: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-22T07:24:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:16 smithi113 ceph-mon[27933]: pgmap v298: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 219 B/s wr, 7 op/s 2024-04-22T07:24:18.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:18 smithi154 ceph-mon[31862]: pgmap v299: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:18.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:18 smithi113 ceph-mon[27933]: pgmap v299: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:19.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:18 smithi133 ceph-mon[31198]: pgmap v299: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:20.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:20 smithi113 ceph-mon[27933]: pgmap v300: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:21.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:20 smithi133 ceph-mon[31198]: pgmap v300: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:21.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:20 smithi154 ceph-mon[31862]: pgmap v300: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:24:22.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:22 smithi113 ceph-mon[27933]: pgmap v301: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:24:23.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:22 smithi133 ceph-mon[31198]: pgmap v301: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:24:23.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:22 smithi154 ceph-mon[31862]: pgmap v301: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:24:24.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:24 smithi113 ceph-mon[27933]: pgmap v302: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:24:25.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:24 smithi133 ceph-mon[31198]: pgmap v302: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:24:25.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:24 smithi154 ceph-mon[31862]: pgmap v302: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:24:26.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:26 smithi113 ceph-mon[27933]: pgmap v303: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:24:27.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:26 smithi133 ceph-mon[31198]: pgmap v303: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:24:27.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:26 smithi154 ceph-mon[31862]: pgmap v303: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:24:28.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:28 smithi113 ceph-mon[27933]: pgmap v304: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:29.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:28 smithi133 ceph-mon[31198]: pgmap v304: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:29.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:28 smithi154 ceph-mon[31862]: pgmap v304: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:30.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:30 smithi113 ceph-mon[27933]: pgmap v305: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:31.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:30 smithi133 ceph-mon[31198]: pgmap v305: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:31.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:30 smithi154 ceph-mon[31862]: pgmap v305: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:33.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:32 smithi133 ceph-mon[31198]: pgmap v306: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:33.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:32 smithi154 ceph-mon[31862]: pgmap v306: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:33.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:32 smithi113 ceph-mon[27933]: pgmap v306: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:33.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:24:33.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:24:33.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 -- 172.21.15.113:0/2539504482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc100d40 msgr2=0x7f32dc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 --2- 172.21.15.113:0/2539504482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc100d40 0x7f32dc101120 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009900 tx=0x7f32cc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 -- 172.21.15.113:0/2539504482 shutdown_connections 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 --2- 172.21.15.113:0/2539504482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc10e8f0 0x7f32dc110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 --2- 172.21.15.113:0/2539504482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32dc101660 0x7f32dc10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 --2- 172.21.15.113:0/2539504482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc100d40 0x7f32dc101120 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 -- 172.21.15.113:0/2539504482 >> 172.21.15.113:0/2539504482 conn(0x7f32dc078000 msgr2=0x7f32dc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 -- 172.21.15.113:0/2539504482 shutdown_connections 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 -- 172.21.15.113:0/2539504482 wait complete. 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 Processor -- start 2024-04-22T07:24:33.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.290+0000 7f32e3132640 1 -- start start 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32dc100d40 0x7f32dc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc101660 0x7f32dc1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 0x7f32dc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc114570 con 0x7f32dc10e8f0 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1143f0 con 0x7f32dc101660 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1146f0 con 0x7f32dc100d40 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 0x7f32dc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 0x7f32dc19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34220/0 (socket says 172.21.15.113:34220) 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 -- 172.21.15.113:0/676284847 learned_addr learned my addr 172.21.15.113:0/676284847 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e0ece640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32dc100d40 0x7f32dc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32dbfff640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc101660 0x7f32dc1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 -- 172.21.15.113:0/676284847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32dc100d40 msgr2=0x7f32dc1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32dc100d40 0x7f32dc1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 -- 172.21.15.113:0/676284847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc101660 msgr2=0x7f32dc1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc101660 0x7f32dc1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 -- 172.21.15.113:0/676284847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc19b020 con 0x7f32dc10e8f0 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32dbfff640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc101660 0x7f32dc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e16cf640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 0x7f32dc19a850 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b9e0 tx=0x7f32d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32d9ffb640 1 -- 172.21.15.113:0/676284847 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4019070 con 0x7f32dc10e8f0 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32d9ffb640 1 -- 172.21.15.113:0/676284847 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32d4014440 con 0x7f32dc10e8f0 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc19b310 con 0x7f32dc10e8f0 2024-04-22T07:24:33.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.291+0000 7f32d9ffb640 1 -- 172.21.15.113:0/676284847 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40125e0 con 0x7f32dc10e8f0 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.292+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc1a7530 con 0x7f32dc10e8f0 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32d9ffb640 1 -- 172.21.15.113:0/676284847 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f32d401c070 con 0x7f32dc10e8f0 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32d9ffb640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f32c40a3d20 0x7f32c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 Processor -- start 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 -- start start 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e0ece640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f32c40a3d20 0x7f32c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32d9ffb640 1 -- 172.21.15.113:0/676284847 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f32d40cfe60 con 0x7f32dc10e8f0 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144580 0x557be1144960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 0x557be114d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557be10a91e0 con 0x557be1144ea0 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557be1046390 con 0x557be114da80 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.294+0000 7f32e5dff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557be10a51a0 con 0x557be1144580 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 0x557be114d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 0x557be114d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34226/0 (socket says 172.21.15.113:34226) 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 -- 172.21.15.113:0/2549424078 learned_addr learned my addr 172.21.15.113:0/2549424078 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32e16cf640 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32e0ece640 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144580 0x557be1144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32e0ece640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f32c40a3d20 0x7f32c40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0096c0 tx=0x7f32cc02c750 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 -- 172.21.15.113:0/2549424078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144580 msgr2=0x557be1144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144580 0x557be1144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 -- 172.21.15.113:0/2549424078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 msgr2=0x557be114fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 -- 172.21.15.113:0/2549424078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557be0e857d0 con 0x557be1144ea0 2024-04-22T07:24:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32dbfff640 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 0x557be114d540 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f32d000bb10 tx=0x7f32d000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35439e6bff9dc927 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32c2ffd640 1 -- 172.21.15.113:0/2549424078 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0017420 con 0x557be1144ea0 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32c2ffd640 1 -- 172.21.15.113:0/2549424078 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32d0017a90 con 0x557be1144ea0 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.295+0000 7f32c2ffd640 1 -- 172.21.15.113:0/2549424078 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d000fb60 con 0x557be1144ea0 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 -- 172.21.15.113:0/2549424078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 msgr2=0x557be114d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 0x557be114d540 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f32d000bb10 tx=0x7f32d000d340 comp rx=0 tx=0).stop 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 -- 172.21.15.113:0/2549424078 shutdown_connections 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144ea0 0x557be114d540 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 --2- 172.21.15.113:0/2549424078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144580 0x557be1144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 -- 172.21.15.113:0/2549424078 >> 172.21.15.113:0/2549424078 conn(0x557be113ecb0 msgr2=0x557be113f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 -- 172.21.15.113:0/2549424078 shutdown_connections 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 -- 172.21.15.113:0/2549424078 wait complete. 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 -- start start 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.296+0000 7f32e5dff740 1 Processor -- start 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144ea0 0x557be114afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557be1150870 con 0x557be1144580 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557be10622f0 con 0x557be114da80 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557be11509f0 con 0x557be1144ea0 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e5dff740 10 client.0 _ll_register_callbacks cb 0x557be106f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34230/0 (socket says 172.21.15.113:34230) 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 -- 172.21.15.113:0/3392440626 learned_addr learned my addr 172.21.15.113:0/3392440626 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144ea0 0x557be114afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e16cf640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 -- 172.21.15.113:0/3392440626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144ea0 msgr2=0x557be114afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144ea0 0x557be114afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 -- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 msgr2=0x557be114b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 -- 172.21.15.113:0/3392440626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557be0dbb3f0 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144ea0 0x557be114afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e16cf640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32e0ece640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114aaa0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f32cc035000 tx=0x7f32cc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fe1c2ac3e13f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.297+0000 7f32bbfff640 1 -- 172.21.15.113:0/3392440626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc050020 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32bbfff640 1 -- 172.21.15.113:0/3392440626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f32cc004ad0 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d0017a90 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32bbfff640 1 -- 172.21.15.113:0/3392440626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f32cc05b050 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 10 client.17709 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557be0d7b0f0 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32bbfff640 1 -- 172.21.15.113:0/3392440626 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32d0017a90 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32bbfff640 1 -- 172.21.15.113:0/3392440626 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f32cc04c480 con 0x557be1144580 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 10 client.17709 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 20 client.17709 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be115daf0 0x557be115ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 4 client.17709 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557be10b6c00 con 0x557be115daf0 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.298+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be115daf0 0x557be115ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.299+0000 7f32bbfff640 10 client.17709 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.299+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be115daf0 0x557be115ff20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b909d7578630f94 server_cookie=a756d979d066c275 in_seq=0 out_seq=0 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32bbfff640 1 -- 172.21.15.113:0/3392440626 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557be10b6c00 con 0x557be115daf0 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32bbfff640 10 client.17709 handle_command_reply: tid=0 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 client.17709 shutdown 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 msgr2=0x557be114aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114aaa0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f32cc035000 tx=0x7f32cc02bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 shutdown_connections 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be115daf0 0x557be115ff20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be114da80 0x557be114b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be1144ea0 0x557be114afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32bbfff640 0 client.17709 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3392440626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114aaa0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 >> 172.21.15.113:0/3392440626 conn(0x557be113ecb0 msgr2=0x557be1140a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 shutdown_connections 2024-04-22T07:24:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 1 -- 172.21.15.113:0/3392440626 wait complete. 2024-04-22T07:24:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.300+0000 7f32e5dff740 20 client.17709 trim_cache size 0 max 16384 2024-04-22T07:24:33.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 Processor -- start 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 -- start start 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be11499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1159730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1b2950 con 0x557be1144580 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1b0220 con 0x557be1144ea0 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e5dff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1ae090 con 0x557be114da80 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e0ece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e0ece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34246/0 (socket says 172.21.15.113:34246) 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.317+0000 7f32e0ece640 1 -- 172.21.15.113:0/293543989 learned_addr learned my addr 172.21.15.113:0/293543989 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32dbfff640 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be11499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e16cf640 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e0ece640 1 -- 172.21.15.113:0/293543989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 msgr2=0x557be1159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e0ece640 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e0ece640 1 -- 172.21.15.113:0/293543989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 msgr2=0x557be11499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e0ece640 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be11499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e0ece640 1 -- 172.21.15.113:0/293543989 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557be1148ff0 con 0x557be1144580 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32e0ece640 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114d1d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f32cc04c8f0 tx=0x7f32cc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fda8958699a430 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32c27fc640 1 -- 172.21.15.113:0/293543989 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc050040 con 0x557be1144580 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32c27fc640 1 -- 172.21.15.113:0/293543989 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32cc046040 con 0x557be1144580 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.318+0000 7f32c27fc640 1 -- 172.21.15.113:0/293543989 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc004d00 con 0x557be1144580 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- 172.21.15.113:0/293543989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 msgr2=0x557be114d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114d1d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f32cc04c8f0 tx=0x7f32cc005bd0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- 172.21.15.113:0/293543989 shutdown_connections 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be11499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- 172.21.15.113:0/293543989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be114d1d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- 172.21.15.113:0/293543989 >> 172.21.15.113:0/293543989 conn(0x557be113ecb0 msgr2=0x557be113dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- 172.21.15.113:0/293543989 shutdown_connections 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- 172.21.15.113:0/293543989 wait complete. 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- start start 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 Processor -- start 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be1155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be1156180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1152c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1ae6e0 con 0x557be1144580 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1afb50 con 0x557be1144ea0 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1aedb0 con 0x557be114da80 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e5dff740 10 client.0 _ll_register_callbacks cb 0x7f32cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.319+0000 7f32e0ece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be1155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be1155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34248/0 (socket says 172.21.15.113:34248) 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 -- 172.21.15.113:0/3319389005 learned_addr learned my addr 172.21.15.113:0/3319389005 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be1156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e16cf640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 -- 172.21.15.113:0/3319389005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 msgr2=0x557be1152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 -- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 msgr2=0x557be1156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be1156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 -- 172.21.15.113:0/3319389005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557be1153420 con 0x557be1144580 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e0ece640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be1155c40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f32cc02bc70 tx=0x7f32cc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d553d7d011af3d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32bbfff640 1 -- 172.21.15.113:0/3319389005 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc06d060 con 0x557be1144580 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32bbfff640 1 -- 172.21.15.113:0/3319389005 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f32cc004a40 con 0x557be1144580 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32cc046040 con 0x557be1144580 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32bbfff640 1 -- 172.21.15.113:0/3319389005 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f32cc050040 con 0x557be1144580 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e5dff740 10 client.17715 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557be11c4b80 con 0x557be1144580 2024-04-22T07:24:33.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.320+0000 7f32bbfff640 1 -- 172.21.15.113:0/3319389005 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32cc046040 con 0x557be1144580 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32bbfff640 1 -- 172.21.15.113:0/3319389005 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f32cc003d20 con 0x557be1144580 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32e5dff740 10 client.17715 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32e5dff740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32e5dff740 20 client.17715 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be11c5190 0x557be11c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32e5dff740 4 client.17715 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557be10b0a70 con 0x557be11c5190 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be11c5190 0x557be11c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32dbfff640 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be11c5190 0x557be11c7580 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbde05b3ab1c340d server_cookie=f9e6709c28407536 in_seq=0 out_seq=0 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32bbfff640 10 client.17715 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.321+0000 7f32bbfff640 1 -- 172.21.15.113:0/3319389005 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557be10b0a70 con 0x557be11c5190 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32bbfff640 10 client.17715 handle_command_reply: tid=0 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 client.17715 shutdown 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 msgr2=0x557be1155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be1155c40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f32cc02bc70 tx=0x7f32cc005c90 comp rx=0 tx=0).stop 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 shutdown_connections 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x557be11c5190 0x557be11c7580 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557be114da80 0x557be1152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557be1144ea0 0x557be1156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32bbfff640 0 client.17715 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 --2- 172.21.15.113:0/3319389005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557be1144580 0x557be1155c40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 >> 172.21.15.113:0/3319389005 conn(0x557be113ecb0 msgr2=0x557be113d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 shutdown_connections 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 1 -- 172.21.15.113:0/3319389005 wait complete. 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e5dff740 20 client.17715 trim_cache size 0 max 16384 2024-04-22T07:24:33.326 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:24:33.327 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:24:33.327 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:24:33.327 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:24:33.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f32c40a3d20 msgr2=0x7f32c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f32c40a3d20 0x7f32c40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0096c0 tx=0x7f32cc02c750 comp rx=0 tx=0).stop 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 msgr2=0x7f32dc19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 0x7f32dc19a850 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b9e0 tx=0x7f32d400bea0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 shutdown_connections 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f32c40a3d20 0x7f32c40a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32dc10e8f0 0x7f32dc19a850 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32dc101660 0x7f32dc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 --2- 172.21.15.113:0/676284847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32dc100d40 0x7f32dc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.322+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 >> 172.21.15.113:0/676284847 conn(0x7f32dc078000 msgr2=0x7f32dc10fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.323+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 shutdown_connections 2024-04-22T07:24:33.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.323+0000 7f32e3132640 1 -- 172.21.15.113:0/676284847 wait complete. 2024-04-22T07:24:33.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:24:33.331 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 -- 172.21.15.113:0/3981896471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc101f50 msgr2=0x7f76cc1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 --2- 172.21.15.113:0/3981896471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc101f50 0x7f76cc1023b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009900 tx=0x7f76b402bcd0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 -- 172.21.15.113:0/3981896471 shutdown_connections 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 --2- 172.21.15.113:0/3981896471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc1028f0 0x7f76cc10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 --2- 172.21.15.113:0/3981896471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc101f50 0x7f76cc1023b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 --2- 172.21.15.113:0/3981896471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 0x7f76cc1083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 -- 172.21.15.113:0/3981896471 >> 172.21.15.113:0/3981896471 conn(0x7f76cc0793e0 msgr2=0x7f76cc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 -- 172.21.15.113:0/3981896471 shutdown_connections 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 -- 172.21.15.113:0/3981896471 wait complete. 2024-04-22T07:24:33.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.434+0000 7f76d1031640 1 Processor -- start 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 -- start start 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc101f50 0x7f76cc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 0x7f76cc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 0x7f76cc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1122c0 con 0x7f76cc1028f0 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc112140 con 0x7f76cc101f50 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76d1031640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc112440 con 0x7f76cc108010 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 0x7f76cc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 0x7f76cc198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34258/0 (socket says 172.21.15.113:34258) 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 -- 172.21.15.113:0/3544217365 learned_addr learned my addr 172.21.15.113:0/3544217365 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76cad9d640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc101f50 0x7f76cc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76cb59e640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 0x7f76cc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 -- 172.21.15.113:0/3544217365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 msgr2=0x7f76cc19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 0x7f76cc19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 -- 172.21.15.113:0/3544217365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc101f50 msgr2=0x7f76cc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc101f50 0x7f76cc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 -- 172.21.15.113:0/3544217365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc19d6f0 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76cb59e640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 0x7f76cc19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.435+0000 7f76ca59c640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 0x7f76cc198c40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f76b4006010 tx=0x7f76b402ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.436+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4043020 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.436+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f76b40042d0 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.436+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76cc19d980 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.436+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4003b20 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.436+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76cc1a5270 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.437+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76cc068de0 con 0x7f76cc1028f0 2024-04-22T07:24:33.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.438+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f76b4046020 con 0x7f76cc1028f0 2024-04-22T07:24:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.438+0000 7f76c3fff640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f76b80a3d20 0x7f76b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.438+0000 7f76cad9d640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f76b80a3d20 0x7f76b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.438+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f76b40f3020 con 0x7f76cc1028f0 2024-04-22T07:24:33.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.439+0000 7f76cad9d640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f76b80a3d20 0x7f76b80a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76c4007ec0 tx=0x7f76c40075b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.442+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76b404d020 con 0x7f76cc1028f0 2024-04-22T07:24:33.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.662+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76cc1a54a0 con 0x7f76cc1028f0 2024-04-22T07:24:33.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.663+0000 7f76c3fff640 1 -- 172.21.15.113:0/3544217365 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7f76b40bf6c0 con 0x7f76cc1028f0 2024-04-22T07:24:33.663 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:24:33.664 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f76b80a3d20 msgr2=0x7f76b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f76b80a3d20 0x7f76b80a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76c4007ec0 tx=0x7f76c40075b0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 msgr2=0x7f76cc198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 0x7f76cc198c40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f76b4006010 tx=0x7f76b402ce90 comp rx=0 tx=0).stop 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 shutdown_connections 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f76b80a3d20 0x7f76b80a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76cc108010 0x7f76cc19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76cc1028f0 0x7f76cc198c40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 --2- 172.21.15.113:0/3544217365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc101f50 0x7f76cc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 >> 172.21.15.113:0/3544217365 conn(0x7f76cc0793e0 msgr2=0x7f76cc1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 shutdown_connections 2024-04-22T07:24:33.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.665+0000 7f76d1031640 1 -- 172.21.15.113:0/3544217365 wait complete. 2024-04-22T07:24:33.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:24:33.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 -- 172.21.15.113:0/4104631407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 msgr2=0x7f13e810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 --2- 172.21.15.113:0/4104631407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e810dc80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009cd0 tx=0x7f13e002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 -- 172.21.15.113:0/4104631407 shutdown_connections 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 --2- 172.21.15.113:0/4104631407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e810dc80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 --2- 172.21.15.113:0/4104631407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e81011a0 0x7f13e8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 --2- 172.21.15.113:0/4104631407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13e8107030 0x7f13e8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 -- 172.21.15.113:0/4104631407 >> 172.21.15.113:0/4104631407 conn(0x7f13e8076c50 msgr2=0x7f13e8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 -- 172.21.15.113:0/4104631407 shutdown_connections 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 -- 172.21.15.113:0/4104631407 wait complete. 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.776+0000 7f13ed081640 1 Processor -- start 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 -- start start 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13e81011a0 0x7f13e819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e8107030 0x7f13e81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8111260 con 0x7f13e8101b40 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81110e0 con 0x7f13e8107030 2024-04-22T07:24:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81113e0 con 0x7f13e81011a0 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34286/0 (socket says 172.21.15.113:34286) 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 -- 172.21.15.113:0/3673965915 learned_addr learned my addr 172.21.15.113:0/3673965915 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13e81011a0 0x7f13e819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13e759e640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e8107030 0x7f13e81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 -- 172.21.15.113:0/3673965915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13e81011a0 msgr2=0x7f13e819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13e81011a0 0x7f13e819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 -- 172.21.15.113:0/3673965915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e8107030 msgr2=0x7f13e81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e8107030 0x7f13e81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 -- 172.21.15.113:0/3673965915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e8198d70 con 0x7f13e8101b40 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13e759e640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e8107030 0x7f13e81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13dffff640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e819e980 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f13d000d950 tx=0x7f13d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13e4d99640 1 -- 172.21.15.113:0/3673965915 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0017030 con 0x7f13e8101b40 2024-04-22T07:24:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13e4d99640 1 -- 172.21.15.113:0/3673965915 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f13d0004020 con 0x7f13e8101b40 2024-04-22T07:24:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e8199060 con 0x7f13e8101b40 2024-04-22T07:24:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.777+0000 7f13e4d99640 1 -- 172.21.15.113:0/3673965915 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00047a0 con 0x7f13e8101b40 2024-04-22T07:24:33.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.778+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e81a5300 con 0x7f13e8101b40 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.780+0000 7f13e4d99640 1 -- 172.21.15.113:0/3673965915 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f13d00041d0 con 0x7f13e8101b40 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.780+0000 7f13efc76740 1 Processor -- start 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.780+0000 7f13efc76740 1 -- start start 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13efc76740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 0x5641b4576710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13efc76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b457f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13efc76740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b45802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13efc76740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5641b44db1f0 con 0x5641b4576330 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13efc76740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5641b44784b0 con 0x5641b457fe40 2024-04-22T07:24:33.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13efc76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5641b44d71b0 con 0x5641b4576ce0 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 0x5641b4576710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 0x5641b4576710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34300/0 (socket says 172.21.15.113:34300) 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 -- 172.21.15.113:0/4189542866 learned_addr learned my addr 172.21.15.113:0/4189542866 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b457f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e4d99640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f13d40a3da0 0x7f13d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13dffff640 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b45802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 -- 172.21.15.113:0/4189542866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 msgr2=0x5641b457f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b457f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 -- 172.21.15.113:0/4189542866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 msgr2=0x5641b45802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b45802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 -- 172.21.15.113:0/4189542866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b446a1b0 con 0x5641b4576330 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e4d99640 1 -- 172.21.15.113:0/3673965915 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f13d00d0380 con 0x7f13e8101b40 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f13d40a3da0 0x7f13d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.781+0000 7f13e759e640 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 0x5641b4576710 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f13e002c700 tx=0x7f13e00324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a456a883db9bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13dd7fa640 1 -- 172.21.15.113:0/4189542866 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0039040 con 0x5641b4576330 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13dd7fa640 1 -- 172.21.15.113:0/4189542866 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f13e0003c30 con 0x5641b4576330 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13e759e640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f13d40a3da0 0x7f13d40a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f13e004b000 tx=0x7f13e0044000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13dd7fa640 1 -- 172.21.15.113:0/4189542866 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00346a0 con 0x5641b4576330 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 -- 172.21.15.113:0/4189542866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 msgr2=0x5641b4576710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 0x5641b4576710 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f13e002c700 tx=0x7f13e00324b0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 -- 172.21.15.113:0/4189542866 shutdown_connections 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b45802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b457f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 --2- 172.21.15.113:0/4189542866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576330 0x5641b4576710 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.782+0000 7f13efc76740 1 -- 172.21.15.113:0/4189542866 >> 172.21.15.113:0/4189542866 conn(0x5641b4570ba0 msgr2=0x5641b4570f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 -- 172.21.15.113:0/4189542866 shutdown_connections 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 -- 172.21.15.113:0/4189542866 wait complete. 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 -- start start 2024-04-22T07:24:33.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 Processor -- start 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b4586210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576ce0 0x5641b45835a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b4583ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5641b4586dc0 con 0x5641b457fe40 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5641b4494500 con 0x5641b4576ce0 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5641b4586f40 con 0x5641b4576330 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13efc76740 10 client.0 _ll_register_callbacks cb 0x5641b4462b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13e759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b4583ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13e759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b4583ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34316/0 (socket says 172.21.15.113:34316) 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13e759e640 1 -- 172.21.15.113:0/1770805706 learned_addr learned my addr 172.21.15.113:0/1770805706 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b4586210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.783+0000 7f13dffff640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576ce0 0x5641b45835a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e759e640 1 -- 172.21.15.113:0/1770805706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 msgr2=0x5641b4586210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e759e640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b4586210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e759e640 1 -- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576ce0 msgr2=0x5641b45835a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e759e640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576ce0 0x5641b45835a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e759e640 1 -- 172.21.15.113:0/1770805706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b41ed3f0 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b4586210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13dffff640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576ce0 0x5641b45835a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13e759e640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b4583ae0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f13e0006010 tx=0x7f13e0035f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b557252606cf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13ce7fc640 1 -- 172.21.15.113:0/1770805706 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0058020 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13ce7fc640 1 -- 172.21.15.113:0/1770805706 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f13e0034440 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e0003c30 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13ce7fc640 1 -- 172.21.15.113:0/1770805706 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f13e0063050 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 10 client.17733 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5641b42b77d0 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13ce7fc640 1 -- 172.21.15.113:0/1770805706 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f13e0003c30 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13ce7fc640 1 -- 172.21.15.113:0/1770805706 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f13e0053070 con 0x5641b457fe40 2024-04-22T07:24:33.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 10 client.17733 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 20 client.17733 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b4593ed0 0x5641b45962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 4 client.17733 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.784+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b447a930 con 0x5641b4593ed0 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.785+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b4593ed0 0x5641b45962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.785+0000 7f13ce7fc640 10 client.17733 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.785+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b4593ed0 0x5641b45962c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d2b28f6247715a server_cookie=a8a3beb209ecefa0 in_seq=0 out_seq=0 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13ce7fc640 1 -- 172.21.15.113:0/1770805706 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641b447a930 con 0x5641b4593ed0 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13ce7fc640 10 client.17733 handle_command_reply: tid=0 2024-04-22T07:24:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 client.17733 shutdown 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 msgr2=0x5641b4583ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b4583ae0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f13e0006010 tx=0x7f13e0035f00 comp rx=0 tx=0).stop 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 shutdown_connections 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b4593ed0 0x5641b45962c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b4583ae0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576ce0 0x5641b45835a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13ce7fc640 0 client.17733 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 --2- 172.21.15.113:0/1770805706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b4586210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 >> 172.21.15.113:0/1770805706 conn(0x5641b4570ba0 msgr2=0x5641b45728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 shutdown_connections 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 1 -- 172.21.15.113:0/1770805706 wait complete. 2024-04-22T07:24:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.786+0000 7f13efc76740 20 client.17733 trim_cache size 0 max 16384 2024-04-22T07:24:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 Processor -- start 2024-04-22T07:24:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 -- start start 2024-04-22T07:24:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b458fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 0x5641b456f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b4585cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81ae540 con 0x5641b4576ce0 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81abe10 con 0x5641b457fe40 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.799+0000 7f13efc76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81a9ca0 con 0x5641b4576330 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 0x5641b456f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 0x5641b456f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34332/0 (socket says 172.21.15.113:34332) 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 -- 172.21.15.113:0/3828367439 learned_addr learned my addr 172.21.15.113:0/3828367439 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b458fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 -- 172.21.15.113:0/3828367439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 msgr2=0x5641b458fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b458fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 -- 172.21.15.113:0/3828367439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 msgr2=0x5641b4585cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b4585cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 -- 172.21.15.113:0/3828367439 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b457ea20 con 0x5641b4576ce0 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dffff640 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 0x5641b456f370 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f13d0098590 tx=0x7f13d009bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4753ead18f6c64d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dcff9640 1 -- 172.21.15.113:0/3828367439 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d009c880 con 0x5641b4576ce0 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dcff9640 1 -- 172.21.15.113:0/3828367439 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f13d00d01a0 con 0x5641b4576ce0 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.800+0000 7f13dcff9640 1 -- 172.21.15.113:0/3828367439 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0097ed0 con 0x5641b4576ce0 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- 172.21.15.113:0/3828367439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 msgr2=0x5641b456f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 0x5641b456f370 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f13d0098590 tx=0x7f13d009bf50 comp rx=0 tx=0).stop 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- 172.21.15.113:0/3828367439 shutdown_connections 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b457fe40 0x5641b4585cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b4576ce0 0x5641b456f370 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- 172.21.15.113:0/3828367439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576330 0x5641b458fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- 172.21.15.113:0/3828367439 >> 172.21.15.113:0/3828367439 conn(0x5641b4570ba0 msgr2=0x5641b457e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- 172.21.15.113:0/3828367439 shutdown_connections 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- 172.21.15.113:0/3828367439 wait complete. 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- start start 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 Processor -- start 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576330 0x5641b4589d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b458a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b458a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81aa2f0 con 0x5641b457fe40 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81ab740 con 0x5641b4576330 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81aa9a0 con 0x5641b4576ce0 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.801+0000 7f13efc76740 10 client.0 _ll_register_callbacks cb 0x7f13e0035da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b458a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b458a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34344/0 (socket says 172.21.15.113:34344) 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 -- 172.21.15.113:0/4040542930 learned_addr learned my addr 172.21.15.113:0/4040542930 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576330 0x5641b4589d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13dffff640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b458a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 -- 172.21.15.113:0/4040542930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 msgr2=0x5641b458a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b458a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 -- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576330 msgr2=0x5641b4589d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576330 0x5641b4589d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 -- 172.21.15.113:0/4040542930 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b45f9210 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13dffff640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b458a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13e759e640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b458a790 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f13e0054520 tx=0x7f13e0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52208274e028efa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13ce7fc640 1 -- 172.21.15.113:0/4040542930 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0058040 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13ce7fc640 1 -- 172.21.15.113:0/4040542930 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f13e0034680 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d00d01a0 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13ce7fc640 1 -- 172.21.15.113:0/4040542930 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f13e006f050 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13efc76740 10 client.17739 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5641b45fa510 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.802+0000 7f13ce7fc640 1 -- 172.21.15.113:0/4040542930 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f13d00d01a0 con 0x5641b457fe40 2024-04-22T07:24:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13ce7fc640 1 -- 172.21.15.113:0/4040542930 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f13e0053070 con 0x5641b457fe40 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 10 client.17739 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 20 client.17739 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b45fab40 0x5641b45fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 4 client.17739 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641b44e8c10 con 0x5641b45fab40 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b45fab40 0x5641b45fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13ce7fc640 10 client.17739 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13e6d9d640 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b45fab40 0x5641b45fcf60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17aa14f45fe3861a server_cookie=102ebce5182c1b72 in_seq=0 out_seq=0 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13ce7fc640 1 -- 172.21.15.113:0/4040542930 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641b44e8c10 con 0x5641b45fab40 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13ce7fc640 10 client.17739 handle_command_reply: tid=0 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.803+0000 7f13efc76740 1 client.17739 shutdown 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 msgr2=0x5641b458a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b458a790 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f13e0054520 tx=0x7f13e0005bd0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 shutdown_connections 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5641b45fab40 0x5641b45fcf60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5641b457fe40 0x5641b458a790 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ce7fc640 0 client.17739 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5641b4576ce0 0x5641b458a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 --2- 172.21.15.113:0/4040542930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5641b4576330 0x5641b4589d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 >> 172.21.15.113:0/4040542930 conn(0x5641b4570ba0 msgr2=0x5641b4571c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 shutdown_connections 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 1 -- 172.21.15.113:0/4040542930 wait complete. 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13efc76740 20 client.17739 trim_cache size 0 max 16384 2024-04-22T07:24:33.808 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:24:33.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f13d40a3da0 msgr2=0x7f13d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f13d40a3da0 0x7f13d40a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f13e004b000 tx=0x7f13e0044000 comp rx=0 tx=0).stop 2024-04-22T07:24:33.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 msgr2=0x7f13e819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e819e980 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f13d000d950 tx=0x7f13d000de10 comp rx=0 tx=0).stop 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 shutdown_connections 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f13d40a3da0 0x7f13d40a6260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13e8107030 0x7f13e81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e8101b40 0x7f13e819e980 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 --2- 172.21.15.113:0/3673965915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13e81011a0 0x7f13e819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 >> 172.21.15.113:0/3673965915 conn(0x7f13e8076c50 msgr2=0x7f13e8105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 shutdown_connections 2024-04-22T07:24:33.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.804+0000 7f13ed081640 1 -- 172.21.15.113:0/3673965915 wait complete. 2024-04-22T07:24:33.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 -- 172.21.15.113:0/3691021200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c1010a0 msgr2=0x7f5e2c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 --2- 172.21.15.113:0/3691021200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c101480 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009900 tx=0x7f5e2002bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 -- 172.21.15.113:0/3691021200 shutdown_connections 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 --2- 172.21.15.113:0/3691021200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c106170 0x7f5e2c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 --2- 172.21.15.113:0/3691021200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c101a50 0x7f5e2c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 --2- 172.21.15.113:0/3691021200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c101480 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 -- 172.21.15.113:0/3691021200 >> 172.21.15.113:0/3691021200 conn(0x7f5e2c077eb0 msgr2=0x7f5e2c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 -- 172.21.15.113:0/3691021200 shutdown_connections 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.914+0000 7f5e310e4640 1 -- 172.21.15.113:0/3691021200 wait complete. 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 Processor -- start 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 -- start start 2024-04-22T07:24:33.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 0x7f5e2c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c19a140 0x7f5e2c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c111210 con 0x7f5e2c106170 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c111090 con 0x7f5e2c19a140 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.915+0000 7f5e310e4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c111390 con 0x7f5e2c1010a0 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 0x7f5e2c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 0x7f5e2c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34368/0 (socket says 172.21.15.113:34368) 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1547516114 learned_addr learned my addr 172.21.15.113:0/1547516114 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c19a140 0x7f5e2c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1547516114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c1010a0 msgr2=0x7f5e2c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1547516114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c19a140 msgr2=0x7f5e2c1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c19a140 0x7f5e2c1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1547516114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c1a4ec0 con 0x7f5e2c106170 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 0x7f5e2c199bb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c970 tx=0x7f5e1c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e13fff640 1 -- 172.21.15.113:0/1547516114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c014070 con 0x7f5e2c106170 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c1a51b0 con 0x7f5e2c106170 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e13fff640 1 -- 172.21.15.113:0/1547516114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5e1c004440 con 0x7f5e2c106170 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e13fff640 1 -- 172.21.15.113:0/1547516114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c003e40 con 0x7f5e2c106170 2024-04-22T07:24:33.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.916+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c1a5750 con 0x7f5e2c106170 2024-04-22T07:24:33.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.918+0000 7f5e13fff640 1 -- 172.21.15.113:0/1547516114 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f5e1c017070 con 0x7f5e2c106170 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 Processor -- start 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 -- start start 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e13fff640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5e140a3d20 0x7f5e140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7791fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e779205430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779207e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e7791611f0 con 0x55e779205a00 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e7790fe4b0 con 0x55e7791fc4d0 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e33db3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e77915d1b0 con 0x55e7791fce80 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5e140a3d20 0x7f5e140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e13fff640 1 -- 172.21.15.113:0/1547516114 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f5e1c0d0810 con 0x7f5e2c106170 2024-04-22T07:24:33.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779207e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7791fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779207e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34382/0 (socket says 172.21.15.113:34382) 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2b59e640 1 -- 172.21.15.113:0/2282360474 learned_addr learned my addr 172.21.15.113:0/2282360474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e779205430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/2282360474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 msgr2=0x55e779205430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e779205430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/2282360474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 msgr2=0x55e779207e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779207e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/2282360474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7790f01b0 con 0x55e7791fc4d0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.919+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779207e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.920+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5e140a3d20 0x7f5e140a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e24014be0 tx=0x7f5e2400adf0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.920+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7791fc8b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e20036040 tx=0x7f5e2002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a93d3e0543d07d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.920+0000 7f5e117fa640 1 -- 172.21.15.113:0/2282360474 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2002cb90 con 0x55e7791fc4d0 2024-04-22T07:24:33.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.920+0000 7f5e117fa640 1 -- 172.21.15.113:0/2282360474 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5e20005040 con 0x55e7791fc4d0 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.920+0000 7f5e117fa640 1 -- 172.21.15.113:0/2282360474 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e200049f0 con 0x55e7791fc4d0 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- 172.21.15.113:0/2282360474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 msgr2=0x55e7791fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7791fc8b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e20036040 tx=0x7f5e2002c650 comp rx=0 tx=0).stop 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- 172.21.15.113:0/2282360474 shutdown_connections 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779207e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e779205430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- 172.21.15.113:0/2282360474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7791fc8b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- 172.21.15.113:0/2282360474 >> 172.21.15.113:0/2282360474 conn(0x55e7791f6ba0 msgr2=0x55e7791f6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- 172.21.15.113:0/2282360474 shutdown_connections 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- 172.21.15.113:0/2282360474 wait complete. 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- start start 2024-04-22T07:24:33.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 Processor -- start 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7792052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 0x55e779202670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779202bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e779208910 con 0x55e7791fce80 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e77911a500 con 0x55e7791fc4d0 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e779208a90 con 0x55e779205a00 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e33db3740 10 client.0 _ll_register_callbacks cb 0x55e7790e8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 0x55e779202670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e2a59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 0x55e779202670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34398/0 (socket says 172.21.15.113:34398) 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1839906860 learned_addr learned my addr 172.21.15.113:0/1839906860 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779202bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.921+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7792052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 msgr2=0x55e779202bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779202bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1839906860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 msgr2=0x55e7792052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7792052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e2a59c640 1 -- 172.21.15.113:0/1839906860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e778e733f0 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 0x55e779202670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c09c5a0 tx=0x7f5e1c09c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ebc2cf04d6f878 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e0eff5640 1 -- 172.21.15.113:0/1839906860 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0a5280 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e0eff5640 1 -- 172.21.15.113:0/1839906860 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f5e1c098c80 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e20005040 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e0eff5640 1 -- 172.21.15.113:0/1839906860 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e1c0035f0 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 10 client.17751 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e778f3d7d0 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e0eff5640 1 -- 172.21.15.113:0/1839906860 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5e20005040 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e0eff5640 1 -- 172.21.15.113:0/1839906860 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f5e1c098360 con 0x55e7791fce80 2024-04-22T07:24:33.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 10 client.17751 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 10 client.17751 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 20 client.17751 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779215b10 0x55e779217f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 4 client.17751 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.922+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e779100930 con 0x55e779215b10 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.923+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779215b10 0x55e779217f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.923+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779215b10 0x55e779217f20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abbe9659ac09ffba server_cookie=fc98d5f139ac418c in_seq=0 out_seq=0 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.923+0000 7f5e0eff5640 10 client.17751 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e0eff5640 1 -- 172.21.15.113:0/1839906860 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e779100930 con 0x55e779215b10 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e0eff5640 10 client.17751 handle_command_reply: tid=0 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 client.17751 shutdown 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 msgr2=0x55e779202670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 0x55e779202670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c09c5a0 tx=0x7f5e1c09c070 comp rx=0 tx=0).stop 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 shutdown_connections 2024-04-22T07:24:33.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779215b10 0x55e779217f20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779202bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e0eff5640 0 client.17751 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fce80 0x55e779202670 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 --2- 172.21.15.113:0/1839906860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e7792052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 >> 172.21.15.113:0/1839906860 conn(0x55e7791f6ba0 msgr2=0x55e7791f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 shutdown_connections 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 1 -- 172.21.15.113:0/1839906860 wait complete. 2024-04-22T07:24:33.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.924+0000 7f5e33db3740 20 client.17751 trim_cache size 0 max 16384 2024-04-22T07:24:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 Processor -- start 2024-04-22T07:24:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 -- start start 2024-04-22T07:24:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 0x55e7792117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fce80 0x55e77920e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779204da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1b08f0 con 0x55e7791fc4d0 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ae1c0 con 0x55e7791fce80 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e33db3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ac030 con 0x55e779205a00 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 0x55e7792117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 0x55e7792117d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34404/0 (socket says 172.21.15.113:34404) 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/7616215 learned_addr learned my addr 172.21.15.113:0/7616215 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fce80 0x55e77920e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/7616215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 msgr2=0x55e779204da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779204da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779204da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/7616215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fce80 msgr2=0x55e77920e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fce80 0x55e77920e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.937+0000 7f5e2ad9d640 1 -- 172.21.15.113:0/7616215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e779201620 con 0x55e7791fc4d0 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fce80 0x55e77920e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779204da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 0x55e7792117d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e2002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399d57ac2b595bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e10ff9640 1 -- 172.21.15.113:0/7616215 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20053070 con 0x55e7791fc4d0 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e10ff9640 1 -- 172.21.15.113:0/7616215 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5e2004edc0 con 0x55e7791fc4d0 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e10ff9640 1 -- 172.21.15.113:0/7616215 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e200052b0 con 0x55e7791fc4d0 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 -- 172.21.15.113:0/7616215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 msgr2=0x55e7792117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 0x55e7792117d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e2002e040 comp rx=0 tx=0).stop 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 -- 172.21.15.113:0/7616215 shutdown_connections 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e779205a00 0x55e779204da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fce80 0x55e77920e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 --2- 172.21.15.113:0/7616215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e7791fc4d0 0x55e7792117d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 -- 172.21.15.113:0/7616215 >> 172.21.15.113:0/7616215 conn(0x55e7791f6ba0 msgr2=0x55e7791fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 -- 172.21.15.113:0/7616215 shutdown_connections 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.938+0000 7f5e33db3740 1 -- 172.21.15.113:0/7616215 wait complete. 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 -- start start 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 Processor -- start 2024-04-22T07:24:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e779274360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e7792748a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779279380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ac680 con 0x55e779205a00 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1adaf0 con 0x55e7791fc4d0 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1acd50 con 0x55e7791fce80 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e33db3740 10 client.0 _ll_register_callbacks cb 0x7f5e1c09c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779279380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779279380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34416/0 (socket says 172.21.15.113:34416) 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 -- 172.21.15.113:0/3792697623 learned_addr learned my addr 172.21.15.113:0/3792697623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e7792748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e779274360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 -- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 msgr2=0x55e7792748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e7792748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 -- 172.21.15.113:0/3792697623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 msgr2=0x55e779274360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e779274360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 -- 172.21.15.113:0/3792697623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7791fcc20 con 0x55e779205a00 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2a59c640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e7792748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e779274360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e2b59e640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779279380 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5e24012680 tx=0x7f5e2400a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31466fc5498fda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.939+0000 7f5e0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e0eff5640 1 -- 172.21.15.113:0/3792697623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2402a630 con 0x55e779205a00 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e0eff5640 1 -- 172.21.15.113:0/3792697623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f5e24021430 con 0x55e779205a00 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e2004edc0 con 0x55e779205a00 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e0eff5640 1 -- 172.21.15.113:0/3792697623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e2400a8a0 con 0x55e779205a00 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 10 client.17757 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e779280c50 con 0x55e779205a00 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e0eff5640 1 -- 172.21.15.113:0/3792697623 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5e2004edc0 con 0x55e779205a00 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e0eff5640 1 -- 172.21.15.113:0/3792697623 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f5e240219f0 con 0x55e779205a00 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 10 client.17757 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 10 client.17757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 20 client.17757 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779281450 0x55e779283870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 4 client.17757 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e77916ec10 con 0x55e779281450 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.940+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779281450 0x55e779283870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e2ad9d640 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779281450 0x55e779283870 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75d712911ead5208 server_cookie=9fbd6a701a3ac56b in_seq=0 out_seq=0 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e0eff5640 10 client.17757 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e0eff5640 1 -- 172.21.15.113:0/3792697623 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e77916ec10 con 0x55e779281450 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e0eff5640 10 client.17757 handle_command_reply: tid=0 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 client.17757 shutdown 2024-04-22T07:24:33.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 msgr2=0x55e779279380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779279380 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5e24012680 tx=0x7f5e2400a4d0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 shutdown_connections 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x55e779281450 0x55e779283870 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e779205a00 0x55e779279380 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e0eff5640 0 client.17757 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e7791fce80 0x55e7792748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 --2- 172.21.15.113:0/3792697623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e7791fc4d0 0x55e779274360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 >> 172.21.15.113:0/3792697623 conn(0x55e7791f6ba0 msgr2=0x55e7791f5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 shutdown_connections 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 1 -- 172.21.15.113:0/3792697623 wait complete. 2024-04-22T07:24:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.941+0000 7f5e33db3740 20 client.17757 trim_cache size 0 max 16384 2024-04-22T07:24:33.950 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5e140a3d20 msgr2=0x7f5e140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5e140a3d20 0x7f5e140a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e24014be0 tx=0x7f5e2400adf0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 msgr2=0x7f5e2c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 0x7f5e2c199bb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c970 tx=0x7f5e1c00ce30 comp rx=0 tx=0).stop 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 shutdown_connections 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f5e140a3d20 0x7f5e140a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c19a140 0x7f5e2c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e2c106170 0x7f5e2c199bb0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 --2- 172.21.15.113:0/1547516114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e2c1010a0 0x7f5e2c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 >> 172.21.15.113:0/1547516114 conn(0x7f5e2c077eb0 msgr2=0x7f5e2c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 shutdown_connections 2024-04-22T07:24:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:33.942+0000 7f5e310e4640 1 -- 172.21.15.113:0/1547516114 wait complete. 2024-04-22T07:24:33.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 -- 172.21.15.113:0/3616179206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 msgr2=0x7fcf98101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 --2- 172.21.15.113:0/3616179206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf98101470 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009900 tx=0x7fcf8802bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 -- 172.21.15.113:0/3616179206 shutdown_connections 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 --2- 172.21.15.113:0/3616179206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106160 0x7fcf9810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 --2- 172.21.15.113:0/3616179206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98101a40 0x7fcf98105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 --2- 172.21.15.113:0/3616179206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf98101470 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 -- 172.21.15.113:0/3616179206 >> 172.21.15.113:0/3616179206 conn(0x7fcf98077f50 msgr2=0x7fcf98078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 -- 172.21.15.113:0/3616179206 shutdown_connections 2024-04-22T07:24:34.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 -- 172.21.15.113:0/3616179206 wait complete. 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 Processor -- start 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.055+0000 7fcf9de74640 1 -- start start 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf9819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106160 0x7fcf9819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98198630 0x7fcf98198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981111e0 con 0x7fcf98101090 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111060 con 0x7fcf98106160 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111360 con 0x7fcf98198630 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf9819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf9819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34448/0 (socket says 172.21.15.113:34448) 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 -- 172.21.15.113:0/4241117598 learned_addr learned my addr 172.21.15.113:0/4241117598 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf97fff640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98198630 0x7fcf98198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106160 0x7fcf9819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 -- 172.21.15.113:0/4241117598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98198630 msgr2=0x7fcf98198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98198630 0x7fcf98198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 -- 172.21.15.113:0/4241117598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106160 msgr2=0x7fcf9819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106160 0x7fcf9819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 -- 172.21.15.113:0/4241117598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf98199230 con 0x7fcf98101090 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf97fff640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98198630 0x7fcf98198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf977fe640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf9819e570 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005940 tx=0x7fcf8802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf957fa640 1 -- 172.21.15.113:0/4241117598 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88041070 con 0x7fcf98101090 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf957fa640 1 -- 172.21.15.113:0/4241117598 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcf880052f0 con 0x7fcf98101090 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981994c0 con 0x7fcf98101090 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf957fa640 1 -- 172.21.15.113:0/4241117598 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88003d60 con 0x7fcf98101090 2024-04-22T07:24:34.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.056+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a5660 con 0x7fcf98101090 2024-04-22T07:24:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.058+0000 7fcf957fa640 1 -- 172.21.15.113:0/4241117598 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fcf88044070 con 0x7fcf98101090 2024-04-22T07:24:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 Processor -- start 2024-04-22T07:24:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 -- start start 2024-04-22T07:24:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf957fa640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fcf840a3d20 0x7fcf840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d80430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d82e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639f1cdc1f0 con 0x5639f1d80a00 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5639f1c794b0 con 0x5639f1d774d0 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639f1cd81b0 con 0x5639f1d77e80 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fcf840a3d20 0x7fcf840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d82e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf957fa640 1 -- 172.21.15.113:0/4241117598 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fcf880f2830 con 0x7fcf98101090 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d82e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34450/0 (socket says 172.21.15.113:34450) 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 -- 172.21.15.113:0/1334380823 learned_addr learned my addr 172.21.15.113:0/1334380823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d80430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 -- 172.21.15.113:0/1334380823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 msgr2=0x5639f1d80430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d80430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 -- 172.21.15.113:0/1334380823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 msgr2=0x5639f1d778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d778b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.059+0000 7fcf97fff640 1 -- 172.21.15.113:0/1334380823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f1c6b1b0 con 0x5639f1d80a00 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcf97fff640 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d82e50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcf90012db0 tx=0x7fcf9000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9fee04b10905d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcf97fff640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fcf840a3d20 0x7fcf840a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000fc90 tx=0x7fcf9001d000 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcf8dffb640 1 -- 172.21.15.113:0/1334380823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90022070 con 0x5639f1d80a00 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcf8dffb640 1 -- 172.21.15.113:0/1334380823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcf90009d20 con 0x5639f1d80a00 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcf8dffb640 1 -- 172.21.15.113:0/1334380823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000de10 con 0x5639f1d80a00 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/1334380823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 msgr2=0x5639f1d82e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.060+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d82e50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcf90012db0 tx=0x7fcf9000f790 comp rx=0 tx=0).stop 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/1334380823 shutdown_connections 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d82e50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d80430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/1334380823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/1334380823 >> 172.21.15.113:0/1334380823 conn(0x5639f1d71ba0 msgr2=0x5639f1d71f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/1334380823 shutdown_connections 2024-04-22T07:24:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/1334380823 wait complete. 2024-04-22T07:24:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- start start 2024-04-22T07:24:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 Processor -- start 2024-04-22T07:24:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 0x5639f1d80420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d77e80 0x5639f1d7d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d80a00 0x5639f1d7def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639f1d83910 con 0x5639f1d774d0 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5639f1c95500 con 0x5639f1d77e80 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639f1d83a90 con 0x5639f1d80a00 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcfa0a6d740 10 client.0 _ll_register_callbacks cb 0x5639f1c63b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 0x5639f1d80420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcf977fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 0x5639f1d80420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34462/0 (socket says 172.21.15.113:34462) 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.061+0000 7fcf977fe640 1 -- 172.21.15.113:0/2720315113 learned_addr learned my addr 172.21.15.113:0/2720315113 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf977fe640 1 -- 172.21.15.113:0/2720315113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d80a00 msgr2=0x5639f1d7def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf97fff640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d80a00 0x5639f1d7def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d77e80 0x5639f1d7d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf977fe640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d80a00 0x5639f1d7def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf977fe640 1 -- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d77e80 msgr2=0x5639f1d7d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf977fe640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d77e80 0x5639f1d7d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf977fe640 1 -- 172.21.15.113:0/2720315113 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f19ee3f0 con 0x5639f1d774d0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf97fff640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d80a00 0x5639f1d7def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d77e80 0x5639f1d7d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf977fe640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 0x5639f1d80420 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcf880eff50 tx=0x7fcf880be510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387b88548a796329 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf76ffd640 1 -- 172.21.15.113:0/2720315113 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8804b070 con 0x5639f1d774d0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf76ffd640 1 -- 172.21.15.113:0/2720315113 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fcf880054a0 con 0x5639f1d774d0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90009d20 con 0x5639f1d774d0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf76ffd640 1 -- 172.21.15.113:0/2720315113 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcf880d7ec0 con 0x5639f1d774d0 2024-04-22T07:24:34.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcfa0a6d740 10 client.17769 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5639f1ab87d0 con 0x5639f1d774d0 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf76ffd640 1 -- 172.21.15.113:0/2720315113 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcf90009d20 con 0x5639f1d774d0 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.062+0000 7fcf76ffd640 1 -- 172.21.15.113:0/2720315113 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fcf880d7040 con 0x5639f1d774d0 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcfa0a6d740 10 client.17769 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcfa0a6d740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcfa0a6d740 20 client.17769 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1d90c50 0x5639f1d930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcfa0a6d740 4 client.17769 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f1c7b930 con 0x5639f1d90c50 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1d90c50 0x5639f1d930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcf76ffd640 10 client.17769 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.063+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1d90c50 0x5639f1d930e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cdf662b49dd4159 server_cookie=d12af4c771b03f36 in_seq=0 out_seq=0 2024-04-22T07:24:34.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcf76ffd640 1 -- 172.21.15.113:0/2720315113 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639f1c7b930 con 0x5639f1d90c50 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcf76ffd640 10 client.17769 handle_command_reply: tid=0 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 client.17769 shutdown 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 msgr2=0x5639f1d80420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 0x5639f1d80420 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcf880eff50 tx=0x7fcf880be510 comp rx=0 tx=0).stop 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 shutdown_connections 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1d90c50 0x5639f1d930e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d80a00 0x5639f1d7def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d77e80 0x5639f1d7d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcf76ffd640 0 client.17769 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2720315113 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d774d0 0x5639f1d80420 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 >> 172.21.15.113:0/2720315113 conn(0x5639f1d71ba0 msgr2=0x5639f1d738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 shutdown_connections 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2720315113 wait complete. 2024-04-22T07:24:34.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.064+0000 7fcfa0a6d740 20 client.17769 trim_cache size 0 max 16384 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 Processor -- start 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 -- start start 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d774d0 0x5639f1d8c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 0x5639f1d7fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d80a00 0x5639f1d80420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b0740 con 0x5639f1d77e80 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ae010 con 0x5639f1d80a00 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.077+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981abe80 con 0x5639f1d774d0 2024-04-22T07:24:34.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 0x5639f1d7fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 0x5639f1d7fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34464/0 (socket says 172.21.15.113:34464) 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 -- 172.21.15.113:0/2354293606 learned_addr learned my addr 172.21.15.113:0/2354293606 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf977fe640 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d774d0 0x5639f1d8c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 -- 172.21.15.113:0/2354293606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d774d0 msgr2=0x5639f1d8c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d774d0 0x5639f1d8c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 -- 172.21.15.113:0/2354293606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d80a00 msgr2=0x5639f1d80420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d80a00 0x5639f1d80420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 -- 172.21.15.113:0/2354293606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f1d770a0 con 0x5639f1d77e80 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 0x5639f1d7fee0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000dc10 tx=0x7fcf8000d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827245712e933703 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8d7fa640 1 -- 172.21.15.113:0/2354293606 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8001e020 con 0x5639f1d77e80 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8d7fa640 1 -- 172.21.15.113:0/2354293606 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcf8001adc0 con 0x5639f1d77e80 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.078+0000 7fcf8d7fa640 1 -- 172.21.15.113:0/2354293606 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80003850 con 0x5639f1d77e80 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2354293606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 msgr2=0x5639f1d7fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 0x5639f1d7fee0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000dc10 tx=0x7fcf8000d670 comp rx=0 tx=0).stop 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2354293606 shutdown_connections 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d80a00 0x5639f1d80420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d77e80 0x5639f1d7fee0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/2354293606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d774d0 0x5639f1d8c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2354293606 >> 172.21.15.113:0/2354293606 conn(0x5639f1d71ba0 msgr2=0x5639f1d7d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2354293606 shutdown_connections 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/2354293606 wait complete. 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- start start 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 Processor -- start 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d88f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d7a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d7a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.079+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ac4d0 con 0x5639f1d80a00 2024-04-22T07:24:34.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ad940 con 0x5639f1d774d0 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcfa0a6d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981acba0 con 0x5639f1d77e80 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcfa0a6d740 10 client.0 _ll_register_callbacks cb 0x7fcf880ba9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d7a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d7a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34474/0 (socket says 172.21.15.113:34474) 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 -- 172.21.15.113:0/889271606 learned_addr learned my addr 172.21.15.113:0/889271606 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d7a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 -- 172.21.15.113:0/889271606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 msgr2=0x5639f1d7a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf977fe640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d88f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d7a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 -- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 msgr2=0x5639f1d88f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d88f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 -- 172.21.15.113:0/889271606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f1d7ac10 con 0x5639f1d80a00 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf8ffff640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d7a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:34.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf97fff640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d7a5a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005cc0 tx=0x7fcf90012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c988965c266fe9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf76ffd640 1 -- 172.21.15.113:0/889271606 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9002a030 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf8001adc0 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf76ffd640 1 -- 172.21.15.113:0/889271606 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fcf9001b070 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcf76ffd640 1 -- 172.21.15.113:0/889271606 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcf90022070 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcfa0a6d740 10 client.17775 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.080+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5639f1dfb4b0 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcf76ffd640 1 -- 172.21.15.113:0/889271606 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcf8001adc0 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcf76ffd640 1 -- 172.21.15.113:0/889271606 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fcf90005040 con 0x5639f1d80a00 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcfa0a6d740 10 client.17775 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcfa0a6d740 10 client.17775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcfa0a6d740 20 client.17775 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1dfb940 0x5639f1dfdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:34.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcfa0a6d740 4 client.17775 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639f1ce9c10 con 0x5639f1dfb940 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcf977fe640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1dfb940 0x5639f1dfdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcf76ffd640 10 client.17775 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.081+0000 7fcf977fe640 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1dfb940 0x5639f1dfdd60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=101c359c1796a545 server_cookie=f61ec4c9e39c0efc in_seq=0 out_seq=0 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf76ffd640 1 -- 172.21.15.113:0/889271606 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639f1ce9c10 con 0x5639f1dfb940 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf76ffd640 10 client.17775 handle_command_reply: tid=0 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 client.17775 shutdown 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 msgr2=0x5639f1d7a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d7a5a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005cc0 tx=0x7fcf90012c30 comp rx=0 tx=0).stop 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 shutdown_connections 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5639f1dfb940 0x5639f1dfdd60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639f1d80a00 0x5639f1d7a5a0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf76ffd640 0 client.17775 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:24:34.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639f1d77e80 0x5639f1d7a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 --2- 172.21.15.113:0/889271606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639f1d774d0 0x5639f1d88f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 >> 172.21.15.113:0/889271606 conn(0x5639f1d71ba0 msgr2=0x5639f1d72fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 shutdown_connections 2024-04-22T07:24:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 1 -- 172.21.15.113:0/889271606 wait complete. 2024-04-22T07:24:34.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcfa0a6d740 20 client.17775 trim_cache size 0 max 16384 2024-04-22T07:24:34.086 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:24:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fcf840a3d20 msgr2=0x7fcf840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fcf840a3d20 0x7fcf840a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000fc90 tx=0x7fcf9001d000 comp rx=0 tx=0).stop 2024-04-22T07:24:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 msgr2=0x7fcf9819e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf9819e570 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005940 tx=0x7fcf8802c6c0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 shutdown_connections 2024-04-22T07:24:34.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fcf840a3d20 0x7fcf840a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf98198630 0x7fcf98198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf98106160 0x7fcf9819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 --2- 172.21.15.113:0/4241117598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf98101090 0x7fcf9819e570 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.082+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 >> 172.21.15.113:0/4241117598 conn(0x7fcf98077f50 msgr2=0x7fcf98069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.083+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 shutdown_connections 2024-04-22T07:24:34.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:34.083+0000 7fcf9de74640 1 -- 172.21.15.113:0/4241117598 wait complete. 2024-04-22T07:24:35.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:34 smithi133 ceph-mon[31198]: pgmap v307: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:35.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:34 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3544217365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:24:35.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:34 smithi154 ceph-mon[31862]: pgmap v307: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:35.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:34 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3544217365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:24:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:34 smithi113 ceph-mon[27933]: pgmap v307: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:34 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3544217365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:24:35.091 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T07:24:35.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:24:35.092 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T07:24:35.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.202+0000 7f921a39f640 1 -- 172.21.15.113:0/1970469210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214101a40 msgr2=0x7f9214105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.202+0000 7f921a39f640 1 --2- 172.21.15.113:0/1970469210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214101a40 0x7f9214105a80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f920802bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 -- 172.21.15.113:0/1970469210 shutdown_connections 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 --2- 172.21.15.113:0/1970469210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214106160 0x7f921410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 --2- 172.21.15.113:0/1970469210 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214101a40 0x7f9214105a80 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 --2- 172.21.15.113:0/1970469210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214101090 0x7f9214101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 -- 172.21.15.113:0/1970469210 >> 172.21.15.113:0/1970469210 conn(0x7f9214077f50 msgr2=0x7f9214078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 -- 172.21.15.113:0/1970469210 shutdown_connections 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 -- 172.21.15.113:0/1970469210 wait complete. 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 Processor -- start 2024-04-22T07:24:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 -- start start 2024-04-22T07:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214101090 0x7f921419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.203+0000 7f921a39f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214106160 0x7f921419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921a39f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 0x7f9214198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921a39f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9214111240 con 0x7f9214198770 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921a39f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92141110c0 con 0x7f9214106160 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921a39f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92141113c0 con 0x7f9214101090 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 0x7f9214198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 0x7f9214198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34494/0 (socket says 172.21.15.113:34494) 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 -- 172.21.15.113:0/3311070328 learned_addr learned my addr 172.21.15.113:0/3311070328 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f9213fff640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214101090 0x7f921419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 -- 172.21.15.113:0/3311070328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214101090 msgr2=0x7f921419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214101090 0x7f921419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f92137fe640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214106160 0x7f921419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 -- 172.21.15.113:0/3311070328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214106160 msgr2=0x7f921419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214106160 0x7f921419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 -- 172.21.15.113:0/3311070328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9214199350 con 0x7f9214198770 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f9213fff640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214101090 0x7f921419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f92137fe640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214106160 0x7f921419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921893c640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 0x7f9214198c20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f920c00cad0 tx=0x7f920c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f92117fa640 1 -- 172.21.15.113:0/3311070328 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c00ae20 con 0x7f9214198770 2024-04-22T07:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92141a5430 con 0x7f9214198770 2024-04-22T07:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f92117fa640 1 -- 172.21.15.113:0/3311070328 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f920c005020 con 0x7f9214198770 2024-04-22T07:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f92117fa640 1 -- 172.21.15.113:0/3311070328 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c00f6b0 con 0x7f9214198770 2024-04-22T07:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.204+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92141a57e0 con 0x7f9214198770 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f92117fa640 1 -- 172.21.15.113:0/3311070328 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f920c0051d0 con 0x7f9214198770 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 Processor -- start 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 -- start start 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08588cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08589290 0x563e08591840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 0x563e08594260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563e084ecf00 con 0x563e08591e10 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563e0848a590 con 0x563e085888e0 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.207+0000 7f921cf8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e084e8ec0 con 0x563e08589290 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 0x563e08594260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92117fa640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91fc0a3d20 0x7f91fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 0x563e08594260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34510/0 (socket says 172.21.15.113:34510) 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 -- 172.21.15.113:0/2055919254 learned_addr learned my addr 172.21.15.113:0/2055919254 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f9213fff640 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08589290 0x563e08591840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f921893c640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91fc0a3d20 0x7f91fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08588cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 -- 172.21.15.113:0/2055919254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08589290 msgr2=0x563e08591840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92117fa640 1 -- 172.21.15.113:0/3311070328 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f920c0d0aa0 con 0x7f9214198770 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08589290 0x563e08591840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 -- 172.21.15.113:0/2055919254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 msgr2=0x563e08588cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08588cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 -- 172.21.15.113:0/2055919254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e082c77e0 con 0x563e08591e10 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f92137fe640 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 0x563e08594260 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f9208009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1394cf684f26c372 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f91fa7fc640 1 -- 172.21.15.113:0/2055919254 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208047070 con 0x563e08591e10 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f921893c640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91fc0a3d20 0x7f91fc0a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ce100 tx=0x7f920c01a040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f91fa7fc640 1 -- 172.21.15.113:0/2055919254 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9208005040 con 0x563e08591e10 2024-04-22T07:24:35.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.208+0000 7f91fa7fc640 1 -- 172.21.15.113:0/2055919254 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208004a40 con 0x563e08591e10 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 -- 172.21.15.113:0/2055919254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 msgr2=0x563e08594260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 0x563e08594260 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f9208009560 comp rx=0 tx=0).stop 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 -- 172.21.15.113:0/2055919254 shutdown_connections 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08591e10 0x563e08594260 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08589290 0x563e08591840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 --2- 172.21.15.113:0/2055919254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08588cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 -- 172.21.15.113:0/2055919254 >> 172.21.15.113:0/2055919254 conn(0x563e08582fb0 msgr2=0x563e08583390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 -- 172.21.15.113:0/2055919254 shutdown_connections 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 -- 172.21.15.113:0/2055919254 wait complete. 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.209+0000 7f921cf8a740 1 -- start start 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 Processor -- start 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e085888e0 0x563e0858ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0858f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08591e10 0x563e0858f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563e08594d20 con 0x563e08589290 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563e084a6580 con 0x563e08591e10 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563e08594ea0 con 0x563e085888e0 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f921cf8a740 10 client.0 _ll_register_callbacks cb 0x563e084ac650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0858f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0858f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34526/0 (socket says 172.21.15.113:34526) 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 -- 172.21.15.113:0/3752014390 learned_addr learned my addr 172.21.15.113:0/3752014390 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f92137fe640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08591e10 0x563e0858f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f92137fe640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e085888e0 0x563e0858ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 -- 172.21.15.113:0/3752014390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e085888e0 msgr2=0x563e0858ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e085888e0 0x563e0858ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 -- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08591e10 msgr2=0x563e0858f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08591e10 0x563e0858f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 -- 172.21.15.113:0/3752014390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e08327f70 con 0x563e08589290 2024-04-22T07:24:35.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f92137fe640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08591e10 0x563e0858f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f9213fff640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0858f370 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f920400f470 tx=0x7f920400f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e88593d9915ed7d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f91f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.210+0000 7f91f37fe640 1 -- 172.21.15.113:0/3752014390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204013850 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f91f37fe640 1 -- 172.21.15.113:0/3752014390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f92040139f0 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9208005040 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f91f37fe640 1 -- 172.21.15.113:0/3752014390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9204013bf0 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 10 client.17787 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563e081bf170 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f91f37fe640 1 -- 172.21.15.113:0/3752014390 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9208005040 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f91f37fe640 1 -- 172.21.15.113:0/3752014390 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f920401b400 con 0x563e08589290 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 10 client.17787 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 20 client.17787 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:35.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e085a2420 0x563e085a4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 4 client.17787 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e084fa920 con 0x563e085a2420 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.211+0000 7f92137fe640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e085a2420 0x563e085a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.212+0000 7f92137fe640 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e085a2420 0x563e085a4880 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875910bd3351e402 server_cookie=42d116dfcf26463d in_seq=0 out_seq=0 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.212+0000 7f91f37fe640 10 client.17787 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.212+0000 7f91f37fe640 1 -- 172.21.15.113:0/3752014390 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e084fa920 con 0x563e085a2420 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.212+0000 7f91f37fe640 10 client.17787 handle_command_reply: tid=0 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.212+0000 7f921cf8a740 1 client.17787 shutdown 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 msgr2=0x563e0858f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0858f370 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f920400f470 tx=0x7f920400f930 comp rx=0 tx=0).stop 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 shutdown_connections 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e085a2420 0x563e085a4880 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08591e10 0x563e0858f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f91f37fe640 0 client.17787 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0858f370 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 --2- 172.21.15.113:0/3752014390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e085888e0 0x563e0858ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 >> 172.21.15.113:0/3752014390 conn(0x563e08582fb0 msgr2=0x563e08584d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 shutdown_connections 2024-04-22T07:24:35.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 1 -- 172.21.15.113:0/3752014390 wait complete. 2024-04-22T07:24:35.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.213+0000 7f921cf8a740 20 client.17787 trim_cache size 0 max 16384 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 Processor -- start 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 -- start start 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08591560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0859dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 0x563e0859a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b0880 con 0x563e08589290 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ae150 con 0x563e085888e0 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f921cf8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92141abfc0 con 0x563e08591e10 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0859dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0859dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34532/0 (socket says 172.21.15.113:34532) 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 -- 172.21.15.113:0/4180433198 learned_addr learned my addr 172.21.15.113:0/4180433198 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f92137fe640 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08591560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 -- 172.21.15.113:0/4180433198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 msgr2=0x563e0859a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 0x563e0859a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 -- 172.21.15.113:0/4180433198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 msgr2=0x563e08591560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08591560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 -- 172.21.15.113:0/4180433198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e0858dde0 con 0x563e08589290 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f92137fe640 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08591560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.230+0000 7f9213fff640 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0859dc10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9204008010 tx=0x7f920400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b91d7d439a240c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f91f9ffb640 1 -- 172.21.15.113:0/4180433198 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920401ba60 con 0x563e08589290 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f91f9ffb640 1 -- 172.21.15.113:0/4180433198 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f920401bc00 con 0x563e08589290 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f91f9ffb640 1 -- 172.21.15.113:0/4180433198 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92040062c0 con 0x563e08589290 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 -- 172.21.15.113:0/4180433198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 msgr2=0x563e0859dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0859dc10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9204008010 tx=0x7f920400a620 comp rx=0 tx=0).stop 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 -- 172.21.15.113:0/4180433198 shutdown_connections 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 0x563e0859a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e08589290 0x563e0859dc10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 --2- 172.21.15.113:0/4180433198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e085888e0 0x563e08591560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 -- 172.21.15.113:0/4180433198 >> 172.21.15.113:0/4180433198 conn(0x563e08582fb0 msgr2=0x563e085843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 -- 172.21.15.113:0/4180433198 shutdown_connections 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 -- 172.21.15.113:0/4180433198 wait complete. 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.231+0000 7f921cf8a740 1 -- start start 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 Processor -- start 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 0x563e0858c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08589290 0x563e0858c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 0x563e0858b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ac610 con 0x563e085888e0 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ada80 con 0x563e08589290 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f92141acce0 con 0x563e08591e10 2024-04-22T07:24:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f921cf8a740 10 client.0 _ll_register_callbacks cb 0x7f920400a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 0x563e0858c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 0x563e0858c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34548/0 (socket says 172.21.15.113:34548) 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 -- 172.21.15.113:0/906966685 learned_addr learned my addr 172.21.15.113:0/906966685 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f9213fff640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08589290 0x563e0858c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 -- 172.21.15.113:0/906966685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 msgr2=0x563e0858b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 0x563e0858b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 -- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08589290 msgr2=0x563e0858c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08589290 0x563e0858c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 -- 172.21.15.113:0/906966685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0858b740 con 0x563e085888e0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f9213fff640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08589290 0x563e0858c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f92137fe640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 0x563e0858c310 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f9208001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d95b9deab69d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.232+0000 7f91f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f91f37fe640 1 -- 172.21.15.113:0/906966685 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208047040 con 0x563e085888e0 2024-04-22T07:24:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920401bc00 con 0x563e085888e0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f91f37fe640 1 -- 172.21.15.113:0/906966685 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f92080054f0 con 0x563e085888e0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f91f37fe640 1 -- 172.21.15.113:0/906966685 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9208004040 con 0x563e085888e0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 10 client.17793 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563e0860d3c0 con 0x563e085888e0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f91f37fe640 1 -- 172.21.15.113:0/906966685 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f920401bc00 con 0x563e085888e0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f91f37fe640 1 -- 172.21.15.113:0/906966685 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f92080506d0 con 0x563e085888e0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 10 client.17793 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 20 client.17793 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e0860d6f0 0x563e0860fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 4 client.17793 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563e084f4790 con 0x563e0860d6f0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.233+0000 7f9213fff640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e0860d6f0 0x563e0860fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.234+0000 7f91f37fe640 10 client.17793 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.234+0000 7f9213fff640 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e0860d6f0 0x563e0860fae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe095170b3623461 server_cookie=3c4b19765bbdc0e6 in_seq=0 out_seq=0 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f91f37fe640 1 -- 172.21.15.113:0/906966685 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563e084f4790 con 0x563e0860d6f0 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f91f37fe640 10 client.17793 handle_command_reply: tid=0 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 client.17793 shutdown 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 msgr2=0x563e0858c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 0x563e0858c310 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f9208001e50 comp rx=0 tx=0).stop 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 shutdown_connections 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x563e0860d6f0 0x563e0860fae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563e08591e10 0x563e0858b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e08589290 0x563e0858c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f91f37fe640 0 client.17793 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 --2- 172.21.15.113:0/906966685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563e085888e0 0x563e0858c310 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 >> 172.21.15.113:0/906966685 conn(0x563e08582fb0 msgr2=0x563e0858f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 shutdown_connections 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 1 -- 172.21.15.113:0/906966685 wait complete. 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.235+0000 7f921cf8a740 20 client.17793 trim_cache size 0 max 16384 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stdout: "return_code": 0, 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stdout: "scrub_tag": "16d404af-17ef-4550-8e73-105b18db6e55", 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": "asynchronous" 2024-04-22T07:24:35.239 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:24:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91fc0a3d20 msgr2=0x7f91fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91fc0a3d20 0x7f91fc0a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ce100 tx=0x7f920c01a040 comp rx=0 tx=0).stop 2024-04-22T07:24:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 msgr2=0x7f9214198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 0x7f9214198c20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f920c00cad0 tx=0x7f920c00cf90 comp rx=0 tx=0).stop 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 shutdown_connections 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f91fc0a3d20 0x7f91fc0a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9214198770 0x7f9214198c20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9214106160 0x7f921419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 --2- 172.21.15.113:0/3311070328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9214101090 0x7f921419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 >> 172.21.15.113:0/3311070328 conn(0x7f9214077f50 msgr2=0x7f92140694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 shutdown_connections 2024-04-22T07:24:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.236+0000 7f921a39f640 1 -- 172.21.15.113:0/3311070328 wait complete. 2024-04-22T07:24:35.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:24:35.244 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:24:35.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 -- 172.21.15.113:0/1405035214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664102b50 msgr2=0x7fd664106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/1405035214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664102b50 0x7fd664106b90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd658009900 tx=0x7fd65802bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:35.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 -- 172.21.15.113:0/1405035214 shutdown_connections 2024-04-22T07:24:35.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/1405035214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 0x7fd66410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/1405035214 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664102b50 0x7fd664106b90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/1405035214 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6641021a0 0x7fd664102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 -- 172.21.15.113:0/1405035214 >> 172.21.15.113:0/1405035214 conn(0x7fd6640793e0 msgr2=0x7fd6640797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 -- 172.21.15.113:0/1405035214 shutdown_connections 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.346+0000 7fd66b0e0640 1 -- 172.21.15.113:0/1405035214 wait complete. 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 Processor -- start 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 -- start start 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6641021a0 0x7fd66419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 0x7fd66419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 0x7fd664198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641122d0 con 0x7fd664198650 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd664112150 con 0x7fd664107270 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66b0e0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd664112450 con 0x7fd6641021a0 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66967d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 0x7fd664198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66967d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 0x7fd664198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34562/0 (socket says 172.21.15.113:34562) 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66967d640 1 -- 172.21.15.113:0/2653769306 learned_addr learned my addr 172.21.15.113:0/2653769306 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd663fff640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 0x7fd66419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd668e7c640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6641021a0 0x7fd66419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.347+0000 7fd66967d640 1 -- 172.21.15.113:0/2653769306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6641021a0 msgr2=0x7fd66419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66967d640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6641021a0 0x7fd66419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66967d640 1 -- 172.21.15.113:0/2653769306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 msgr2=0x7fd66419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66967d640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 0x7fd66419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66967d640 1 -- 172.21.15.113:0/2653769306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd664199250 con 0x7fd664198650 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd663fff640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 0x7fd66419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66967d640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 0x7fd664198ad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00eaa0 tx=0x7fd65c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd661ffb640 1 -- 172.21.15.113:0/2653769306 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00cdb0 con 0x7fd664198650 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd661ffb640 1 -- 172.21.15.113:0/2653769306 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd65c005020 con 0x7fd664198650 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6641a5330 con 0x7fd664198650 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd661ffb640 1 -- 172.21.15.113:0/2653769306 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0116d0 con 0x7fd664198650 2024-04-22T07:24:35.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.348+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6641a56e0 con 0x7fd664198650 2024-04-22T07:24:35.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.350+0000 7fd661ffb640 1 -- 172.21.15.113:0/2653769306 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd65c0051d0 con 0x7fd664198650 2024-04-22T07:24:35.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.350+0000 7fd66ddac740 1 Processor -- start 2024-04-22T07:24:35.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd661ffb640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd64c0a3d20 0x7fd64c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 -- start start 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd661ffb640 1 -- 172.21.15.113:0/2653769306 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fd65c0d0c80 con 0x7fd664198650 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd663fff640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd64c0a3d20 0x7fd64c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374580 0x55b4ea374960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea37d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 0x55b4ea37fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea2d91e0 con 0x55b4ea37da80 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea276390 con 0x55b4ea374580 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66ddac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea2d51a0 con 0x55b4ea374ea0 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 0x55b4ea37fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 0x55b4ea37fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34566/0 (socket says 172.21.15.113:34566) 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 -- 172.21.15.113:0/1648996338 learned_addr learned my addr 172.21.15.113:0/1648996338 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd668e7c640 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374580 0x55b4ea374960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd663fff640 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea37d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 -- 172.21.15.113:0/1648996338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 msgr2=0x55b4ea37d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea37d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 -- 172.21.15.113:0/1648996338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374580 msgr2=0x55b4ea374960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374580 0x55b4ea374960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 -- 172.21.15.113:0/1648996338 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ea0b57d0 con 0x55b4ea37da80 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd663fff640 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea37d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd663fff640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd64c0a3d20 0x7fd64c0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd658036040 tx=0x7fd65802c690 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd66967d640 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 0x55b4ea37fe70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0ce830 tx=0x7fd65c09cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5867fffac647bed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd64affd640 1 -- 172.21.15.113:0/1648996338 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c01f040 con 0x55b4ea37da80 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd64affd640 1 -- 172.21.15.113:0/1648996338 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd65c005580 con 0x55b4ea37da80 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.351+0000 7fd64affd640 1 -- 172.21.15.113:0/1648996338 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c098a30 con 0x55b4ea37da80 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 -- 172.21.15.113:0/1648996338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 msgr2=0x55b4ea37fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 0x55b4ea37fe70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0ce830 tx=0x7fd65c09cbc0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 -- 172.21.15.113:0/1648996338 shutdown_connections 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea37da80 0x55b4ea37fe70 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea37d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 --2- 172.21.15.113:0/1648996338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374580 0x55b4ea374960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 -- 172.21.15.113:0/1648996338 >> 172.21.15.113:0/1648996338 conn(0x55b4ea36ecb0 msgr2=0x55b4ea36f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 -- 172.21.15.113:0/1648996338 shutdown_connections 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 -- 172.21.15.113:0/1648996338 wait complete. 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.352+0000 7fd66ddac740 1 -- start start 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 Processor -- start 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374580 0x55b4ea37ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 0x55b4ea37b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea37b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea380870 con 0x55b4ea374ea0 2024-04-22T07:24:35.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea2922f0 con 0x55b4ea37da80 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea3809f0 con 0x55b4ea374580 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66ddac740 10 client.0 _ll_register_callbacks cb 0x55b4ea29f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 0x55b4ea37b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 0x55b4ea37b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34570/0 (socket says 172.21.15.113:34570) 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 -- 172.21.15.113:0/649485430 learned_addr learned my addr 172.21.15.113:0/649485430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd66967d640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea37b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd668e7c640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374580 0x55b4ea37ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 -- 172.21.15.113:0/649485430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374580 msgr2=0x55b4ea37ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374580 0x55b4ea37ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 -- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 msgr2=0x55b4ea37b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea37b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 -- 172.21.15.113:0/649485430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e9feb3f0 con 0x55b4ea374ea0 2024-04-22T07:24:35.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd663fff640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 0x55b4ea37b050 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd658035000 tx=0x7fd65802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0da22ea9eeec978 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.353+0000 7fd643fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd643fff640 1 -- 172.21.15.113:0/649485430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6580046c0 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd643fff640 1 -- 172.21.15.113:0/649485430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fd658004860 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd65c005580 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd643fff640 1 -- 172.21.15.113:0/649485430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd658058050 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 10 client.17805 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b4e9fab0f0 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd643fff640 1 -- 172.21.15.113:0/649485430 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd65c005580 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd643fff640 1 -- 172.21.15.113:0/649485430 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fd658003cc0 con 0x55b4ea374ea0 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 10 client.17805 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 20 client.17805 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:35.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea38db30 0x55b4ea38ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 4 client.17805 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.354+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4ea2e6c00 con 0x55b4ea38db30 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.355+0000 7fd668e7c640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea38db30 0x55b4ea38ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.355+0000 7fd668e7c640 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea38db30 0x55b4ea38ff90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97466ecfc512ded7 server_cookie=42c0382ca8504d6e in_seq=0 out_seq=0 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.355+0000 7fd643fff640 10 client.17805 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd643fff640 1 -- 172.21.15.113:0/649485430 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4ea2e6c00 con 0x55b4ea38db30 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd643fff640 10 client.17805 handle_command_reply: tid=0 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 client.17805 shutdown 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 msgr2=0x55b4ea37b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 0x55b4ea37b050 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd658035000 tx=0x7fd65802bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 shutdown_connections 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea38db30 0x55b4ea38ff90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea37b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374ea0 0x55b4ea37b050 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 --2- 172.21.15.113:0/649485430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374580 0x55b4ea37ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd643fff640 0 client.17805 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 >> 172.21.15.113:0/649485430 conn(0x55b4ea36ecb0 msgr2=0x55b4ea370a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 shutdown_connections 2024-04-22T07:24:35.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 1 -- 172.21.15.113:0/649485430 wait complete. 2024-04-22T07:24:35.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.356+0000 7fd66ddac740 20 client.17805 trim_cache size 0 max 16384 2024-04-22T07:24:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 Processor -- start 2024-04-22T07:24:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 -- start start 2024-04-22T07:24:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea37d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374ea0 0x55b4ea389730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea37da80 0x55b4ea386020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641b0860 con 0x55b4ea374580 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641ae130 con 0x55b4ea374ea0 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd66ddac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641abfa0 con 0x55b4ea37da80 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea37d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea37d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34572/0 (socket says 172.21.15.113:34572) 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 -- 172.21.15.113:0/86063587 learned_addr learned my addr 172.21.15.113:0/86063587 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 -- 172.21.15.113:0/86063587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea37da80 msgr2=0x55b4ea386020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea37da80 0x55b4ea386020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 -- 172.21.15.113:0/86063587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374ea0 msgr2=0x55b4ea389730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374ea0 0x55b4ea389730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.373+0000 7fd668e7c640 1 -- 172.21.15.113:0/86063587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ea374cd0 con 0x55b4ea374580 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd668e7c640 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea37d240 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x55b4ea3907d0 tx=0x7fd654005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857b3253a05f9b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd64a7fc640 1 -- 172.21.15.113:0/86063587 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65401e070 con 0x55b4ea374580 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd64a7fc640 1 -- 172.21.15.113:0/86063587 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd65401adc0 con 0x55b4ea374580 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd64a7fc640 1 -- 172.21.15.113:0/86063587 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654011a50 con 0x55b4ea374580 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 -- 172.21.15.113:0/86063587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 msgr2=0x55b4ea37d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea37d240 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x55b4ea3907d0 tx=0x7fd654005c70 comp rx=0 tx=0).stop 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 -- 172.21.15.113:0/86063587 shutdown_connections 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea37da80 0x55b4ea386020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea374ea0 0x55b4ea389730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 --2- 172.21.15.113:0/86063587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea37d240 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 -- 172.21.15.113:0/86063587 >> 172.21.15.113:0/86063587 conn(0x55b4ea36ecb0 msgr2=0x55b4ea37aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 -- 172.21.15.113:0/86063587 shutdown_connections 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.374+0000 7fd66ddac740 1 -- 172.21.15.113:0/86063587 wait complete. 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 -- start start 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 Processor -- start 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea3ec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea3ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea3f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641ac5f0 con 0x55b4ea374580 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641ada60 con 0x55b4ea37da80 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641accc0 con 0x55b4ea374ea0 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66ddac740 10 client.0 _ll_register_callbacks cb 0x7fd658009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea3ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea3ec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34574/0 (socket says 172.21.15.113:34574) 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 -- 172.21.15.113:0/2351833948 learned_addr learned my addr 172.21.15.113:0/2351833948 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd663fff640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea3ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd66967d640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea3f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 -- 172.21.15.113:0/2351833948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 msgr2=0x55b4ea3ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea3ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 -- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 msgr2=0x55b4ea3f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea3f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd668e7c640 1 -- 172.21.15.113:0/2351833948 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ea3f7650 con 0x55b4ea374580 2024-04-22T07:24:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.375+0000 7fd663fff640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea3ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd668e7c640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea3ec280 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd654011790 tx=0x7fd6540117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806182e3822c8f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd643fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd643fff640 1 -- 172.21.15.113:0/2351833948 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654030060 con 0x55b4ea374580 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd643fff640 1 -- 172.21.15.113:0/2351833948 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fd654011df0 con 0x55b4ea374580 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd65401adc0 con 0x55b4ea374580 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd643fff640 1 -- 172.21.15.113:0/2351833948 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd654037050 con 0x55b4ea374580 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 10 client.17811 fetch_fsmap learned FSMap version 52 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b4ea3f89a0 con 0x55b4ea374580 2024-04-22T07:24:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd643fff640 1 -- 172.21.15.113:0/2351833948 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd65401adc0 con 0x55b4ea374580 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd643fff640 1 -- 172.21.15.113:0/2351833948 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fd65402a030 con 0x55b4ea374580 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 10 client.17811 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 20 client.17811 populate_metadata read hostname 'smithi113' 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea3f9150 0x55b4ea3fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 4 client.17811 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.376+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4ea2e0a70 con 0x55b4ea3f9150 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd663fff640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea3f9150 0x55b4ea3fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd663fff640 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea3f9150 0x55b4ea3fb570 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67c5719c0873a2 server_cookie=65c508113e19077 in_seq=0 out_seq=0 2024-04-22T07:24:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd643fff640 10 client.17811 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd643fff640 1 -- 172.21.15.113:0/2351833948 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b4ea2e0a70 con 0x55b4ea3f9150 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd643fff640 10 client.17811 handle_command_reply: tid=0 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 client.17811 shutdown 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 msgr2=0x55b4ea3ec280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea3ec280 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd654011790 tx=0x7fd6540117c0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 shutdown_connections 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x55b4ea3f9150 0x55b4ea3fb570 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4ea37da80 0x55b4ea3f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd643fff640 0 client.17811 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:24:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b4ea374ea0 0x55b4ea3ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 --2- 172.21.15.113:0/2351833948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b4ea374580 0x55b4ea3ec280 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 >> 172.21.15.113:0/2351833948 conn(0x55b4ea36ecb0 msgr2=0x55b4ea36dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 shutdown_connections 2024-04-22T07:24:35.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 1 -- 172.21.15.113:0/2351833948 wait complete. 2024-04-22T07:24:35.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.377+0000 7fd66ddac740 20 client.17811 trim_cache size 0 max 16384 2024-04-22T07:24:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": { 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: "16d404af-17ef-4550-8e73-105b18db6e55": { 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: "path": "/", 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: "tag": "16d404af-17ef-4550-8e73-105b18db6e55", 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: "options": "recursive,force" 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:24:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd64c0a3d20 msgr2=0x7fd64c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd64c0a3d20 0x7fd64c0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd658036040 tx=0x7fd65802c690 comp rx=0 tx=0).stop 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 msgr2=0x7fd664198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 0x7fd664198ad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00eaa0 tx=0x7fd65c00ef60 comp rx=0 tx=0).stop 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 shutdown_connections 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd64c0a3d20 0x7fd64c0a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd664198650 0x7fd664198ad0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd664107270 0x7fd66419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 --2- 172.21.15.113:0/2653769306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6641021a0 0x7fd66419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 >> 172.21.15.113:0/2653769306 conn(0x7fd6640793e0 msgr2=0x7fd6640ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 shutdown_connections 2024-04-22T07:24:35.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:35.378+0000 7fd66b0e0640 1 -- 172.21.15.113:0/2653769306 wait complete. 2024-04-22T07:24:35.389 INFO:tasks.cephfs.filesystem:scrub status for tag:16d404af-17ef-4550-8e73-105b18db6e55 - {'path': '/', 'tag': '16d404af-17ef-4550-8e73-105b18db6e55', 'options': 'recursive,force'} 2024-04-22T07:24:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:36 smithi133 ceph-mon[31198]: scrub queued for path: / 2024-04-22T07:24:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:36 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:36 smithi133 ceph-mon[31198]: scrub summary: active paths [/] 2024-04-22T07:24:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:36 smithi133 ceph-mon[31198]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:36 smithi133 ceph-mon[31198]: pgmap v308: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:24:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:36 smithi133 ceph-mon[31198]: scrub summary: idle 2024-04-22T07:24:37.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:36 smithi154 ceph-mon[31862]: scrub queued for path: / 2024-04-22T07:24:37.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:36 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:37.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:36 smithi154 ceph-mon[31862]: scrub summary: active paths [/] 2024-04-22T07:24:37.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:36 smithi154 ceph-mon[31862]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:37.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:36 smithi154 ceph-mon[31862]: pgmap v308: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:24:37.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:36 smithi154 ceph-mon[31862]: scrub summary: idle 2024-04-22T07:24:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:36 smithi113 ceph-mon[27933]: scrub queued for path: / 2024-04-22T07:24:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:36 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:36 smithi113 ceph-mon[27933]: scrub summary: active paths [/] 2024-04-22T07:24:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:36 smithi113 ceph-mon[27933]: scrub summary: idle+waiting paths [/] 2024-04-22T07:24:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:36 smithi113 ceph-mon[27933]: pgmap v308: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:24:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:36 smithi113 ceph-mon[27933]: scrub summary: idle 2024-04-22T07:24:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:38 smithi133 ceph-mon[31198]: pgmap v309: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:39.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:38 smithi154 ceph-mon[31862]: pgmap v309: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:39.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:38 smithi113 ceph-mon[27933]: pgmap v309: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:41.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:40 smithi133 ceph-mon[31198]: pgmap v310: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:41.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:40 smithi154 ceph-mon[31862]: pgmap v310: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:41.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:40 smithi113 ceph-mon[27933]: pgmap v310: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:43.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:42 smithi133 ceph-mon[31198]: pgmap v311: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:24:43.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:42 smithi154 ceph-mon[31862]: pgmap v311: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:24:43.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:42 smithi113 ceph-mon[27933]: pgmap v311: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-22T07:24:44.866 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:44 smithi133 ceph-mon[31198]: pgmap v312: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:44.867 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:44 smithi154 ceph-mon[31862]: pgmap v312: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:45.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:44 smithi113 ceph-mon[27933]: pgmap v312: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-22T07:24:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:46 smithi133 ceph-mon[31198]: pgmap v313: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-22T07:24:47.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:46 smithi154 ceph-mon[31862]: pgmap v313: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-22T07:24:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:46 smithi113 ceph-mon[27933]: pgmap v313: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s 2024-04-22T07:24:49.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:48 smithi133 ceph-mon[31198]: pgmap v314: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:49.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:48 smithi154 ceph-mon[31862]: pgmap v314: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:49.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:48 smithi113 ceph-mon[27933]: pgmap v314: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:51.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:50 smithi133 ceph-mon[31198]: pgmap v315: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:51.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:50 smithi154 ceph-mon[31862]: pgmap v315: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:51.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:50 smithi113 ceph-mon[27933]: pgmap v315: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:53.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:52 smithi133 ceph-mon[31198]: pgmap v316: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:53.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:52 smithi154 ceph-mon[31862]: pgmap v316: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:53.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:52 smithi113 ceph-mon[27933]: pgmap v316: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:55.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:54 smithi133 ceph-mon[31198]: pgmap v317: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:55.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:54 smithi154 ceph-mon[31862]: pgmap v317: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:55.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:54 smithi113 ceph-mon[27933]: pgmap v317: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-22T07:24:57.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:56 smithi133 ceph-mon[31198]: pgmap v318: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:57.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:56 smithi154 ceph-mon[31862]: pgmap v318: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:57.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:56 smithi113 ceph-mon[27933]: pgmap v318: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:24:57.781 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 -- 172.21.15.113:0/216819551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88107290 msgr2=0x7f1a8810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 --2- 172.21.15.113:0/216819551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88107290 0x7f1a8810ee60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009ca0 tx=0x7f1a8002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 -- 172.21.15.113:0/216819551 shutdown_connections 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 --2- 172.21.15.113:0/216819551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88107290 0x7f1a8810ee60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 --2- 172.21.15.113:0/216819551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a88102b70 0x7f1a88106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 --2- 172.21.15.113:0/216819551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a881021c0 0x7f1a881025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 -- 172.21.15.113:0/216819551 >> 172.21.15.113:0/216819551 conn(0x7f1a880fda20 msgr2=0x7f1a880ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.893+0000 7f1a90352640 1 -- 172.21.15.113:0/216819551 shutdown_connections 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 -- 172.21.15.113:0/216819551 wait complete. 2024-04-22T07:24:57.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 Processor -- start 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 -- start start 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a881021c0 0x7f1a88198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a88107290 0x7f1a88198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 0x7f1a8819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a88112330 con 0x7f1a8819cfe0 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a881121b0 con 0x7f1a881021c0 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a90352640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1a881124b0 con 0x7f1a88107290 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a8e8ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 0x7f1a8819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a8e8ef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 0x7f1a8819d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45192/0 (socket says 172.21.15.113:45192) 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.894+0000 7f1a8e8ef640 1 -- 172.21.15.113:0/4012934516 learned_addr learned my addr 172.21.15.113:0/4012934516 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e0ee640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a881021c0 0x7f1a88198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8d8ed640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a88107290 0x7f1a88198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:57.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e8ef640 1 -- 172.21.15.113:0/4012934516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a88107290 msgr2=0x7f1a88198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e8ef640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a88107290 0x7f1a88198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e8ef640 1 -- 172.21.15.113:0/4012934516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a881021c0 msgr2=0x7f1a88198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e8ef640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a881021c0 0x7f1a88198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e8ef640 1 -- 172.21.15.113:0/4012934516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8819dad0 con 0x7f1a8819cfe0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8d8ed640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a88107290 0x7f1a88198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a8e8ef640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 0x7f1a8819d490 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1a8002c700 tx=0x7f1a80035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80039040 con 0x7f1a8819cfe0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8819dd60 con 0x7f1a8819cfe0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1a80044070 con 0x7f1a8819cfe0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800043e0 con 0x7f1a8819cfe0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.895+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a881a5530 con 0x7f1a8819cfe0 2024-04-22T07:24:57.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.896+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8804fce0 con 0x7f1a8819cfe0 2024-04-22T07:24:57.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.897+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1a80004580 con 0x7f1a8819cfe0 2024-04-22T07:24:57.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.898+0000 7f1a7f7fe640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a740a3dc0 0x7f1a740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:57.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.898+0000 7f1a8e0ee640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a740a3dc0 0x7f1a740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:57.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.898+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f1a800f2ed0 con 0x7f1a8819cfe0 2024-04-22T07:24:57.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.898+0000 7f1a8e0ee640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a740a3dc0 0x7f1a740a6280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a84005fd0 tx=0x7f1a8400b390 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:57.902+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a800bf4e0 con 0x7f1a8819cfe0 2024-04-22T07:24:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.101+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1a88102d50 con 0x7f1a740a3dc0 2024-04-22T07:24:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.110+0000 7f1a7f7fe640 1 -- 172.21.15.113:0/4012934516 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f1a88102d50 con 0x7f1a740a3dc0 2024-04-22T07:24:58.107 INFO:teuthology.orchestra.run.smithi113.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-22T00:00:00", "created": "2024-04-22T07:14:56", "first": "2024-04-22T07:15:00", "last": "2024-04-22T07:24:00", "last_pruned": "2024-04-22T07:24:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-22T07:24:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a740a3dc0 msgr2=0x7f1a740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a740a3dc0 0x7f1a740a6280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a84005fd0 tx=0x7f1a8400b390 comp rx=0 tx=0).stop 2024-04-22T07:24:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 msgr2=0x7f1a8819d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 0x7f1a8819d490 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1a8002c700 tx=0x7f1a80035d80 comp rx=0 tx=0).stop 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 shutdown_connections 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f1a740a3dc0 0x7f1a740a6280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a8819cfe0 0x7f1a8819d490 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a88107290 0x7f1a88198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 --2- 172.21.15.113:0/4012934516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a881021c0 0x7f1a88198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 >> 172.21.15.113:0/4012934516 conn(0x7f1a880fda20 msgr2=0x7f1a880fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 shutdown_connections 2024-04-22T07:24:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.112+0000 7f1a90352640 1 -- 172.21.15.113:0/4012934516 wait complete. 2024-04-22T07:24:58.119 DEBUG:teuthology.orchestra.run.smithi113:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-22T07:24:58.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 -- 172.21.15.113:0/3819581834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 msgr2=0x7f6138101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 --2- 172.21.15.113:0/3819581834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f6138101600 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6120009930 tx=0x7f612002bc30 comp rx=0 tx=0).stop 2024-04-22T07:24:58.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 -- 172.21.15.113:0/3819581834 shutdown_connections 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 --2- 172.21.15.113:0/3819581834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6138101b40 0x7f613810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 --2- 172.21.15.113:0/3819581834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f6138101600 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 --2- 172.21.15.113:0/3819581834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6138106fe0 0x7f61381073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 -- 172.21.15.113:0/3819581834 >> 172.21.15.113:0/3819581834 conn(0x7f6138076b80 msgr2=0x7f6138076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 -- 172.21.15.113:0/3819581834 shutdown_connections 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.221+0000 7f613db59640 1 -- 172.21.15.113:0/3819581834 wait complete. 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 Processor -- start 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 -- start start 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f61381995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6138101b40 0x7f6138199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6138106fe0 0x7f61381a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6138111210 con 0x7f61381011a0 2024-04-22T07:24:58.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6138111090 con 0x7f6138101b40 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f613db59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6138111390 con 0x7f6138106fe0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f61377fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f61381995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f61377fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f61381995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45208/0 (socket says 172.21.15.113:45208) 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f61377fe640 1 -- 172.21.15.113:0/2711417277 learned_addr learned my addr 172.21.15.113:0/2711417277 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f612ffff640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6138101b40 0x7f6138199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.222+0000 7f6137fff640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6138106fe0 0x7f61381a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61377fe640 1 -- 172.21.15.113:0/2711417277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6138106fe0 msgr2=0x7f61381a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61377fe640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6138106fe0 0x7f61381a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61377fe640 1 -- 172.21.15.113:0/2711417277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6138101b40 msgr2=0x7f6138199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61377fe640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6138101b40 0x7f6138199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61377fe640 1 -- 172.21.15.113:0/2711417277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61381a4bb0 con 0x7f61381011a0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61377fe640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f61381995f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f612800c890 tx=0x7f612800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128004220 con 0x7f61381011a0 2024-04-22T07:24:58.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f61280043c0 con 0x7f61381011a0 2024-04-22T07:24:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61381a4d80 con 0x7f61381011a0 2024-04-22T07:24:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280049c0 con 0x7f61381011a0 2024-04-22T07:24:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.223+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61381a5240 con 0x7f61381011a0 2024-04-22T07:24:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.224+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6138102550 con 0x7f61381011a0 2024-04-22T07:24:58.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.225+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f6128005140 con 0x7f61381011a0 2024-04-22T07:24:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.225+0000 7f61357fa640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61240a3dc0 0x7f61240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:24:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.226+0000 7f612ffff640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61240a3dc0 0x7f61240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:24:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.226+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f61280d06b0 con 0x7f61381011a0 2024-04-22T07:24:58.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.226+0000 7f612ffff640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61240a3dc0 0x7f61240a6280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6120006010 tx=0x7f6120000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:24:58.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.230+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6128014030 con 0x7f61381011a0 2024-04-22T07:24:58.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.428+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 --> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6138100560 con 0x7f61240a3dc0 2024-04-22T07:24:58.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.436+0000 7f61357fa640 1 -- 172.21.15.113:0/2711417277 <== mgr.15279 v2:172.21.15.113:6800/2258977739 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6138100560 con 0x7f61240a3dc0 2024-04-22T07:24:58.433 INFO:teuthology.orchestra.run.smithi113.stdout:/ 1m 6m3h 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61240a3dc0 msgr2=0x7f61240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61240a3dc0 0x7f61240a6280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6120006010 tx=0x7f6120000bc0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 msgr2=0x7f61381995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f61381995f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f612800c890 tx=0x7f612800cd50 comp rx=0 tx=0).stop 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 shutdown_connections 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f61240a3dc0 0x7f61240a6280 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6138106fe0 0x7f61381a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6138101b40 0x7f6138199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 --2- 172.21.15.113:0/2711417277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61381011a0 0x7f61381995f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 >> 172.21.15.113:0/2711417277 conn(0x7f6138076b80 msgr2=0x7f61380ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 shutdown_connections 2024-04-22T07:24:58.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:24:58.438+0000 7f613db59640 1 -- 172.21.15.113:0/2711417277 wait complete. 2024-04-22T07:24:58.446 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-22T07:24:58.456 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-22T07:24:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:58 smithi113 ceph-mon[27933]: pgmap v319: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:58.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:58 smithi113 ceph-mon[27933]: from='client.17817 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:24:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:58 smithi133 ceph-mon[31198]: pgmap v319: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:59.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:58 smithi133 ceph-mon[31198]: from='client.17817 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:24:59.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:58 smithi154 ceph-mon[31862]: pgmap v319: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:24:59.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:58 smithi154 ceph-mon[31862]: from='client.17817 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:25:00.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:24:59 smithi133 ceph-mon[31198]: from='client.17823 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:25:00.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:24:59 smithi154 ceph-mon[31862]: from='client.17823 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:25:00.083 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:24:59 smithi113 ceph-mon[27933]: from='client.17823 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T07:25:01.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:00 smithi133 ceph-mon[31198]: pgmap v320: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:01.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:00 smithi154 ceph-mon[31862]: pgmap v320: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:01.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:00 smithi113 ceph-mon[27933]: pgmap v320: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:02.886 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:02 smithi113 ceph-mon[27933]: pgmap v321: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-22T07:25:03.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:02 smithi133 ceph-mon[31198]: pgmap v321: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-22T07:25:03.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:02 smithi154 ceph-mon[31862]: pgmap v321: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-22T07:25:03.746 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:03 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:25:03.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:03 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:25:03.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:03 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T07:25:05.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T07:25:05.390 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T07:25:05.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.494+0000 7fdb4fb23640 1 -- 172.21.15.113:0/1309101914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb48102b50 msgr2=0x7fdb48106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.494+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/1309101914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb48102b50 0x7fdb48106b90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009900 tx=0x7fdb3802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- 172.21.15.113:0/1309101914 shutdown_connections 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/1309101914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 0x7fdb4810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/1309101914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb48102b50 0x7fdb48106b90 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/1309101914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb481021a0 0x7fdb48102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- 172.21.15.113:0/1309101914 >> 172.21.15.113:0/1309101914 conn(0x7fdb480793e0 msgr2=0x7fdb480797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- 172.21.15.113:0/1309101914 shutdown_connections 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- 172.21.15.113:0/1309101914 wait complete. 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 Processor -- start 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- start start 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 0x7fdb4819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 0x7fdb4819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb48198670 0x7fdb48198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481122f0 con 0x7fdb481021a0 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112170 con 0x7fdb48198670 2024-04-22T07:25:05.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.495+0000 7fdb4fb23640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112470 con 0x7fdb48107270 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 0x7fdb4819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 0x7fdb4819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45230/0 (socket says 172.21.15.113:45230) 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 -- 172.21.15.113:0/4156734144 learned_addr learned my addr 172.21.15.113:0/4156734144 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb48198670 0x7fdb48198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 0x7fdb4819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 -- 172.21.15.113:0/4156734144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 msgr2=0x7fdb4819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 0x7fdb4819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 -- 172.21.15.113:0/4156734144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb48198670 msgr2=0x7fdb48198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb48198670 0x7fdb48198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 -- 172.21.15.113:0/4156734144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb48199270 con 0x7fdb481021a0 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 0x7fdb4819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb48198670 0x7fdb48198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4d8bf640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 0x7fdb4819e5b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000eab0 tx=0x7fdb4000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb3effd640 1 -- 172.21.15.113:0/4156734144 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000ce40 con 0x7fdb481021a0 2024-04-22T07:25:05.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb3effd640 1 -- 172.21.15.113:0/4156734144 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdb40011940 con 0x7fdb481021a0 2024-04-22T07:25:05.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb481a5350 con 0x7fdb481021a0 2024-04-22T07:25:05.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb3effd640 1 -- 172.21.15.113:0/4156734144 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40010aa0 con 0x7fdb481021a0 2024-04-22T07:25:05.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.496+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a5700 con 0x7fdb481021a0 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.498+0000 7fdb3effd640 1 -- 172.21.15.113:0/4156734144 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fdb40011af0 con 0x7fdb481021a0 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 Processor -- start 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 -- start start 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737580 0x5590e0737960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb3effd640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb340a3d20 0x7fdb340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e0740540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e0742e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5590e069c1e0 con 0x5590e0737ea0 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5590e0639390 con 0x5590e0737580 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb527ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5590e06981a0 con 0x5590e0740a80 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb4d0be640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e0740540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb4d8bf640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb340a3d20 0x7fdb340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb4d0be640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e0740540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45246/0 (socket says 172.21.15.113:45246) 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.499+0000 7fdb4d0be640 1 -- 172.21.15.113:0/4252719865 learned_addr learned my addr 172.21.15.113:0/4252719865 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737580 0x5590e0737960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb3effd640 1 -- 172.21.15.113:0/4156734144 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fdb400d10e0 con 0x7fdb481021a0 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e0742e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d0be640 1 -- 172.21.15.113:0/4252719865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 msgr2=0x5590e0742e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e0742e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d0be640 1 -- 172.21.15.113:0/4252719865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737580 msgr2=0x5590e0737960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737580 0x5590e0737960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d0be640 1 -- 172.21.15.113:0/4252719865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e04787d0 con 0x5590e0737ea0 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737580 0x5590e0737960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e0740540 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdb38000c00 tx=0x7fdb3802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b430b96c84fae4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb4d8bf640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb340a3d20 0x7fdb340a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb4009bd40 tx=0x7fdb400bd040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb33fff640 1 -- 172.21.15.113:0/4252719865 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3802ca50 con 0x5590e0737ea0 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb33fff640 1 -- 172.21.15.113:0/4252719865 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdb3802cbf0 con 0x5590e0737ea0 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.500+0000 7fdb33fff640 1 -- 172.21.15.113:0/4252719865 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38004920 con 0x5590e0737ea0 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 -- 172.21.15.113:0/4252719865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 msgr2=0x5590e0740540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e0740540 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdb38000c00 tx=0x7fdb3802c5d0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 -- 172.21.15.113:0/4252719865 shutdown_connections 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e0742e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e0740540 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 --2- 172.21.15.113:0/4252719865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737580 0x5590e0737960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 -- 172.21.15.113:0/4252719865 >> 172.21.15.113:0/4252719865 conn(0x5590e0731cb0 msgr2=0x5590e0732090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 -- 172.21.15.113:0/4252719865 shutdown_connections 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 -- 172.21.15.113:0/4252719865 wait complete. 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 -- start start 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.501+0000 7fdb527ef740 1 Processor -- start 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e073da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 0x5590e073dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e073e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5590e0743810 con 0x5590e0737580 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5590e06552f0 con 0x5590e0737ea0 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5590e0743990 con 0x5590e0740a80 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 10 client.0 _ll_register_callbacks cb 0x5590e0662580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e073da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e073da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45250/0 (socket says 172.21.15.113:45250) 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/1093726221 learned_addr learned my addr 172.21.15.113:0/1093726221 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/1093726221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 msgr2=0x5590e073e4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e073e4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 msgr2=0x5590e073dfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 0x5590e073dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/1093726221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e03ae3f0 con 0x5590e0737580 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e073da70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fdb44012c20 tx=0x7fdb440117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb04853e5f804c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb317fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb317fa640 1 -- 172.21.15.113:0/1093726221 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44012e50 con 0x5590e0737580 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb317fa640 1 -- 172.21.15.113:0/1093726221 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fdb4401f440 con 0x5590e0737580 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.502+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3802cbf0 con 0x5590e0737580 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb317fa640 1 -- 172.21.15.113:0/1093726221 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdb4401f850 con 0x5590e0737580 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 10 client.17835 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5590e036e0f0 con 0x5590e0737580 2024-04-22T07:25:05.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb317fa640 1 -- 172.21.15.113:0/1093726221 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdb3802cbf0 con 0x5590e0737580 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb317fa640 1 -- 172.21.15.113:0/1093726221 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fdb4401d020 con 0x5590e0737580 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 10 client.17835 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 20 client.17835 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e0750ad0 0x5590e0752ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 4 client.17835 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e06a9c00 con 0x5590e0750ad0 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.503+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e0750ad0 0x5590e0752ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.504+0000 7fdb317fa640 10 client.17835 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.504+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e0750ad0 0x5590e0752ec0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4177626b83e0bf53 server_cookie=bb81cb819eac2dc in_seq=0 out_seq=0 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.504+0000 7fdb317fa640 1 -- 172.21.15.113:0/1093726221 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590e06a9c00 con 0x5590e0750ad0 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.504+0000 7fdb317fa640 10 client.17835 handle_command_reply: tid=0 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.504+0000 7fdb527ef740 1 client.17835 shutdown 2024-04-22T07:25:05.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 msgr2=0x5590e073da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e073da70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fdb44012c20 tx=0x7fdb440117b0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 shutdown_connections 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e0750ad0 0x5590e0752ec0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e073e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 0x5590e073dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 --2- 172.21.15.113:0/1093726221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e073da70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb317fa640 0 client.17835 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 >> 172.21.15.113:0/1093726221 conn(0x5590e0731cb0 msgr2=0x5590e0733a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 shutdown_connections 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 1 -- 172.21.15.113:0/1093726221 wait complete. 2024-04-22T07:25:05.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.505+0000 7fdb527ef740 20 client.17835 trim_cache size 0 max 16384 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 Processor -- start 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 -- start start 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e07401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 0x5590e074c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e073c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481b08d0 con 0x5590e0737580 2024-04-22T07:25:05.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ae1a0 con 0x5590e0737ea0 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb527ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ac010 con 0x5590e0740a80 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e07401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e07401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45260/0 (socket says 172.21.15.113:45260) 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/136809535 learned_addr learned my addr 172.21.15.113:0/136809535 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/136809535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 msgr2=0x5590e073c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e073c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/136809535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 msgr2=0x5590e074c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 0x5590e074c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.522+0000 7fdb4e0c0640 1 -- 172.21.15.113:0/136809535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e0731ae0 con 0x5590e0737580 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e07401a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005ca0 tx=0x7fdb4400cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8958f6f4ba18d89a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb337fe640 1 -- 172.21.15.113:0/136809535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4401fbf0 con 0x5590e0737580 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb337fe640 1 -- 172.21.15.113:0/136809535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdb4401fd90 con 0x5590e0737580 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb337fe640 1 -- 172.21.15.113:0/136809535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44004d00 con 0x5590e0737580 2024-04-22T07:25:05.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 -- 172.21.15.113:0/136809535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 msgr2=0x5590e07401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e07401a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005ca0 tx=0x7fdb4400cfb0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 -- 172.21.15.113:0/136809535 shutdown_connections 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0740a80 0x5590e073c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0737ea0 0x5590e074c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 --2- 172.21.15.113:0/136809535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737580 0x5590e07401a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 -- 172.21.15.113:0/136809535 >> 172.21.15.113:0/136809535 conn(0x5590e0731cb0 msgr2=0x5590e0730d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 -- 172.21.15.113:0/136809535 shutdown_connections 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.523+0000 7fdb527ef740 1 -- 172.21.15.113:0/136809535 wait complete. 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 -- start start 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 Processor -- start 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0737580 0x5590e07af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e07af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0740a80 0x5590e07b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ac660 con 0x5590e0737ea0 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481adad0 con 0x5590e0740a80 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481acd30 con 0x5590e0737580 2024-04-22T07:25:05.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb527ef740 10 client.0 _ll_register_callbacks cb 0x7fdb44012830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb4d0be640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e07af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb4d0be640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e07af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45266/0 (socket says 172.21.15.113:45266) 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb4d0be640 1 -- 172.21.15.113:0/751098069 learned_addr learned my addr 172.21.15.113:0/751098069 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0737580 0x5590e07af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.524+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0740a80 0x5590e07b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4d0be640 1 -- 172.21.15.113:0/751098069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0737580 msgr2=0x5590e07af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0737580 0x5590e07af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4d0be640 1 -- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0740a80 msgr2=0x5590e07b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0740a80 0x5590e07b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4d0be640 1 -- 172.21.15.113:0/751098069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e0742e70 con 0x5590e0737ea0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0740a80 0x5590e07b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb4d0be640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e07af730 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fdb380096c0 tx=0x7fdb3802c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fcc38d53246535 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb317fa640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb317fa640 1 -- 172.21.15.113:0/751098069 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3802cc20 con 0x5590e0737ea0 2024-04-22T07:25:05.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb317fa640 1 -- 172.21.15.113:0/751098069 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fdb38004040 con 0x5590e0737ea0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4401fd90 con 0x5590e0737ea0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb317fa640 1 -- 172.21.15.113:0/751098069 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdb3805d050 con 0x5590e0737ea0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 10 client.17841 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5590e07bb810 con 0x5590e0737ea0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb317fa640 1 -- 172.21.15.113:0/751098069 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdb4401fd90 con 0x5590e0737ea0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb317fa640 1 -- 172.21.15.113:0/751098069 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fdb380037f0 con 0x5590e0737ea0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 10 client.17841 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 20 client.17841 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e07bbc60 0x5590e07be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 4 client.17841 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.525+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590e06a3a70 con 0x5590e07bbc60 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e07bbc60 0x5590e07be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb4e0c0640 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e07bbc60 0x5590e07be050 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80094bddc4849567 server_cookie=231ab4fa5054e592 in_seq=0 out_seq=0 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb317fa640 10 client.17841 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb317fa640 1 -- 172.21.15.113:0/751098069 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5590e06a3a70 con 0x5590e07bbc60 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb317fa640 10 client.17841 handle_command_reply: tid=0 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 client.17841 shutdown 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 msgr2=0x5590e07af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e07af730 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fdb380096c0 tx=0x7fdb3802c510 comp rx=0 tx=0).stop 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 shutdown_connections 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5590e07bbc60 0x5590e07be050 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590e0740a80 0x5590e07b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5590e0737ea0 0x5590e07af730 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb317fa640 0 client.17841 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.526+0000 7fdb527ef740 1 --2- 172.21.15.113:0/751098069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5590e0737580 0x5590e07af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 >> 172.21.15.113:0/751098069 conn(0x5590e0731cb0 msgr2=0x5590e07304e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 shutdown_connections 2024-04-22T07:25:05.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb527ef740 1 -- 172.21.15.113:0/751098069 wait complete. 2024-04-22T07:25:05.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb527ef740 20 client.17841 trim_cache size 0 max 16384 2024-04-22T07:25:05.532 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:25:05.532 INFO:teuthology.orchestra.run.smithi113.stdout: "status": "no active scrubs running", 2024-04-22T07:25:05.532 INFO:teuthology.orchestra.run.smithi113.stdout: "scrubs": {} 2024-04-22T07:25:05.532 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb340a3d20 msgr2=0x7fdb340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb340a3d20 0x7fdb340a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb4009bd40 tx=0x7fdb400bd040 comp rx=0 tx=0).stop 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 msgr2=0x7fdb4819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 0x7fdb4819e5b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000eab0 tx=0x7fdb4000ef70 comp rx=0 tx=0).stop 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 shutdown_connections 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb340a3d20 0x7fdb340a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb48198670 0x7fdb48198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb48107270 0x7fdb4819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 --2- 172.21.15.113:0/4156734144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb481021a0 0x7fdb4819e5b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 >> 172.21.15.113:0/4156734144 conn(0x7fdb480793e0 msgr2=0x7fdb480ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 shutdown_connections 2024-04-22T07:25:05.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.527+0000 7fdb4fb23640 1 -- 172.21.15.113:0/4156734144 wait complete. 2024-04-22T07:25:05.536 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T07:25:05.536 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: pgmap v322: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:25:05.585 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:05 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1072849546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78101110 msgr2=0x7fdb78101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1072849546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78101110 0x7fdb78101590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009900 tx=0x7fdb6802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1072849546 shutdown_connections 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1072849546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101ad0 0x7fdb7810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1072849546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78101110 0x7fdb78101590 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1072849546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb78106f30 0x7fdb78107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1072849546 >> 172.21.15.113:0/1072849546 conn(0x7fdb78077e30 msgr2=0x7fdb78078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1072849546 shutdown_connections 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1072849546 wait complete. 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.637+0000 7fdb7ff59640 1 Processor -- start 2024-04-22T07:25:05.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 -- start start 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101110 0x7fdb7819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb78101ad0 0x7fdb7819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 0x7fdb781985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78111200 con 0x7fdb78106f30 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78111080 con 0x7fdb78101ad0 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7ff59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78111380 con 0x7fdb78101110 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 0x7fdb781985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 0x7fdb781985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43224/0 (socket says 172.21.15.113:43224) 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 -- 172.21.15.113:0/1537137645 learned_addr learned my addr 172.21.15.113:0/1537137645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7d4f4640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb78101ad0 0x7fdb7819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7dcf5640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101110 0x7fdb7819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 -- 172.21.15.113:0/1537137645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101110 msgr2=0x7fdb7819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101110 0x7fdb7819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 -- 172.21.15.113:0/1537137645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb78101ad0 msgr2=0x7fdb7819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb78101ad0 0x7fdb7819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 -- 172.21.15.113:0/1537137645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb78198d20 con 0x7fdb78106f30 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb7e4f6640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 0x7fdb781985a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000d750 tx=0x7fdb7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.638+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70017070 con 0x7fdb78106f30 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.639+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdb700054f0 con 0x7fdb78106f30 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.639+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70004cd0 con 0x7fdb78106f30 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.639+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb78199010 con 0x7fdb78106f30 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.639+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781a52b0 con 0x7fdb78106f30 2024-04-22T07:25:05.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.639+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb78069230 con 0x7fdb78106f30 2024-04-22T07:25:05.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.641+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fdb70009bd0 con 0x7fdb78106f30 2024-04-22T07:25:05.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.641+0000 7fdb6effd640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb640a3d20 0x7fdb640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.641+0000 7fdb7dcf5640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb640a3d20 0x7fdb640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.641+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fdb700d0850 con 0x7fdb78106f30 2024-04-22T07:25:05.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.642+0000 7fdb7dcf5640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb640a3d20 0x7fdb640a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005fd0 tx=0x7fdb74009500 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.645+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb7009c700 con 0x7fdb78106f30 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: pgmap v322: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:25:05.815 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:05 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: pgmap v322: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T07:25:05.817 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:05 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' 2024-04-22T07:25:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.859+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb781a5510 con 0x7fdb78106f30 2024-04-22T07:25:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.860+0000 7fdb6effd640 1 -- 172.21.15.113:0/1537137645 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7fdb700a0a00 con 0x7fdb78106f30 2024-04-22T07:25:05.859 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:05.859 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:25:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:25:05.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb640a3d20 msgr2=0x7fdb640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb640a3d20 0x7fdb640a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005fd0 tx=0x7fdb74009500 comp rx=0 tx=0).stop 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 msgr2=0x7fdb781985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 0x7fdb781985a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000d750 tx=0x7fdb7000dc10 comp rx=0 tx=0).stop 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 shutdown_connections 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fdb640a3d20 0x7fdb640a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb78106f30 0x7fdb781985a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb78101ad0 0x7fdb7819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 --2- 172.21.15.113:0/1537137645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb78101110 0x7fdb7819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 >> 172.21.15.113:0/1537137645 conn(0x7fdb78077e30 msgr2=0x7fdb7806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 shutdown_connections 2024-04-22T07:25:05.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.862+0000 7fdb7ff59640 1 -- 172.21.15.113:0/1537137645 wait complete. 2024-04-22T07:25:05.872 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 -- 172.21.15.113:0/1047479014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 msgr2=0x7fe804100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 --2- 172.21.15.113:0/1047479014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe804100ed0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009900 tx=0x7fe7f402bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 -- 172.21.15.113:0/1047479014 shutdown_connections 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 --2- 172.21.15.113:0/1047479014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 0x7fe80410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 --2- 172.21.15.113:0/1047479014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8041014a0 0x7fe804105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 --2- 172.21.15.113:0/1047479014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe804100ed0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 -- 172.21.15.113:0/1047479014 >> 172.21.15.113:0/1047479014 conn(0x7fe8040760e0 msgr2=0x7fe8040764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 -- 172.21.15.113:0/1047479014 shutdown_connections 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.973+0000 7fe809a8c640 1 -- 172.21.15.113:0/1047479014 wait complete. 2024-04-22T07:25:05.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 Processor -- start 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 -- start start 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe80419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 0x7fe80419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe804198690 0x7fe804198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041109e0 con 0x7fe804100af0 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe804110860 con 0x7fe804198690 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe809a8c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe804110b60 con 0x7fe804105970 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe80419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe80419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43238/0 (socket says 172.21.15.113:43238) 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe8037fe640 1 -- 172.21.15.113:0/165918042 learned_addr learned my addr 172.21.15.113:0/165918042 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.974+0000 7fe803fff640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe804198690 0x7fe804198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe802ffd640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 0x7fe80419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe8037fe640 1 -- 172.21.15.113:0/165918042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 msgr2=0x7fe80419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe8037fe640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 0x7fe80419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe8037fe640 1 -- 172.21.15.113:0/165918042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe804198690 msgr2=0x7fe804198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe8037fe640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe804198690 0x7fe804198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe8037fe640 1 -- 172.21.15.113:0/165918042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe804199290 con 0x7fe804100af0 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe802ffd640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 0x7fe80419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe8037fe640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe80419e5d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000c00 tx=0x7fe7f402cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe800ff9640 1 -- 172.21.15.113:0/165918042 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4041070 con 0x7fe804100af0 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe804199520 con 0x7fe804100af0 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe800ff9640 1 -- 172.21.15.113:0/165918042 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7f40044d0 con 0x7fe804100af0 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe800ff9640 1 -- 172.21.15.113:0/165918042 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4003d50 con 0x7fe804100af0 2024-04-22T07:25:05.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.975+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041a5710 con 0x7fe804100af0 2024-04-22T07:25:05.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.977+0000 7fe800ff9640 1 -- 172.21.15.113:0/165918042 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe7f4044070 con 0x7fe804100af0 2024-04-22T07:25:05.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.977+0000 7fe80c322740 1 Processor -- start 2024-04-22T07:25:05.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.977+0000 7fe80c322740 1 -- start start 2024-04-22T07:25:05.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe80c322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3971710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe80c322740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe80c322740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe80c322740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5623f38d61f0 con 0x5623f397ae40 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe80c322740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5623f38734b0 con 0x5623f3971ce0 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe80c322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5623f38d21b0 con 0x5623f3971330 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43254/0 (socket says 172.21.15.113:43254) 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 -- 172.21.15.113:0/3361057843 learned_addr learned my addr 172.21.15.113:0/3361057843 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe803fff640 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3971710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe8037fe640 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 -- 172.21.15.113:0/3361057843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 msgr2=0x5623f3971710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe800ff9640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7ec0a3d20 0x7fe7ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3971710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 -- 172.21.15.113:0/3361057843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 msgr2=0x5623f397a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 -- 172.21.15.113:0/3361057843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f38651b0 con 0x5623f397ae40 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe803fff640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7ec0a3d20 0x7fe7ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe800ff9640 1 -- 172.21.15.113:0/165918042 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fe7f40f2270 con 0x7fe804100af0 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe802ffd640 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397b2c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009c40 tx=0x7fe7f8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32320a4a1df582c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.978+0000 7fe7e9ffb640 1 -- 172.21.15.113:0/3361057843 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004030 con 0x5623f397ae40 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe7e9ffb640 1 -- 172.21.15.113:0/3361057843 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7f80041d0 con 0x5623f397ae40 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe7e9ffb640 1 -- 172.21.15.113:0/3361057843 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8012690 con 0x5623f397ae40 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 -- 172.21.15.113:0/3361057843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 msgr2=0x5623f397b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe803fff640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7ec0a3d20 0x7fe7ec0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005fd0 tx=0x7fe7fc004a40 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397b2c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009c40 tx=0x7fe7f8010740 comp rx=0 tx=0).stop 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 -- 172.21.15.113:0/3361057843 shutdown_connections 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397b2c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 --2- 172.21.15.113:0/3361057843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3971710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 -- 172.21.15.113:0/3361057843 >> 172.21.15.113:0/3361057843 conn(0x5623f396bba0 msgr2=0x5623f396bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 -- 172.21.15.113:0/3361057843 shutdown_connections 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.979+0000 7fe80c322740 1 -- 172.21.15.113:0/3361057843 wait complete. 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 -- start start 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 Processor -- start 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3981260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5623f3981dc0 con 0x5623f397ae40 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5623f388f500 con 0x5623f3971ce0 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5623f3981f40 con 0x5623f3971330 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe80c322740 10 client.0 _ll_register_callbacks cb 0x5623f385db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43266/0 (socket says 172.21.15.113:43266) 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 -- 172.21.15.113:0/2285996055 learned_addr learned my addr 172.21.15.113:0/2285996055 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:05.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe802ffd640 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 -- 172.21.15.113:0/2285996055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 msgr2=0x5623f3981260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3981260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 -- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 msgr2=0x5623f397e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.980+0000 7fe803fff640 1 -- 172.21.15.113:0/2285996055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f35e83f0 con 0x5623f397ae40 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe803fff640 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397eb70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0126a0 tx=0x7fe7fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b5f6de99f0a27e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe7e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/2285996055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc010890 con 0x5623f397ae40 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/2285996055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fe7fc010a30 con 0x5623f397ae40 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f80041d0 con 0x5623f397ae40 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/2285996055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe7fc010c30 con 0x5623f397ae40 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 10 client.17859 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:05.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5623f36b27d0 con 0x5623f397ae40 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/2285996055 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7f80041d0 con 0x5623f397ae40 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/2285996055 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fe7fc00f040 con 0x5623f397ae40 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 10 client.17859 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 20 client.17859 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f398f0e0 0x5623f39914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 4 client.17859 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623f3875930 con 0x5623f398f0e0 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.981+0000 7fe8037fe640 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f398f0e0 0x5623f39914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.982+0000 7fe8037fe640 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f398f0e0 0x5623f39914d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9176cc2fd70046a server_cookie=4d2f7413181e9c43 in_seq=0 out_seq=0 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.982+0000 7fe7e2ffd640 10 client.17859 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.982+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/2285996055 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623f3875930 con 0x5623f398f0e0 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.982+0000 7fe7e2ffd640 10 client.17859 handle_command_reply: tid=0 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.982+0000 7fe80c322740 1 client.17859 shutdown 2024-04-22T07:25:05.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 msgr2=0x5623f397eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397eb70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0126a0 tx=0x7fe7fc004ab0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 shutdown_connections 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f398f0e0 0x5623f39914d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f397eb70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971ce0 0x5623f397e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe7e2ffd640 0 client.17859 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 --2- 172.21.15.113:0/2285996055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f3981260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 >> 172.21.15.113:0/2285996055 conn(0x5623f396bba0 msgr2=0x5623f396d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 shutdown_connections 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 1 -- 172.21.15.113:0/2285996055 wait complete. 2024-04-22T07:25:05.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.983+0000 7fe80c322740 20 client.17859 trim_cache size 0 max 16384 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 Processor -- start 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 -- start start 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971330 0x5623f397c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971ce0 0x5623f398acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f396aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041ae6c0 con 0x5623f397ae40 2024-04-22T07:25:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041abf90 con 0x5623f3971330 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe80c322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041a9e00 con 0x5623f3971ce0 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f396aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f396aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43274/0 (socket says 172.21.15.113:43274) 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 -- 172.21.15.113:0/1071005775 learned_addr learned my addr 172.21.15.113:0/1071005775 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe8037fe640 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971330 0x5623f397c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe802ffd640 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971ce0 0x5623f398acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 -- 172.21.15.113:0/1071005775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971ce0 msgr2=0x5623f398acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971ce0 0x5623f398acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 -- 172.21.15.113:0/1071005775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971330 msgr2=0x5623f397c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971330 0x5623f397c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 -- 172.21.15.113:0/1071005775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f3991840 con 0x5623f397ae40 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe8037fe640 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971330 0x5623f397c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe803fff640 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f396aa80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005bc0 tx=0x7fe7fc010480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8921553d324332d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe7e97fa640 1 -- 172.21.15.113:0/1071005775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc024070 con 0x5623f397ae40 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe7e97fa640 1 -- 172.21.15.113:0/1071005775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7fc025020 con 0x5623f397ae40 2024-04-22T07:25:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.996+0000 7fe7e97fa640 1 -- 172.21.15.113:0/1071005775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0112a0 con 0x5623f397ae40 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 -- 172.21.15.113:0/1071005775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 msgr2=0x5623f396aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f396aa80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005bc0 tx=0x7fe7fc010480 comp rx=0 tx=0).stop 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 -- 172.21.15.113:0/1071005775 shutdown_connections 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f397ae40 0x5623f396aa80 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971ce0 0x5623f398acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 --2- 172.21.15.113:0/1071005775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f3971330 0x5623f397c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 -- 172.21.15.113:0/1071005775 >> 172.21.15.113:0/1071005775 conn(0x5623f396bba0 msgr2=0x5623f396a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 -- 172.21.15.113:0/1071005775 shutdown_connections 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 -- 172.21.15.113:0/1071005775 wait complete. 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.997+0000 7fe80c322740 1 -- start start 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 Processor -- start 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f39803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 0x5623f39808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f397ae40 0x5623f3980e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041aa450 con 0x5623f3971ce0 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041ab8c0 con 0x5623f397ae40 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041aab20 con 0x5623f3971330 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 10 client.0 _ll_register_callbacks cb 0x7fe7fc0048e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 0x5623f39808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 0x5623f39808e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43276/0 (socket says 172.21.15.113:43276) 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 -- 172.21.15.113:0/1442157503 learned_addr learned my addr 172.21.15.113:0/1442157503 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe8037fe640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f39803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 -- 172.21.15.113:0/1442157503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 msgr2=0x5623f39803a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f39803a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 -- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f397ae40 msgr2=0x5623f3980e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f397ae40 0x5623f3980e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 -- 172.21.15.113:0/1442157503 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f39f47c0 con 0x5623f3971ce0 2024-04-22T07:25:06.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe8037fe640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f39803a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe802ffd640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 0x5623f39808e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009c40 tx=0x7fe7f8009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd9e4fe572acc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe7e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/1442157503 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8005540 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/1442157503 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7fe7f80035f0 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.998+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7fc025020 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/1442157503 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe7f8031050 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 10 client.17865 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5623f39f5a60 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/1442157503 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7fc025020 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/1442157503 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7fe7f800f040 con 0x5623f3971ce0 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 10 client.17865 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.e 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 20 client.17865 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f39f5ea0 0x5623f39f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 4 client.17865 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 --> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623f38e3c10 con 0x5623f39f5ea0 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:05.999+0000 7fe8037fe640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f39f5ea0 0x5623f39f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe7e2ffd640 10 client.17865 ms_handle_connect on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe8037fe640 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f39f5ea0 0x5623f39f82b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=100d13b354dcde81 server_cookie=648cdaeca4bffd3f in_seq=0 out_seq=0 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe7e2ffd640 1 -- 172.21.15.113:0/1442157503 <== mds.0 v2:172.21.15.133:6834/3935683574 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623f38e3c10 con 0x5623f39f5ea0 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe7e2ffd640 10 client.17865 handle_command_reply: tid=0 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 client.17865 shutdown 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 msgr2=0x5623f39808e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 0x5623f39808e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009c40 tx=0x7fe7f8009980 comp rx=0 tx=0).stop 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 shutdown_connections 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:6834/3935683574,v1:172.21.15.133:6835/3935683574] conn(0x5623f39f5ea0 0x5623f39f82b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623f397ae40 0x5623f3980e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623f3971ce0 0x5623f39808e0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 --2- 172.21.15.113:0/1442157503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5623f3971330 0x5623f39803a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 >> 172.21.15.113:0/1442157503 conn(0x5623f396bba0 msgr2=0x5623f396ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe7e2ffd640 0 client.17865 ms_handle_reset on v2:172.21.15.133:6834/3935683574 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 shutdown_connections 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 1 -- 172.21.15.113:0/1442157503 wait complete. 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.000+0000 7fe80c322740 20 client.17865 trim_cache size 0 max 16384 2024-04-22T07:25:06.006 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:25:06.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7ec0a3d20 msgr2=0x7fe7ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7ec0a3d20 0x7fe7ec0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005fd0 tx=0x7fe7fc004a40 comp rx=0 tx=0).stop 2024-04-22T07:25:06.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 msgr2=0x7fe80419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe80419e5d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000c00 tx=0x7fe7f402cdf0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 shutdown_connections 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe7ec0a3d20 0x7fe7ec0a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe804198690 0x7fe804198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe804105970 0x7fe80419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 --2- 172.21.15.113:0/165918042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe804100af0 0x7fe80419e5d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 >> 172.21.15.113:0/165918042 conn(0x7fe8040760e0 msgr2=0x7fe804104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 shutdown_connections 2024-04-22T07:25:06.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.001+0000 7fe809a8c640 1 -- 172.21.15.113:0/165918042 wait complete. 2024-04-22T07:25:06.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T07:25:06.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 -- 172.21.15.113:0/3619294906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344101a50 msgr2=0x7f0344105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 --2- 172.21.15.113:0/3619294906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344101a50 0x7f0344105a90 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f0338009900 tx=0x7f033802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 -- 172.21.15.113:0/3619294906 shutdown_connections 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 --2- 172.21.15.113:0/3619294906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0344106170 0x7f034410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 --2- 172.21.15.113:0/3619294906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344101a50 0x7f0344105a90 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 --2- 172.21.15.113:0/3619294906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 0x7f0344101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 -- 172.21.15.113:0/3619294906 >> 172.21.15.113:0/3619294906 conn(0x7f0344077eb0 msgr2=0x7f03440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 -- 172.21.15.113:0/3619294906 shutdown_connections 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.111+0000 7f034b12b640 1 -- 172.21.15.113:0/3619294906 wait complete. 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 Processor -- start 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 -- start start 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 0x7f0344199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 0x7f0344199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034419a160 0x7f03441a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f03441111f0 con 0x7f0344106170 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0344111070 con 0x7f03441010a0 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f034b12b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0344111370 con 0x7f034419a160 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 0x7f0344199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 0x7f0344199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43294/0 (socket says 172.21.15.113:43294) 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f0343fff640 1 -- 172.21.15.113:0/3268749128 learned_addr learned my addr 172.21.15.113:0/3268749128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f03496c8640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034419a160 0x7f03441a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f0348ec7640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 0x7f0344199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.112+0000 7f0343fff640 1 -- 172.21.15.113:0/3268749128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034419a160 msgr2=0x7f03441a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0343fff640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034419a160 0x7f03441a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0343fff640 1 -- 172.21.15.113:0/3268749128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 msgr2=0x7f0344199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0343fff640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 0x7f0344199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0343fff640 1 -- 172.21.15.113:0/3268749128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03441a4ee0 con 0x7f0344106170 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f03496c8640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034419a160 0x7f03441a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0348ec7640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 0x7f0344199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0343fff640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 0x7f0344199bd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f033802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0341ffb640 1 -- 172.21.15.113:0/3268749128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338041070 con 0x7f0344106170 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0341ffb640 1 -- 172.21.15.113:0/3268749128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03380044d0 con 0x7f0344106170 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f0341ffb640 1 -- 172.21.15.113:0/3268749128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338003d50 con 0x7f0344106170 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03441a5170 con 0x7f0344106170 2024-04-22T07:25:06.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.113+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03441a5670 con 0x7f0344106170 2024-04-22T07:25:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.115+0000 7f0341ffb640 1 -- 172.21.15.113:0/3268749128 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0338044070 con 0x7f0344106170 2024-04-22T07:25:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.115+0000 7f034ddf8740 1 Processor -- start 2024-04-22T07:25:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 -- start start 2024-04-22T07:25:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f03f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f03fe80 0x56481f048430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 0x56481f04ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56481efa41f0 con 0x56481f048a00 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56481ef414b0 con 0x56481f03fe80 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f034ddf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56481efa01b0 con 0x56481f03f4d0 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 0x56481f04ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 0x56481f04ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43298/0 (socket says 172.21.15.113:43298) 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 -- 172.21.15.113:0/2350269433 learned_addr learned my addr 172.21.15.113:0/2350269433 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0341ffb640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f032c0a3d20 0x7f032c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f03496c8640 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f03f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0348ec7640 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f03fe80 0x56481f048430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f03496c8640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f032c0a3d20 0x7f032c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 -- 172.21.15.113:0/2350269433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 msgr2=0x56481f03f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f03f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 -- 172.21.15.113:0/2350269433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f03fe80 msgr2=0x56481f048430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f03fe80 0x56481f048430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0341ffb640 1 -- 172.21.15.113:0/3268749128 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f03380f23d0 con 0x7f0344106170 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.116+0000 7f0343fff640 1 -- 172.21.15.113:0/2350269433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481ef331b0 con 0x56481f048a00 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f0343fff640 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 0x56481f04ae50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f03380be010 tx=0x7f0338009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e942a25a89da7c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f03496c8640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f032c0a3d20 0x7f032c0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f033c013b30 tx=0x7f033c005c60 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f032affd640 1 -- 172.21.15.113:0/2350269433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033804b070 con 0x56481f048a00 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f032affd640 1 -- 172.21.15.113:0/2350269433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03380bea10 con 0x56481f048a00 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f032affd640 1 -- 172.21.15.113:0/2350269433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03380c6ef0 con 0x56481f048a00 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 -- 172.21.15.113:0/2350269433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 msgr2=0x56481f04ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 0x56481f04ae50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f03380be010 tx=0x7f0338009260 comp rx=0 tx=0).stop 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 -- 172.21.15.113:0/2350269433 shutdown_connections 2024-04-22T07:25:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f048a00 0x56481f04ae50 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f03fe80 0x56481f048430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2350269433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f03f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 -- 172.21.15.113:0/2350269433 >> 172.21.15.113:0/2350269433 conn(0x56481f039ba0 msgr2=0x56481f039f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 -- 172.21.15.113:0/2350269433 shutdown_connections 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.117+0000 7f034ddf8740 1 -- 172.21.15.113:0/2350269433 wait complete. 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 -- start start 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 Processor -- start 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f048320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f0456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f045c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56481efa41f0 con 0x56481f03fe80 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56481ef5d500 con 0x56481f048a00 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56481f04b730 con 0x56481f03f4d0 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f034ddf8740 10 client.0 _ll_register_callbacks cb 0x56481ef2bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f0456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f0456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43304/0 (socket says 172.21.15.113:43304) 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f0343fff640 1 -- 172.21.15.113:0/4080835458 learned_addr learned my addr 172.21.15.113:0/4080835458 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f03496c8640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f045c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f0348ec7640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f048320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f0343fff640 1 -- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 msgr2=0x56481f048320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.118+0000 7f0343fff640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f048320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0343fff640 1 -- 172.21.15.113:0/4080835458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 msgr2=0x56481f045c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0343fff640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f045c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0343fff640 1 -- 172.21.15.113:0/4080835458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481ecb63f0 con 0x56481f03fe80 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0348ec7640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f048320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0343fff640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f0456b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f03380ba770 tx=0x7f0338003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a267d03e147832 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0323fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0323fff640 1 -- 172.21.15.113:0/4080835458 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033804b070 con 0x56481f03fe80 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0323fff640 1 -- 172.21.15.113:0/4080835458 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f03380c6c70 con 0x56481f03fe80 2024-04-22T07:25:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03380bea10 con 0x56481f03fe80 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0323fff640 1 -- 172.21.15.113:0/4080835458 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f03380bde50 con 0x56481f03fe80 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 10 client.17877 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56481ed807d0 con 0x56481f03fe80 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0323fff640 1 -- 172.21.15.113:0/4080835458 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03380bea10 con 0x56481f03fe80 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f0323fff640 1 -- 172.21.15.113:0/4080835458 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f03380bd760 con 0x56481f03fe80 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 10 client.17877 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 10 client.17877 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 20 client.17877 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0586e0 0x56481f05aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 4 client.17877 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.119+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56481ef43930 con 0x56481f0586e0 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.120+0000 7f0348ec7640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0586e0 0x56481f05aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.120+0000 7f0323fff640 10 client.17877 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.120+0000 7f0348ec7640 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0586e0 0x56481f05aad0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e79de7c7c143d5e2 server_cookie=bcb474a9e3df85e5 in_seq=0 out_seq=0 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f0323fff640 1 -- 172.21.15.113:0/4080835458 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56481ef43930 con 0x56481f0586e0 2024-04-22T07:25:06.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f0323fff640 10 client.17877 handle_command_reply: tid=0 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 client.17877 shutdown 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 msgr2=0x56481f0456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f0456b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f03380ba770 tx=0x7f0338003820 comp rx=0 tx=0).stop 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 shutdown_connections 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0586e0 0x56481f05aad0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f045c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f0456b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f0323fff640 0 client.17877 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 --2- 172.21.15.113:0/4080835458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f048320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 >> 172.21.15.113:0/4080835458 conn(0x56481f039ba0 msgr2=0x56481f03b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 shutdown_connections 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 1 -- 172.21.15.113:0/4080835458 wait complete. 2024-04-22T07:25:06.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.121+0000 7f034ddf8740 20 client.17877 trim_cache size 0 max 16384 2024-04-22T07:25:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 Processor -- start 2024-04-22T07:25:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 -- start start 2024-04-22T07:25:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f054470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f043e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f047de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f03441b0880 con 0x56481f03fe80 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03441ae150 con 0x56481f048a00 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f034ddf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03441abfc0 con 0x56481f03f4d0 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f043e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f0343fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f043e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43310/0 (socket says 172.21.15.113:43310) 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.134+0000 7f0343fff640 1 -- 172.21.15.113:0/346207377 learned_addr learned my addr 172.21.15.113:0/346207377 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f03496c8640 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f047de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f0343fff640 1 -- 172.21.15.113:0/346207377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 msgr2=0x56481f054470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f0343fff640 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f054470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f0343fff640 1 -- 172.21.15.113:0/346207377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 msgr2=0x56481f047de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f0343fff640 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f047de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f0343fff640 1 -- 172.21.15.113:0/346207377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481f044d40 con 0x56481f03fe80 2024-04-22T07:25:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f03496c8640 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f047de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f0343fff640 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f043e90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f03380bace0 tx=0x7f0338049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7024bf9738ab61ec server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f032a7fc640 1 -- 172.21.15.113:0/346207377 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033804b070 con 0x56481f03fe80 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f032a7fc640 1 -- 172.21.15.113:0/346207377 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03380bea10 con 0x56481f03fe80 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.135+0000 7f032a7fc640 1 -- 172.21.15.113:0/346207377 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03380bdc70 con 0x56481f03fe80 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- 172.21.15.113:0/346207377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 msgr2=0x56481f043e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f043e90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f03380bace0 tx=0x7f0338049000 comp rx=0 tx=0).stop 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- 172.21.15.113:0/346207377 shutdown_connections 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f047de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03fe80 0x56481f043e90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- 172.21.15.113:0/346207377 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03f4d0 0x56481f054470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- 172.21.15.113:0/346207377 >> 172.21.15.113:0/346207377 conn(0x56481f039ba0 msgr2=0x56481f038c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- 172.21.15.113:0/346207377 shutdown_connections 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- 172.21.15.113:0/346207377 wait complete. 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- start start 2024-04-22T07:25:06.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 Processor -- start 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 0x56481f050980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03fe80 0x56481f050ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f041db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f03441ac610 con 0x56481f03f4d0 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03441ada80 con 0x56481f048a00 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03441acce0 con 0x56481f03fe80 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f034ddf8740 10 client.0 _ll_register_callbacks cb 0x7f03380be1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f0348ec7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 0x56481f050980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f0348ec7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 0x56481f050980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43316/0 (socket says 172.21.15.113:43316) 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f0348ec7640 1 -- 172.21.15.113:0/2914698064 learned_addr learned my addr 172.21.15.113:0/2914698064 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.136+0000 7f03496c8640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f041db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0343fff640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03fe80 0x56481f050ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0348ec7640 1 -- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03fe80 msgr2=0x56481f050ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0348ec7640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03fe80 0x56481f050ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0348ec7640 1 -- 172.21.15.113:0/2914698064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 msgr2=0x56481f041db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0348ec7640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f041db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0348ec7640 1 -- 172.21.15.113:0/2914698064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481f0425e0 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0348ec7640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 0x56481f050980 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x56481f05b280 tx=0x7f0334014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38469c256651105d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0323fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0323fff640 1 -- 172.21.15.113:0/2914698064 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334014ea0 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0323fff640 1 -- 172.21.15.113:0/2914698064 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f0334004220 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03380bea10 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0323fff640 1 -- 172.21.15.113:0/2914698064 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f033402d050 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 10 client.17883 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56481f0be6c0 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0323fff640 1 -- 172.21.15.113:0/2914698064 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03380bea10 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f0323fff640 1 -- 172.21.15.113:0/2914698064 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f033400f040 con 0x56481f03f4d0 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 10 client.17883 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 10 client.17883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-22T07:25:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 20 client.17883 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0bede0 0x56481f0c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.137+0000 7f034ddf8740 4 client.17883 mds_command: new command op to 25258 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 --> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56481efb1c10 con 0x56481f0bede0 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f0343fff640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0bede0 0x56481f0c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f0323fff640 10 client.17883 ms_handle_connect on v2:172.21.15.154:6838/3008705527 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f0343fff640 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0bede0 0x56481f0c11d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65c09b117908782d server_cookie=2a53efd1affc35c6 in_seq=0 out_seq=0 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f0323fff640 1 -- 172.21.15.113:0/2914698064 <== mds.2 v2:172.21.15.154:6838/3008705527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56481efb1c10 con 0x56481f0bede0 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f0323fff640 10 client.17883 handle_command_reply: tid=0 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 client.17883 shutdown 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 msgr2=0x56481f050980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 0x56481f050980 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x56481f05b280 tx=0x7f0334014840 comp rx=0 tx=0).stop 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 shutdown_connections 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:6838/3008705527,v1:172.21.15.154:6839/3008705527] conn(0x56481f0bede0 0x56481f0c11d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56481f048a00 0x56481f041db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f0323fff640 0 client.17883 ms_handle_reset on v2:172.21.15.154:6838/3008705527 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56481f03fe80 0x56481f050ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 --2- 172.21.15.113:0/2914698064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56481f03f4d0 0x56481f050980 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.138+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 >> 172.21.15.113:0/2914698064 conn(0x56481f039ba0 msgr2=0x56481f0402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 shutdown_connections 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034ddf8740 1 -- 172.21.15.113:0/2914698064 wait complete. 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034ddf8740 20 client.17883 trim_cache size 0 max 16384 2024-04-22T07:25:06.144 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:25:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f032c0a3d20 msgr2=0x7f032c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f032c0a3d20 0x7f032c0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f033c013b30 tx=0x7f033c005c60 comp rx=0 tx=0).stop 2024-04-22T07:25:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 msgr2=0x7f0344199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 0x7f0344199bd0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f0338000c00 tx=0x7f033802cdf0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 shutdown_connections 2024-04-22T07:25:06.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f032c0a3d20 0x7f032c0a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034419a160 0x7f03441a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0344106170 0x7f0344199bd0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 --2- 172.21.15.113:0/3268749128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03441010a0 0x7f0344199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 >> 172.21.15.113:0/3268749128 conn(0x7f0344077eb0 msgr2=0x7f03440ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 shutdown_connections 2024-04-22T07:25:06.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.139+0000 7f034b12b640 1 -- 172.21.15.113:0/3268749128 wait complete. 2024-04-22T07:25:06.148 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T07:25:06.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 -- 172.21.15.113:0/4158180290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f856010e830 msgr2=0x7f8560110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 --2- 172.21.15.113:0/4158180290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f856010e830 0x7f8560110c20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8558009ca0 tx=0x7f855802c180 comp rx=0 tx=0).stop 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 -- 172.21.15.113:0/4158180290 shutdown_connections 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 --2- 172.21.15.113:0/4158180290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f856010e830 0x7f8560110c20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 --2- 172.21.15.113:0/4158180290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8560074b40 0x7f856010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 --2- 172.21.15.113:0/4158180290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8560074220 0x7f8560074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 -- 172.21.15.113:0/4158180290 >> 172.21.15.113:0/4158180290 conn(0x7f85600793a0 msgr2=0x7f85600797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 -- 172.21.15.113:0/4158180290 shutdown_connections 2024-04-22T07:25:06.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.250+0000 7f856523a640 1 -- 172.21.15.113:0/4158180290 wait complete. 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 Processor -- start 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 -- start start 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 0x7f85601a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8560074b40 0x7f85601a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f856010e830 0x7f856019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8560114500 con 0x7f8560074220 2024-04-22T07:25:06.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8560114380 con 0x7f856010e830 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f856523a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8560114680 con 0x7f8560074b40 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 0x7f85601a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 0x7f85601a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43336/0 (socket says 172.21.15.113:43336) 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 -- 172.21.15.113:0/1558308896 learned_addr learned my addr 172.21.15.113:0/1558308896 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855f59e640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f856010e830 0x7f856019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855e59c640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8560074b40 0x7f85601a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 -- 172.21.15.113:0/1558308896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8560074b40 msgr2=0x7f85601a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8560074b40 0x7f85601a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 -- 172.21.15.113:0/1558308896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f856010e830 msgr2=0x7f856019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f856010e830 0x7f856019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.251+0000 7f855ed9d640 1 -- 172.21.15.113:0/1558308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856019b010 con 0x7f8560074220 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.252+0000 7f855ed9d640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 0x7f85601a06d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f854800d950 tx=0x7f854800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.252+0000 7f8557fff640 1 -- 172.21.15.113:0/1558308896 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548017070 con 0x7f8560074220 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.252+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856019b300 con 0x7f8560074220 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.252+0000 7f8557fff640 1 -- 172.21.15.113:0/1558308896 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8548009c30 con 0x7f8560074220 2024-04-22T07:25:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.252+0000 7f8557fff640 1 -- 172.21.15.113:0/1558308896 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854800f6b0 con 0x7f8560074220 2024-04-22T07:25:06.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.252+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a7520 con 0x7f8560074220 2024-04-22T07:25:06.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.254+0000 7f8557fff640 1 -- 172.21.15.113:0/1558308896 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f85480109c0 con 0x7f8560074220 2024-04-22T07:25:06.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 Processor -- start 2024-04-22T07:25:06.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 -- start start 2024-04-22T07:25:06.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5561dc7191f0 con 0x5561dc7b4e80 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5561dc6b64b0 con 0x5561dc7b44d0 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8567e21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5561dc7151b0 con 0x5561dc7bda00 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f855ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f855ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7bd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43346/0 (socket says 172.21.15.113:43346) 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f855ed9d640 1 -- 172.21.15.113:0/200297185 learned_addr learned my addr 172.21.15.113:0/200297185 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f8557fff640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f854c0a3d20 0x7f854c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f855f59e640 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f855f59e640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f854c0a3d20 0x7f854c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.255+0000 7f855e59c640 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f8557fff640 1 -- 172.21.15.113:0/1558308896 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f85480d05e0 con 0x7f8560074220 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855ed9d640 1 -- 172.21.15.113:0/200297185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 msgr2=0x5561dc7bfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855ed9d640 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855ed9d640 1 -- 172.21.15.113:0/200297185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 msgr2=0x5561dc7b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855ed9d640 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855ed9d640 1 -- 172.21.15.113:0/200297185 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561dc6a81b0 con 0x5561dc7b4e80 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855ed9d640 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7bd430 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8548098840 tx=0x7f8548005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130951614556bfdd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f855f59e640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f854c0a3d20 0x7f854c0a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8558005cc0 tx=0x7f8558005c50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f85557fa640 1 -- 172.21.15.113:0/200297185 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548020070 con 0x5561dc7b4e80 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f85557fa640 1 -- 172.21.15.113:0/200297185 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f85480d7040 con 0x5561dc7b4e80 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.256+0000 7f85557fa640 1 -- 172.21.15.113:0/200297185 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548098ad0 con 0x5561dc7b4e80 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- 172.21.15.113:0/200297185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 msgr2=0x5561dc7bd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7bd430 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8548098840 tx=0x7f8548005880 comp rx=0 tx=0).stop 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- 172.21.15.113:0/200297185 shutdown_connections 2024-04-22T07:25:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7bd430 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- 172.21.15.113:0/200297185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- 172.21.15.113:0/200297185 >> 172.21.15.113:0/200297185 conn(0x5561dc7aeba0 msgr2=0x5561dc7aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- 172.21.15.113:0/200297185 shutdown_connections 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- 172.21.15.113:0/200297185 wait complete. 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- start start 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 Processor -- start 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5561dc7c0970 con 0x5561dc7b4e80 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5561dc6d2500 con 0x5561dc7b44d0 2024-04-22T07:25:06.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5561dc7c0af0 con 0x5561dc7bda00 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f8567e21740 10 client.0 _ll_register_callbacks cb 0x5561dc6a0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f855e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f855e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7ba5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43352/0 (socket says 172.21.15.113:43352) 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.257+0000 7f855e59c640 1 -- 172.21.15.113:0/1015869823 learned_addr learned my addr 172.21.15.113:0/1015869823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855f59e640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855ed9d640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855e59c640 1 -- 172.21.15.113:0/1015869823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 msgr2=0x5561dc7bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855e59c640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855e59c640 1 -- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 msgr2=0x5561dc7bd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855e59c640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7bd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855e59c640 1 -- 172.21.15.113:0/1015869823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561dc42b3f0 con 0x5561dc7b4e80 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f855e59c640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7ba5f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f85500103a0 tx=0x7f85500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef280d44a08cbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f85467fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f85467fc640 1 -- 172.21.15.113:0/1015869823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550004230 con 0x5561dc7b4e80 2024-04-22T07:25:06.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f85467fc640 1 -- 172.21.15.113:0/1015869823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f85500043d0 con 0x5561dc7b4e80 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85480d7040 con 0x5561dc7b4e80 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f85467fc640 1 -- 172.21.15.113:0/1015869823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8550012660 con 0x5561dc7b4e80 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f8567e21740 10 client.17895 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5561dc4f57d0 con 0x5561dc7b4e80 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f85467fc640 1 -- 172.21.15.113:0/1015869823 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f85480d7040 con 0x5561dc7b4e80 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.258+0000 7f85467fc640 1 -- 172.21.15.113:0/1015869823 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f855000f040 con 0x5561dc7b4e80 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f8567e21740 10 client.17895 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f8567e21740 10 client.17895 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f8567e21740 20 client.17895 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f8567e21740 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc7cdc50 0x5561dc7d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f8567e21740 4 client.17895 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561dc6b8930 con 0x5561dc7cdc50 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f855ed9d640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc7cdc50 0x5561dc7d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f855ed9d640 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc7cdc50 0x5561dc7d0090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcb1840c61e55fdb server_cookie=69cfdd83f4730df in_seq=0 out_seq=0 2024-04-22T07:25:06.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.259+0000 7f85467fc640 10 client.17895 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f85467fc640 1 -- 172.21.15.113:0/1015869823 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561dc6b8930 con 0x5561dc7cdc50 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f85467fc640 10 client.17895 handle_command_reply: tid=0 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 client.17895 shutdown 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 msgr2=0x5561dc7ba5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7ba5f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f85500103a0 tx=0x7f85500107a0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 shutdown_connections 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc7cdc50 0x5561dc7d0090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f85467fc640 0 client.17895 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7ba5f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 --2- 172.21.15.113:0/1015869823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7bd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 >> 172.21.15.113:0/1015869823 conn(0x5561dc7aeba0 msgr2=0x5561dc7b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 shutdown_connections 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 1 -- 172.21.15.113:0/1015869823 wait complete. 2024-04-22T07:25:06.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.260+0000 7f8567e21740 20 client.17895 trim_cache size 0 max 16384 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.273+0000 7f8567e21740 1 Processor -- start 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.273+0000 7f8567e21740 1 -- start start 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8567e21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7b44d0 0x5561dc7c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8567e21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b4e80 0x5561dc7b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8567e21740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 0x5561dc7bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8567e21740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85601b2b40 con 0x5561dc7bda00 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8567e21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85601b0410 con 0x5561dc7b4e80 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8567e21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f85601ae280 con 0x5561dc7b44d0 2024-04-22T07:25:06.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 0x5561dc7bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 0x5561dc7bcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43356/0 (socket says 172.21.15.113:43356) 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 -- 172.21.15.113:0/3675113322 learned_addr learned my addr 172.21.15.113:0/3675113322 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855ed9d640 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7b44d0 0x5561dc7c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855e59c640 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b4e80 0x5561dc7b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 -- 172.21.15.113:0/3675113322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7b44d0 msgr2=0x5561dc7c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7b44d0 0x5561dc7c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 -- 172.21.15.113:0/3675113322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b4e80 msgr2=0x5561dc7b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b4e80 0x5561dc7b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 -- 172.21.15.113:0/3675113322 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561dc7afdd0 con 0x5561dc7bda00 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f855f59e640 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 0x5561dc7bcd20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8558009ca0 tx=0x7f855802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6568b525e51f383 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8554ff9640 1 -- 172.21.15.113:0/3675113322 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558039040 con 0x5561dc7bda00 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8554ff9640 1 -- 172.21.15.113:0/3675113322 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8558055070 con 0x5561dc7bda00 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.274+0000 7f8554ff9640 1 -- 172.21.15.113:0/3675113322 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580507c0 con 0x5561dc7bda00 2024-04-22T07:25:06.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 -- 172.21.15.113:0/3675113322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 msgr2=0x5561dc7bcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 0x5561dc7bcd20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8558009ca0 tx=0x7f855802e040 comp rx=0 tx=0).stop 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 -- 172.21.15.113:0/3675113322 shutdown_connections 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7bda00 0x5561dc7bcd20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b4e80 0x5561dc7b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 --2- 172.21.15.113:0/3675113322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7b44d0 0x5561dc7c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 -- 172.21.15.113:0/3675113322 >> 172.21.15.113:0/3675113322 conn(0x5561dc7aeba0 msgr2=0x5561dc7afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 -- 172.21.15.113:0/3675113322 shutdown_connections 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 -- 172.21.15.113:0/3675113322 wait complete. 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 -- start start 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.275+0000 7f8567e21740 1 Processor -- start 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f85601ae8d0 con 0x5561dc7b4e80 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85601afd40 con 0x5561dc7b44d0 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f85601aefa0 con 0x5561dc7bda00 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 10 client.0 _ll_register_callbacks cb 0x7f855000ebc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7b7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43368/0 (socket says 172.21.15.113:43368) 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 -- 172.21.15.113:0/2709819969 learned_addr learned my addr 172.21.15.113:0/2709819969 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855f59e640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855ed9d640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 -- 172.21.15.113:0/2709819969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 msgr2=0x5561dc7b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 -- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 msgr2=0x5561dc7b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 -- 172.21.15.113:0/2709819969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561dc833300 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855f59e640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855ed9d640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f855e59c640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7b7990 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x5561dc7b9e50 tx=0x7f855000e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0f4c0c5058bc53 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f85467fc640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f85467fc640 1 -- 172.21.15.113:0/2709819969 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855002f060 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f85467fc640 1 -- 172.21.15.113:0/2709819969 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8943+0+0 (secure 0 0 0) 0x7f85500052c0 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8558055070 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f85467fc640 1 -- 172.21.15.113:0/2709819969 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f85500054c0 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 10 client.17901 fetch_fsmap learned FSMap version 52 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.276+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5561dc834670 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f85467fc640 1 -- 172.21.15.113:0/2709819969 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8558055070 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f85467fc640 1 -- 172.21.15.113:0/2709819969 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 52) ==== 8338+0+0 (secure 0 0 0) 0x7f855002a5f0 con 0x5561dc7b4e80 2024-04-22T07:25:06.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f8567e21740 10 client.17901 fetch_fsmap finished waiting for FSMap version 52 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f8567e21740 10 client.17901 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f8567e21740 20 client.17901 populate_metadata read hostname 'smithi113' 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f8567e21740 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc834df0 0x5561dc837210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f8567e21740 4 client.17901 mds_command: new command op to 25367 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 --> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561dc726c10 con 0x5561dc834df0 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f855ed9d640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc834df0 0x5561dc837210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f85467fc640 10 client.17901 ms_handle_connect on v2:172.21.15.133:6836/3335686802 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.277+0000 7f855ed9d640 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc834df0 0x5561dc837210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24d9299e47128ac4 server_cookie=635e47f5f81937 in_seq=0 out_seq=0 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f85467fc640 1 -- 172.21.15.113:0/2709819969 <== mds.1 v2:172.21.15.133:6836/3335686802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561dc726c10 con 0x5561dc834df0 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f85467fc640 10 client.17901 handle_command_reply: tid=0 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 client.17901 shutdown 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 msgr2=0x5561dc7b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7b7990 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x5561dc7b9e50 tx=0x7f855000e410 comp rx=0 tx=0).stop 2024-04-22T07:25:06.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 shutdown_connections 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:6836/3335686802,v1:172.21.15.133:6837/3335686802] conn(0x5561dc834df0 0x5561dc837210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5561dc7bda00 0x5561dc7b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5561dc7b4e80 0x5561dc7b7990 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f85467fc640 0 client.17901 ms_handle_reset on v2:172.21.15.133:6836/3335686802 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 --2- 172.21.15.113:0/2709819969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561dc7b44d0 0x5561dc7b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 >> 172.21.15.113:0/2709819969 conn(0x5561dc7aeba0 msgr2=0x5561dc7adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 shutdown_connections 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 1 -- 172.21.15.113:0/2709819969 wait complete. 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f8567e21740 20 client.17901 trim_cache size 0 max 16384 2024-04-22T07:25:06.284 INFO:teuthology.orchestra.run.smithi113.stdout:[] 2024-04-22T07:25:06.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f854c0a3d20 msgr2=0x7f854c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f854c0a3d20 0x7f854c0a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8558005cc0 tx=0x7f8558005c50 comp rx=0 tx=0).stop 2024-04-22T07:25:06.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 msgr2=0x7f85601a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 0x7f85601a06d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f854800d950 tx=0x7f854800de10 comp rx=0 tx=0).stop 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 shutdown_connections 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f854c0a3d20 0x7f854c0a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f856010e830 0x7f856019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8560074b40 0x7f85601a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 --2- 172.21.15.113:0/1558308896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8560074220 0x7f85601a06d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 >> 172.21.15.113:0/1558308896 conn(0x7f85600793a0 msgr2=0x7f85600fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 shutdown_connections 2024-04-22T07:25:06.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.278+0000 7f856523a640 1 -- 172.21.15.113:0/1558308896 wait complete. 2024-04-22T07:25:06.287 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-22T07:25:06.288 INFO:tasks.fwd_scrub:done joining 2024-04-22T07:25:06.288 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-22T07:25:06.300 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/3519748600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88106150 msgr2=0x7f8b8810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/3519748600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88106150 0x7f8b8810dd20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009ca0 tx=0x7f8b8002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/3519748600 shutdown_connections 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/3519748600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88106150 0x7f8b8810dd20 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/3519748600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b881021f0 0x7f8b88105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/3519748600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88101840 0x7f8b88101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/3519748600 >> 172.21.15.113:0/3519748600 conn(0x7f8b88077450 msgr2=0x7f8b880fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/3519748600 shutdown_connections 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.405+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/3519748600 wait complete. 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 Processor -- start 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 -- start start 2024-04-22T07:25:06.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 0x7f8b8819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88106150 0x7f8b8819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b881985d0 0x7f8b88198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881122f0 con 0x7f8b88101840 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88112170 con 0x7f8b88106150 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b8e8a8640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88112470 con 0x7f8b881985d0 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.406+0000 7f8b87fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 0x7f8b8819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 0x7f8b8819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43388/0 (socket says 172.21.15.113:43388) 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 -- 172.21.15.113:0/1367643402 learned_addr learned my addr 172.21.15.113:0/1367643402 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 -- 172.21.15.113:0/1367643402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b881985d0 msgr2=0x7f8b88198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b881985d0 0x7f8b88198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 -- 172.21.15.113:0/1367643402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88106150 msgr2=0x7f8b8819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b877fe640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88106150 0x7f8b8819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88106150 0x7f8b8819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 -- 172.21.15.113:0/1367643402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b881990f0 con 0x7f8b88101840 2024-04-22T07:25:06.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b877fe640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88106150 0x7f8b8819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b87fff640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 0x7f8b8819e420 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800d950 tx=0x7f8b7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78017030 con 0x7f8b88101840 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8b78004020 con 0x7f8b88101840 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b881993e0 con 0x7f8b88101840 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b780047a0 con 0x7f8b88101840 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.407+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b881a5630 con 0x7f8b88101840 2024-04-22T07:25:06.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.408+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b88199b00 con 0x7f8b88101840 2024-04-22T07:25:06.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.409+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f8b780041d0 con 0x7f8b88101840 2024-04-22T07:25:06.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.410+0000 7f8b857fa640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b700a3d20 0x7f8b700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.410+0000 7f8b877fe640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b700a3d20 0x7f8b700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.410+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8987+0+0 (secure 0 0 0) 0x7f8b780d0480 con 0x7f8b88101840 2024-04-22T07:25:06.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.410+0000 7f8b877fe640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b700a3d20 0x7f8b700a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b880ffa10 tx=0x7f8b7c005cb0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.415+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b7809c950 con 0x7f8b88101840 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: Unable to set osd_memory_target on smithi154 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: Unable to set osd_memory_target on smithi133 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: Unable to set osd_memory_target on smithi113 to 184773120: error parsing value: Value '184773120' is below minimum 939524096 2024-04-22T07:25:06.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:06 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1537137645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:06.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.599+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b8802d7a0 con 0x7f8b88101840 2024-04-22T07:25:06.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.601+0000 7f8b857fa640 1 -- 172.21.15.113:0/1367643402 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+23894 (secure 0 0 0) 0x7f8b7809c250 con 0x7f8b88101840 2024-04-22T07:25:06.599 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:06.600 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":102,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:25:06.326282+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1615298521":"2024-04-23T07:17:49.535084+0000","172.21.15.154:6836/1615298521":"2024-04-23T07:17:49.535084+0000","172.21.15.154:6834/217490889":"2024-04-23T07:17:49.535032+0000","172.21.15.133:6838/982201245":"2024-04-23T07:17:48.307009+0000","172.21.15.154:6839/3404626603":"2024-04-23T07:15:51.118540+0000","172.21.15.154:6838/3404626603":"2024-04-23T07:15:51.118540+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.154:6835/217490889":"2024-04-23T07:17:49.535032+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.133:6839/982201245":"2024-04-23T07:17:48.307009+0000","172.21.15.133:6837/65907218":"2024-04-23T07:15:51.098366+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.133:6836/65907218":"2024-04-23T07:15:51.098366+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b700a3d20 msgr2=0x7f8b700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b700a3d20 0x7f8b700a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b880ffa10 tx=0x7f8b7c005cb0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 msgr2=0x7f8b8819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 0x7f8b8819e420 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800d950 tx=0x7f8b7800de10 comp rx=0 tx=0).stop 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 shutdown_connections 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f8b700a3d20 0x7f8b700a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b881985d0 0x7f8b88198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b88106150 0x7f8b8819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 --2- 172.21.15.113:0/1367643402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b88101840 0x7f8b8819e420 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 >> 172.21.15.113:0/1367643402 conn(0x7f8b88077450 msgr2=0x7f8b881042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 shutdown_connections 2024-04-22T07:25:06.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.604+0000 7f8b8e8a8640 1 -- 172.21.15.113:0/1367643402 wait complete. 2024-04-22T07:25:06.612 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:25:06.613 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:25:06.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 -- 172.21.15.113:0/2862395153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48102180 msgr2=0x7f0a48102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/2862395153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48102180 0x7f0a48102560 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009900 tx=0x7f0a3802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:06.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 -- 172.21.15.113:0/2862395153 shutdown_connections 2024-04-22T07:25:06.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/2862395153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 0x7f0a4810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/2862395153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a48102b30 0x7f0a48106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/2862395153 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48102180 0x7f0a48102560 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 -- 172.21.15.113:0/2862395153 >> 172.21.15.113:0/2862395153 conn(0x7f0a480fda00 msgr2=0x7f0a480ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 -- 172.21.15.113:0/2862395153 shutdown_connections 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.715+0000 7f0a4e05c640 1 -- 172.21.15.113:0/2862395153 wait complete. 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 Processor -- start 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 -- start start 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a48102180 0x7f0a4819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 0x7f0a4819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 0x7f0a48198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481122d0 con 0x7f0a48198750 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112150 con 0x7f0a48102180 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a4e05c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112450 con 0x7f0a48107250 2024-04-22T07:25:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 0x7f0a48198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 0x7f0a48198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43418/0 (socket says 172.21.15.113:43418) 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.716+0000 7f0a47fff640 1 -- 172.21.15.113:0/1738699714 learned_addr learned my addr 172.21.15.113:0/1738699714 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a477fe640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a48102180 0x7f0a4819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a46ffd640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 0x7f0a4819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a47fff640 1 -- 172.21.15.113:0/1738699714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 msgr2=0x7f0a4819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a47fff640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 0x7f0a4819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a47fff640 1 -- 172.21.15.113:0/1738699714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a48102180 msgr2=0x7f0a4819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a47fff640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a48102180 0x7f0a4819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a47fff640 1 -- 172.21.15.113:0/1738699714 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a48199350 con 0x7f0a48198750 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a46ffd640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 0x7f0a4819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a47fff640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 0x7f0a48198bd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009fd0 tx=0x7f0a4000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40017070 con 0x7f0a48198750 2024-04-22T07:25:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0a40003e40 con 0x7f0a48198750 2024-04-22T07:25:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a481a5430 con 0x7f0a48198750 2024-04-22T07:25:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004820 con 0x7f0a48198750 2024-04-22T07:25:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.717+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a481a57e0 con 0x7f0a48198750 2024-04-22T07:25:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.718+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a48103650 con 0x7f0a48198750 2024-04-22T07:25:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.719+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0a40003970 con 0x7f0a48198750 2024-04-22T07:25:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.720+0000 7f0a44ff9640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0a300a3d20 0x7f0a300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.720+0000 7f0a477fe640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0a300a3d20 0x7f0a300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.720+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8987+0+0 (secure 0 0 0) 0x7f0a400d0730 con 0x7f0a48198750 2024-04-22T07:25:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.720+0000 7f0a477fe640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0a300a3d20 0x7f0a300a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009230 tx=0x7f0a38005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:06.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.725+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a40003c60 con 0x7f0a48198750 2024-04-22T07:25:06.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-22T07:25:06.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-22T07:25:06.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: Unable to set osd_memory_target on smithi154 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-22T07:25:06.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: Unable to set osd_memory_target on smithi133 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-22T07:25:06.817 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-04-22T07:25:06.817 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: Unable to set osd_memory_target on smithi113 to 184773120: error parsing value: Value '184773120' is below minimum 939524096 2024-04-22T07:25:06.817 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:06 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1537137645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi154 to 176.2M 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: Unable to set osd_memory_target on smithi154 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: Unable to set osd_memory_target on smithi133 to 184768819: error parsing value: Value '184768819' is below minimum 939524096 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: Unable to set osd_memory_target on smithi113 to 184773120: error parsing value: Value '184773120' is below minimum 939524096 2024-04-22T07:25:06.818 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:06 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1537137645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:06.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.909+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a48068d70 con 0x7f0a48198750 2024-04-22T07:25:06.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.909+0000 7f0a44ff9640 1 -- 172.21.15.113:0/1738699714 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f0a4009cc00 con 0x7f0a48198750 2024-04-22T07:25:06.907 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0a300a3d20 msgr2=0x7f0a300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0a300a3d20 0x7f0a300a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009230 tx=0x7f0a38005fd0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 msgr2=0x7f0a48198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 0x7f0a48198bd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009fd0 tx=0x7f0a4000b260 comp rx=0 tx=0).stop 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 shutdown_connections 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f0a300a3d20 0x7f0a300a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a48198750 0x7f0a48198bd0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a48107250 0x7f0a4819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 --2- 172.21.15.113:0/1738699714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a48102180 0x7f0a4819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:06.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 >> 172.21.15.113:0/1738699714 conn(0x7f0a480fda00 msgr2=0x7f0a480feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:06.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 shutdown_connections 2024-04-22T07:25:06.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:06.911+0000 7f0a4e05c640 1 -- 172.21.15.113:0/1738699714 wait complete. 2024-04-22T07:25:06.920 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 -- 172.21.15.113:0/3899171804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c101a50 msgr2=0x7faa9c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 --2- 172.21.15.113:0/3899171804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c101a50 0x7faa9c105a90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 -- 172.21.15.113:0/3899171804 shutdown_connections 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 --2- 172.21.15.113:0/3899171804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 0x7faa9c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 --2- 172.21.15.113:0/3899171804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c101a50 0x7faa9c105a90 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 --2- 172.21.15.113:0/3899171804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa9c1010a0 0x7faa9c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 -- 172.21.15.113:0/3899171804 >> 172.21.15.113:0/3899171804 conn(0x7faa9c0771d0 msgr2=0x7faa9c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:07.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 -- 172.21.15.113:0/3899171804 shutdown_connections 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.024+0000 7faaa3596640 1 -- 172.21.15.113:0/3899171804 wait complete. 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 Processor -- start 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 -- start start 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa9c1010a0 0x7faa9c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 0x7faa9c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 0x7faa9c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c111250 con 0x7faa9c1975b0 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c1110d0 con 0x7faa9c106170 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c1113d0 con 0x7faa9c1010a0 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 0x7faa9c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 0x7faa9c197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43434/0 (socket says 172.21.15.113:43434) 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 -- 172.21.15.113:0/4188379636 learned_addr learned my addr 172.21.15.113:0/4188379636 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa0b31640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 0x7faa9c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1332640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa9c1010a0 0x7faa9c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 -- 172.21.15.113:0/4188379636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa9c1010a0 msgr2=0x7faa9c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa9c1010a0 0x7faa9c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 -- 172.21.15.113:0/4188379636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 msgr2=0x7faa9c19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 0x7faa9c19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 -- 172.21.15.113:0/4188379636 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c1980d0 con 0x7faa9c1975b0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa0b31640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 0x7faa9c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa1b33640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 0x7faa9c197a60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7faa9400eaa0 tx=0x7faa9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9400cdb0 con 0x7faa9c1975b0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c1983c0 con 0x7faa9c1975b0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.025+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faa94005020 con 0x7faa9c1975b0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.026+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa940116d0 con 0x7faa9c1975b0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.026+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c1a45c0 con 0x7faa9c1975b0 2024-04-22T07:25:07.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.026+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c0691c0 con 0x7faa9c1975b0 2024-04-22T07:25:07.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.027+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7faa940051d0 con 0x7faa9c1975b0 2024-04-22T07:25:07.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.028+0000 7faa927fc640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faa880a3d20 0x7faa880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.028+0000 7faaa1332640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faa880a3d20 0x7faa880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.028+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8987+0+0 (secure 0 0 0) 0x7faa940d0cd0 con 0x7faa9c1975b0 2024-04-22T07:25:07.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.028+0000 7faaa1332640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faa880a3d20 0x7faa880a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa98009890 tx=0x7faa98008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:07.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.032+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa94016030 con 0x7faa9c1975b0 2024-04-22T07:25:07.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.210+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa9c1093a0 con 0x7faa9c1975b0 2024-04-22T07:25:07.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.210+0000 7faa927fc640 1 -- 172.21.15.113:0/4188379636 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7faa9409d1a0 con 0x7faa9c1975b0 2024-04-22T07:25:07.208 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:25:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faa880a3d20 msgr2=0x7faa880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faa880a3d20 0x7faa880a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa98009890 tx=0x7faa98008040 comp rx=0 tx=0).stop 2024-04-22T07:25:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 msgr2=0x7faa9c197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 0x7faa9c197a60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7faa9400eaa0 tx=0x7faa9400ef60 comp rx=0 tx=0).stop 2024-04-22T07:25:07.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 shutdown_connections 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7faa880a3d20 0x7faa880a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa9c1975b0 0x7faa9c197a60 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa9c106170 0x7faa9c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 --2- 172.21.15.113:0/4188379636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa9c1010a0 0x7faa9c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 >> 172.21.15.113:0/4188379636 conn(0x7faa9c0771d0 msgr2=0x7faa9c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.212+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 shutdown_connections 2024-04-22T07:25:07.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.213+0000 7faaa3596640 1 -- 172.21.15.113:0/4188379636 wait complete. 2024-04-22T07:25:07.221 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:25:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- 172.21.15.113:0/1194535345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c101090 msgr2=0x7f142c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- 172.21.15.113:0/1194535345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c101090 0x7f142c101470 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1428009ca0 tx=0x7f142802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- 172.21.15.113:0/1194535345 shutdown_connections 2024-04-22T07:25:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- 172.21.15.113:0/1194535345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 0x7f142c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- 172.21.15.113:0/1194535345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101a40 0x7f142c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- 172.21.15.113:0/1194535345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c101090 0x7f142c101470 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- 172.21.15.113:0/1194535345 >> 172.21.15.113:0/1194535345 conn(0x7f142c077f50 msgr2=0x7f142c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- 172.21.15.113:0/1194535345 shutdown_connections 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- 172.21.15.113:0/1194535345 wait complete. 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 Processor -- start 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- start start 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101090 0x7f142c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 0x7f142c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 0x7f142c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f142c111200 con 0x7f142c19a190 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f142c111080 con 0x7f142c106160 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.325+0000 7f1434677640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f142c111380 con 0x7f142c101090 2024-04-22T07:25:07.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 0x7f142c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 0x7f142c1a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43448/0 (socket says 172.21.15.113:43448) 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 -- 172.21.15.113:0/1181629347 learned_addr learned my addr 172.21.15.113:0/1181629347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432413640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101090 0x7f142c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 -- 172.21.15.113:0/1181629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101090 msgr2=0x7f142c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101090 0x7f142c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 -- 172.21.15.113:0/1181629347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 msgr2=0x7f142c199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1431c12640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 0x7f142c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 0x7f142c199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 -- 172.21.15.113:0/1181629347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c1a4f10 con 0x7f142c19a190 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432413640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101090 0x7f142c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1431c12640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 0x7f142c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1432c14640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 0x7f142c1a4740 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f142400d750 tx=0x7f142400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424017070 con 0x7f142c19a190 2024-04-22T07:25:07.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f14240054f0 con 0x7f142c19a190 2024-04-22T07:25:07.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c1a5200 con 0x7f142c19a190 2024-04-22T07:25:07.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424004cd0 con 0x7f142c19a190 2024-04-22T07:25:07.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.326+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c1a5700 con 0x7f142c19a190 2024-04-22T07:25:07.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.327+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c068d70 con 0x7f142c19a190 2024-04-22T07:25:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.328+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f1424009bd0 con 0x7f142c19a190 2024-04-22T07:25:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.329+0000 7f14237fe640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f14180a3d20 0x7f14180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.329+0000 7f1432413640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f14180a3d20 0x7f14180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.329+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8987+0+0 (secure 0 0 0) 0x7f14240d04e0 con 0x7f142c19a190 2024-04-22T07:25:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.329+0000 7f1432413640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f14180a3d20 0x7f14180a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1428032540 tx=0x7f1428005d80 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.333+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f142409c9b0 con 0x7f142c19a190 2024-04-22T07:25:07.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.512+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f142c101a40 con 0x7f142c19a190 2024-04-22T07:25:07.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.512+0000 7f14237fe640 1 -- 172.21.15.113:0/1181629347 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f1424096e20 con 0x7f142c19a190 2024-04-22T07:25:07.510 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f14180a3d20 msgr2=0x7f14180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f14180a3d20 0x7f14180a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1428032540 tx=0x7f1428005d80 comp rx=0 tx=0).stop 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 msgr2=0x7f142c1a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 0x7f142c1a4740 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f142400d750 tx=0x7f142400dc10 comp rx=0 tx=0).stop 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 shutdown_connections 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f14180a3d20 0x7f14180a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f142c19a190 0x7f142c1a4740 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c106160 0x7f142c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 --2- 172.21.15.113:0/1181629347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f142c101090 0x7f142c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 >> 172.21.15.113:0/1181629347 conn(0x7f142c077f50 msgr2=0x7f142c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 shutdown_connections 2024-04-22T07:25:07.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.514+0000 7f1434677640 1 -- 172.21.15.113:0/1181629347 wait complete. 2024-04-22T07:25:07.521 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:07 smithi113 ceph-mon[27933]: pgmap v323: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-22T07:25:07.521 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:07 smithi113 ceph-mon[27933]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T07:25:07.521 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:07 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1367643402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:07.521 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:07 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1738699714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:25:07.521 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:07 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4188379636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:25:07.522 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-22T07:25:07.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 -- 172.21.15.113:0/3658352867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 msgr2=0x7f7f6c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 --2- 172.21.15.113:0/3658352867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c101470 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009900 tx=0x7f7f5c02bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:07.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 -- 172.21.15.113:0/3658352867 shutdown_connections 2024-04-22T07:25:07.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 --2- 172.21.15.113:0/3658352867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c106160 0x7f7f6c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 --2- 172.21.15.113:0/3658352867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6c101a40 0x7f7f6c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 --2- 172.21.15.113:0/3658352867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c101470 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 -- 172.21.15.113:0/3658352867 >> 172.21.15.113:0/3658352867 conn(0x7f7f6c0771d0 msgr2=0x7f7f6c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 -- 172.21.15.113:0/3658352867 shutdown_connections 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.622+0000 7f7f7190e640 1 -- 172.21.15.113:0/3658352867 wait complete. 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 Processor -- start 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 -- start start 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6c106160 0x7f7f6c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c19a160 0x7f7f6c1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c111260 con 0x7f7f6c101090 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1110e0 con 0x7f7f6c19a160 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f7190e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1113e0 con 0x7f7f6c106160 2024-04-22T07:25:07.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43472/0 (socket says 172.21.15.113:43472) 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 -- 172.21.15.113:0/1038139317 learned_addr learned my addr 172.21.15.113:0/1038139317 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6a7fc640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6c106160 0x7f7f6c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6b7fe640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c19a160 0x7f7f6c1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 -- 172.21.15.113:0/1038139317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6c106160 msgr2=0x7f7f6c199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6c106160 0x7f7f6c199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 -- 172.21.15.113:0/1038139317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c19a160 msgr2=0x7f7f6c1a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.623+0000 7f7f6affd640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c19a160 0x7f7f6c1a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f6affd640 1 -- 172.21.15.113:0/1038139317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1a4f00 con 0x7f7f6c101090 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f6b7fe640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c19a160 0x7f7f6c1a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f6affd640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c1996e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005940 tx=0x7f7f5c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c041070 con 0x7f7f6c101090 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7f5c0052f0 con 0x7f7f6c101090 2024-04-22T07:25:07.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1a5190 con 0x7f7f6c101090 2024-04-22T07:25:07.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c034440 con 0x7f7f6c101090 2024-04-22T07:25:07.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.624+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1a56e0 con 0x7f7f6c101090 2024-04-22T07:25:07.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.625+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c102560 con 0x7f7f6c101090 2024-04-22T07:25:07.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.626+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f7f5c044070 con 0x7f7f6c101090 2024-04-22T07:25:07.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.626+0000 7f7f7090c640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7f540a3d50 0x7f7f540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:07.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.627+0000 7f7f6a7fc640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7f540a3d50 0x7f7f540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:07.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.627+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8987+0+0 (secure 0 0 0) 0x7f7f5c0f27d0 con 0x7f7f6c101090 2024-04-22T07:25:07.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.627+0000 7f7f6a7fc640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7f540a3d50 0x7f7f540a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c06b0d0 tx=0x7f7f60008040 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:07.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.630+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f5c0bedb0 con 0x7f7f6c101090 2024-04-22T07:25:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:07 smithi133 ceph-mon[31198]: pgmap v323: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-22T07:25:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:07 smithi133 ceph-mon[31198]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T07:25:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:07 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1367643402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:07 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1738699714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:25:07.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:07 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4188379636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:25:07.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:07 smithi154 ceph-mon[31862]: pgmap v323: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-04-22T07:25:07.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:07 smithi154 ceph-mon[31862]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T07:25:07.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:07 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1367643402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:07.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:07 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1738699714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:25:07.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:07 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4188379636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:25:07.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.855+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7f6c109390 con 0x7f7f6c101090 2024-04-22T07:25:07.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.863+0000 7f7f7090c640 1 -- 172.21.15.113:0/1038139317 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746071 (secure 0 0 0) 0x7f7f5c0be6b0 con 0x7f7f6c101090 2024-04-22T07:25:07.867 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:07.867 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "epoch": 26, 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "active_gid": 15279, 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "active_name": "x", 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "active_addrs": { 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:6800", 2024-04-22T07:25:07.868 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2258977739 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:6801", 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2258977739 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "active_addr": "172.21.15.113:6801/2258977739", 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "active_change": "2024-04-22T07:14:49.346954+0000", 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "active_mgr_features": 4540701547738038271, 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "available": true, 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: "standbys": [ 2024-04-22T07:25:07.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "gid": 24859, 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "z", 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr_features": 4540701547738038271, 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "available_modules": [ 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "alerts", 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.870 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.871 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.872 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.873 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:07.875 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_destination": { 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_destination", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.876 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Email address to send alerts to", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_from_name": { 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_from_name", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.877 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "Ceph", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Email From: name", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_host": { 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_host", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.878 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP server", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_password": { 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_password", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.879 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Password to authenticate with", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_port": { 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_port", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "465", 2024-04-22T07:25:07.880 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP port", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_sender": { 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_sender", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.881 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP envelope sender", 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_ssl": { 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_ssl", 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.882 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_user": { 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_user", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.883 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "User to authenticate as", 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "balancer", 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:07.884 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "active": { 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "active", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.885 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "begin_time": { 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "begin_time", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0000", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.886 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "begin_weekday": { 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "begin_weekday", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "6", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.887 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_max_iterations": { 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_max_iterations", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "25", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "250", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.888 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_metrics": { 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_metrics", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_step": { 2024-04-22T07:25:07.889 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_step", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0.5", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0.001", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "0.999", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "aggressiveness of optimization", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "end_time": { 2024-04-22T07:25:07.890 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "end_time", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "2359", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "end_weekday": { 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "end_weekday", 2024-04-22T07:25:07.891 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "6", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.892 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.893 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.894 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.895 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.896 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.897 INFO:teuthology.orchestra.run.smithi113.stdout: "min_score": { 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "min_score", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.898 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": { 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mode", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "upmap", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "crush-compat", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "none", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "read", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap", 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap-read" 2024-04-22T07:25:07.899 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Balancer mode", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "pool_ids": { 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pool_ids", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.900 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.901 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap_max_deviation": { 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "upmap_max_deviation", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.902 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap_max_optimizations": { 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "upmap_max_optimizations", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T07:25:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cephadm", 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_down_multiplier": { 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_down_multiplier", 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:07.904 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3.0", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_refresh_rate": { 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_refresh_rate", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.905 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "20", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_starting_port": { 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_starting_port", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.906 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "4721", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_ptrace": { 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_ptrace", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.907 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "autotune_interval": { 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "autotune_interval", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:07.908 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "autotune_memory_target_ratio": { 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0.7", 2024-04-22T07:25:07.909 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm_log_destination": { 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cephadm_log_destination", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.910 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "file", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "file,syslog", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "syslog" 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "cgroups_split": { 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cgroups_split", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.911 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "config_checks_enabled": { 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_checks_enabled", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.912 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "config_dashboard": { 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_dashboard", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.913 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_alertmanager": { 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_alertmanager", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.914 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_base": { 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_base", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.915 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Container image name, without the tag", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_elasticsearch": { 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_elasticsearch", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "elasticsearch container image", 2024-04-22T07:25:07.916 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_grafana": { 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_grafana", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_haproxy": { 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_haproxy", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "HAproxy container image", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_agent": { 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_agent", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger agent container image", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.919 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_collector": { 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_collector", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger collector container image", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_query": { 2024-04-22T07:25:07.920 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_query", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger query container image", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_keepalived": { 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_keepalived", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.921 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Keepalived container image", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_loki": { 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_loki", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.922 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Loki container image", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_node_exporter": { 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_node_exporter", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.923 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_nvmeof": { 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_nvmeof", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.924 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Nvme-of container image", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_prometheus": { 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_prometheus", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.925 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_promtail": { 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_promtail", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.926 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Promtail container image", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_samba": { 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_samba", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.927 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Samba/SMB container image", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_snmp_gateway": { 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_snmp_gateway", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.928 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SNMP Gateway container image", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "container_init": { 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_init", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.929 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "daemon_cache_timeout": { 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "daemon_cache_timeout", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:07.930 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "default_cephadm_command_timeout": { 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T07:25:07.931 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "900", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "default_registry": { 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "default_registry", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.932 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "device_cache_timeout": { 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_cache_timeout", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1800", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.933 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache device inventory", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "device_enhanced_scan": { 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_enhanced_scan", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.934 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "facts_cache_timeout": { 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "facts_cache_timeout", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.935 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache host facts data", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "grafana_dashboards_path": { 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "grafana_dashboards_path", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.936 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "host_check_interval": { 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "host_check_interval", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to perform a host check", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.937 INFO:teuthology.orchestra.run.smithi113.stdout: "hw_monitoring": { 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "hw_monitoring", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "inventory_list_all": { 2024-04-22T07:25:07.938 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "inventory_list_all", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:07.939 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.940 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "log_refresh_metadata": { 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_refresh_metadata", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T07:25:07.941 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T07:25:07.942 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.943 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.944 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.945 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.946 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "*", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "max_count_per_host": { 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_count_per_host", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.947 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "max number of daemons per service per host", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "max_osd_draining_count": { 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_osd_draining_count", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:07.948 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "migration_current": { 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "migration_current", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.949 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "internal - do not modify", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": { 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mode", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "root", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.950 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm-package", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "root" 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "oob_default_addr": { 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "oob_default_addr", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.951 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "169.254.1.1", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "prometheus_alerts_path": { 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prometheus_alerts_path", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.952 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_insecure": { 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_insecure", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.953 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_password": { 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_password", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.954 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_url": { 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_url", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.955 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_username": { 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_username", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.956 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "secure_monitoring_stack": { 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "secure_monitoring_stack", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.957 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "service_discovery_port": { 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "service_discovery_port", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8765", 2024-04-22T07:25:07.958 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "cephadm service discovery port", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "ssh_config_file": { 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssh_config_file", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.959 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "use_agent": { 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "use_agent", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.960 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "use_repo_digest": { 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "use_repo_digest", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T07:25:07.961 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_failed_host_check": { 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_failed_host_check", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T07:25:07.962 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_stray_daemons": { 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_stray_daemons", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.963 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_stray_hosts": { 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_stray_hosts", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.964 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crash", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.965 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.967 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:07.968 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.969 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "retain_interval": { 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "retain_interval", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "31536000", 2024-04-22T07:25:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_recent_interval": { 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_recent_interval", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1209600", 2024-04-22T07:25:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "dashboard", 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T07:25:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T07:25:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T07:25:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "AUDIT_API_ENABLED": { 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T07:25:07.975 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T07:25:07.976 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.977 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T07:25:07.978 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T07:25:07.979 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T07:25:07.980 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.981 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.982 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.983 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.984 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.985 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:07.986 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.987 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_URL": { 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_URL", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.988 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.989 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.990 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.991 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.992 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.993 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "[]", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.994 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "{}", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.995 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.996 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.997 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:07.999 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.000 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T07:25:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T07:25:08.002 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T07:25:08.003 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T07:25:08.004 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T07:25:08.005 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.006 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.007 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.008 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "45", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.009 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.010 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.011 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.012 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.013 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.014 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "cross_origin_url": { 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cross_origin_url", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.017 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "crt_file": { 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crt_file", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "debug": { 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "debug", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.019 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable/disable debug options", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "jwt_token_ttl": { 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "jwt_token_ttl", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "28800", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.020 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "key_file": { 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "key_file", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.021 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.022 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.023 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.024 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.025 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "motd": { 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "motd", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "The message of the day", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.027 INFO:teuthology.orchestra.run.smithi113.stdout: "redirect_resolve_ip_addr": { 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.028 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "::", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.029 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8080", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl": { 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl", 2024-04-22T07:25:08.030 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl_server_port": { 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl_server_port", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.031 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8443", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_behaviour": { 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_behaviour", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.032 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "redirect", 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "redirect" 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_error_status_code": { 2024-04-22T07:25:08.033 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_error_status_code", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "500", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "400", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "599", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "url_prefix": { 2024-04-22T07:25:08.034 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "url_prefix", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.035 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "devicehealth", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "enable_monitoring": { 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enable_monitoring", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.036 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "monitor device health metrics", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.037 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.038 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.039 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.040 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.041 INFO:teuthology.orchestra.run.smithi113.stdout: "mark_out_threshold": { 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mark_out_threshold", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "2419200", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "pool_name": { 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pool_name", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "device_health_metrics", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.043 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "retention_period": { 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "retention_period", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15552000", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long to retain device health metrics", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.044 INFO:teuthology.orchestra.run.smithi113.stdout: "scrape_frequency": { 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "scrape_frequency", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "86400", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "self_heal": { 2024-04-22T07:25:08.045 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "self_heal", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.046 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_threshold": { 2024-04-22T07:25:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_threshold", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "7257600", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "diskprediction_local", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.049 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.050 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.051 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.052 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.053 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "predict_interval": { 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "predict_interval", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "86400", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.054 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "predictor_model": { 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "predictor_model", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "prophetstor", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.055 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.056 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "influx", 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": false, 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "influxdb python module not found", 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "batch_size": { 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "batch_size", 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.057 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5000", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "database": { 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "database", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.058 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "ceph", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": { 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "hostname", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.059 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB server hostname", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "30", 2024-04-22T07:25:08.060 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "5", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.061 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.062 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.063 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.064 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.065 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "password": { 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "password", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "password of InfluxDB server user", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.066 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "port": { 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "port", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8086", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB server port", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.067 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl": { 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "false", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.068 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "threads": { 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "threads", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "32", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.069 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "username": { 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "username", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "username of InfluxDB server user", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.070 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "verify_ssl": { 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "verify_ssl", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "true", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.071 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "insights", 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.072 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.073 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.074 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.075 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.076 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "iostat", 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.077 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.078 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.079 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.080 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.081 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.082 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.083 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "k8sevents", 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_event_retention_days": { 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ceph_event_retention_days", 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.085 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "7", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "config_check_secs": { 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_check_secs", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.086 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "10", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.087 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.088 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.090 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.091 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "localpool", 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "failure_domain": { 2024-04-22T07:25:08.092 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "failure_domain", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "host", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "failure domain for any created local pool", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.093 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.094 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.095 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.096 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.097 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "min_size": { 2024-04-22T07:25:08.098 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "min_size", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default min_size for any created local pool", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "num_rep": { 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "num_rep", 2024-04-22T07:25:08.099 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default replica count for any created local pool", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_num": { 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pg_num", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.100 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "128", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "prefix": { 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prefix", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.101 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "name prefix for any created local pool", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "subtree": { 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "subtree", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.102 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "rack", 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mds_autoscaler", 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.103 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.104 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.105 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.106 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.107 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.108 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mirroring", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.109 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.110 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.111 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.112 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.113 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "nfs", 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.114 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.115 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.116 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.117 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.118 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "orchestrator", 2024-04-22T07:25:08.119 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "fail_fs": { 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "fail_fs", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.120 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.121 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.122 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.123 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.124 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.125 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator": { 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "orchestrator", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "rook", 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: "test_orchestrator" 2024-04-22T07:25:08.126 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Orchestrator backend", 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "osd_perf_query", 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.127 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.128 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.129 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.130 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.131 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "osd_support", 2024-04-22T07:25:08.132 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.133 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.134 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.135 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.136 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.137 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pg_autoscaler", 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.138 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.141 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.142 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.143 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "threshold": { 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "threshold", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3.0", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1.0", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.144 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "scaling threshold", 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "progress", 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_pg_recovery_event": { 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_pg_recovery_event", 2024-04-22T07:25:08.145 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "enabled": { 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enabled", 2024-04-22T07:25:08.146 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.147 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.148 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.150 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.151 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "max_completed_events": { 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_completed_events", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "50", 2024-04-22T07:25:08.152 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "number of past completed events to remember", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.153 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long the module is going to sleep", 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prometheus", 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.154 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "cache": { 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cache", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "exclude_perf_counters": { 2024-04-22T07:25:08.155 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "exclude_perf_counters", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.156 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.157 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.158 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.159 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.160 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_stats_pools": { 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_stats_pools", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.161 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.162 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "300", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "scrape_interval": { 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "scrape_interval", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.163 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15.0", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.164 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "::", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.165 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "9283", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "stale_cache_strategy": { 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "stale_cache_strategy", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.166 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "log", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_behaviour": { 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_behaviour", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "default", 2024-04-22T07:25:08.167 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "default", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "error" 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_error_status_code": { 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_error_status_code", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.168 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "500", 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "400", 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "599", 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_support", 2024-04-22T07:25:08.169 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.170 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.171 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.172 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.174 INFO:teuthology.orchestra.run.smithi113.stdout: "max_concurrent_snap_create": { 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_concurrent_snap_create", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "mirror_snapshot_schedule": { 2024-04-22T07:25:08.175 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.176 INFO:teuthology.orchestra.run.smithi113.stdout: "trash_purge_schedule": { 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "trash_purge_schedule", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.177 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "restful", 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "enable_auth": { 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enable_auth", 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.178 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "key_file": { 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "key_file", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.179 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.180 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.181 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.182 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.183 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.184 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.185 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.186 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rgw", 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.187 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.188 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.189 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.190 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.191 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rook", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.192 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.193 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.194 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.195 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.196 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "storage_class": { 2024-04-22T07:25:08.197 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "storage_class", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "local", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.198 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "selftest", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.199 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.200 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.201 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.202 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.203 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "roption1": { 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "roption1", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.204 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "roption2": { 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "roption2", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "xyz", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption1": { 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption1", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.206 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption2": { 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption2", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.207 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption3": { 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption3", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.208 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption4": { 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption4", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.209 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption5": { 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption5", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.210 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption6": { 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption6", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption7": { 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption7", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "42", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.212 INFO:teuthology.orchestra.run.smithi113.stdout: "testkey": { 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testkey", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "testlkey": { 2024-04-22T07:25:08.213 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testlkey", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "testnewline": { 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testnewline", 2024-04-22T07:25:08.214 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.215 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snap_schedule", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_m_granularity": { 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_m_granularity", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.216 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "dump_on_update": { 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "dump_on_update", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "dump database to debug log on update", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.217 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.218 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.219 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.220 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.221 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.222 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "stats", 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.223 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.224 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.225 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.226 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.227 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "status", 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.228 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.229 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.230 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.231 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.232 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "telegraf", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "address": { 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "address", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.234 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.235 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.236 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.237 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.238 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.239 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.240 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "telemetry", 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_basic": { 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_basic", 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.241 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_crash": { 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_crash", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.242 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_device": { 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_device", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.243 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_ident": { 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_ident", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.244 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_perf": { 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_perf", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.245 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "contact": { 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "contact", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.246 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "description": { 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "description", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.247 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "device_url": { 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_url", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "enabled": { 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enabled", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.249 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "24", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "8", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.250 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "last_opt_revision": { 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "last_opt_revision", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.251 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "leaderboard": { 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "leaderboard", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.252 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "leaderboard_description": { 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "leaderboard_description", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.253 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.254 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.255 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.256 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.257 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.258 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "organization": { 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "organization", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.259 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "proxy": { 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "proxy", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.260 INFO:teuthology.orchestra.run.smithi113.stdout: "url": { 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "url", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.261 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "test_orchestrator", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.262 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.263 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.264 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.265 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.266 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "volumes", 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.267 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.268 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.269 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.270 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.271 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "max_concurrent_clones": { 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_concurrent_clones", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.272 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "4", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "periodic_async_work": { 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "periodic_async_work", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.273 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Periodically check for async work", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "snapshot_clone_delay": { 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snapshot_clone_delay", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.274 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "snapshot_clone_no_wait": { 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.275 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix", 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.276 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "discovery_interval": { 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "discovery_interval", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "100", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.277 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "identifier": { 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "identifier", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.278 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.279 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.280 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.281 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.282 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.283 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_host": { 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_host", 2024-04-22T07:25:08.284 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_port": { 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_port", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.285 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10051", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_sender": { 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_sender", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.286 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.287 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T07:25:08.287 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:08.290 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "gid": 24890, 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "y", 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "mgr_features": 4540701547738038271, 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "available_modules": [ 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "alerts", 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.291 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.292 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.293 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.295 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.296 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_destination": { 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_destination", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Email address to send alerts to", 2024-04-22T07:25:08.297 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_from_name": { 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_from_name", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "Ceph", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.298 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Email From: name", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_host": { 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_host", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP server", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.299 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_password": { 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_password", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Password to authenticate with", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.300 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_port": { 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_port", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "465", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP port", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.301 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_sender": { 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_sender", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP envelope sender", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.302 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_ssl": { 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_ssl", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_user": { 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_user", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "User to authenticate as", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.304 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "balancer", 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "active": { 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "active", 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.305 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "begin_time": { 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "begin_time", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.306 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0000", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "begin_weekday": { 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "begin_weekday", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.307 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "6", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_max_iterations": { 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_max_iterations", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "25", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.308 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "250", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_metrics": { 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_metrics", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.309 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_step": { 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_step", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0.5", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0.001", 2024-04-22T07:25:08.310 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "0.999", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "aggressiveness of optimization", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "end_time": { 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "end_time", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "2359", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.311 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "end_weekday": { 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "end_weekday", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "6", 2024-04-22T07:25:08.312 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.313 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.314 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.315 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.316 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.317 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "min_score": { 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "min_score", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.318 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": { 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mode", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "upmap", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "crush-compat", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "none", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "read", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap", 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap-read" 2024-04-22T07:25:08.319 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Balancer mode", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "pool_ids": { 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pool_ids", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T07:25:08.320 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.321 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap_max_deviation": { 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "upmap_max_deviation", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T07:25:08.322 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap_max_optimizations": { 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "upmap_max_optimizations", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.323 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cephadm", 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_down_multiplier": { 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_down_multiplier", 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.324 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3.0", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_refresh_rate": { 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_refresh_rate", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.325 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "20", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_starting_port": { 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_starting_port", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.326 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "4721", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_ptrace": { 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_ptrace", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.327 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "autotune_interval": { 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "autotune_interval", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.328 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "autotune_memory_target_ratio": { 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.329 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0.7", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm_log_destination": { 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cephadm_log_destination", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.330 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "file", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "file,syslog", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "syslog" 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "cgroups_split": { 2024-04-22T07:25:08.331 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cgroups_split", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "config_checks_enabled": { 2024-04-22T07:25:08.332 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_checks_enabled", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "config_dashboard": { 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_dashboard", 2024-04-22T07:25:08.333 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_alertmanager": { 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_alertmanager", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.334 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_base": { 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_base", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.335 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Container image name, without the tag", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_elasticsearch": { 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_elasticsearch", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T07:25:08.336 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "elasticsearch container image", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_grafana": { 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_grafana", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.337 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_haproxy": { 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_haproxy", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.338 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "HAproxy container image", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_agent": { 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_agent", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger agent container image", 2024-04-22T07:25:08.339 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_collector": { 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_collector", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger collector container image", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.340 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_query": { 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_query", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger query container image", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.341 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_keepalived": { 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_keepalived", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Keepalived container image", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_loki": { 2024-04-22T07:25:08.342 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_loki", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Loki container image", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_node_exporter": { 2024-04-22T07:25:08.343 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_node_exporter", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_nvmeof": { 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_nvmeof", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.344 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Nvme-of container image", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_prometheus": { 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_prometheus", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.345 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_promtail": { 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_promtail", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.346 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Promtail container image", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_samba": { 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_samba", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.347 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Samba/SMB container image", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_snmp_gateway": { 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_snmp_gateway", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.348 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SNMP Gateway container image", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "container_init": { 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_init", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.349 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "daemon_cache_timeout": { 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "daemon_cache_timeout", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.350 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "default_cephadm_command_timeout": { 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "900", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.351 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "default_registry": { 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "default_registry", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.352 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "device_cache_timeout": { 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_cache_timeout", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1800", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.353 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache device inventory", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "device_enhanced_scan": { 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_enhanced_scan", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "facts_cache_timeout": { 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "facts_cache_timeout", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache host facts data", 2024-04-22T07:25:08.355 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "grafana_dashboards_path": { 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "grafana_dashboards_path", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.356 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "host_check_interval": { 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "host_check_interval", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to perform a host check", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "hw_monitoring": { 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "hw_monitoring", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "inventory_list_all": { 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "inventory_list_all", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.360 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "log_refresh_metadata": { 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_refresh_metadata", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.361 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.362 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.363 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.364 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.365 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.366 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "*", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "max_count_per_host": { 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_count_per_host", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.367 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "max number of daemons per service per host", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "max_osd_draining_count": { 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_osd_draining_count", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.368 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "migration_current": { 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "migration_current", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.369 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "internal - do not modify", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": { 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mode", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "root", 2024-04-22T07:25:08.370 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm-package", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "root" 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "oob_default_addr": { 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "oob_default_addr", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.371 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "169.254.1.1", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "prometheus_alerts_path": { 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prometheus_alerts_path", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T07:25:08.372 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_insecure": { 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_insecure", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.373 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_password": { 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_password", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.374 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_url": { 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_url", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.375 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_username": { 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_username", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.376 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "secure_monitoring_stack": { 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "secure_monitoring_stack", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "service_discovery_port": { 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "service_discovery_port", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8765", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.378 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "cephadm service discovery port", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "ssh_config_file": { 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssh_config_file", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.379 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "use_agent": { 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "use_agent", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T07:25:08.380 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "use_repo_digest": { 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "use_repo_digest", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.381 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_failed_host_check": { 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_failed_host_check", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.382 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_stray_daemons": { 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_stray_daemons", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_stray_hosts": { 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_stray_hosts", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.384 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crash", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.385 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.386 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.387 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.388 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.389 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "retain_interval": { 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "retain_interval", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "31536000", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.390 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_recent_interval": { 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_recent_interval", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1209600", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.391 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "dashboard", 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.392 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.393 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.394 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "AUDIT_API_ENABLED": { 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.395 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.396 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.397 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.398 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.399 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.400 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.401 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.402 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.403 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.404 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.405 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.406 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.407 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_URL": { 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_URL", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.408 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.409 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.410 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.411 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.412 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.413 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "[]", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.414 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "{}", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.416 INFO:teuthology.orchestra.run.smithi113.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T07:25:08.417 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T07:25:08.418 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.419 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.420 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.421 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.422 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.423 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.424 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.425 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.426 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.427 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "45", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.428 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.429 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.430 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.432 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.434 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.435 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.436 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "cross_origin_url": { 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cross_origin_url", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.437 INFO:teuthology.orchestra.run.smithi113.stdout: "crt_file": { 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crt_file", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.438 INFO:teuthology.orchestra.run.smithi113.stdout: "debug": { 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "debug", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable/disable debug options", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.439 INFO:teuthology.orchestra.run.smithi113.stdout: "jwt_token_ttl": { 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "jwt_token_ttl", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "28800", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.440 INFO:teuthology.orchestra.run.smithi113.stdout: "key_file": { 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "key_file", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.441 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.443 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.444 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.445 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "motd": { 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "motd", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.446 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "The message of the day", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "redirect_resolve_ip_addr": { 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.447 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "::", 2024-04-22T07:25:08.448 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8080", 2024-04-22T07:25:08.449 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl": { 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.450 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl_server_port": { 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl_server_port", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8443", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.451 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_behaviour": { 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_behaviour", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "redirect", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: "redirect" 2024-04-22T07:25:08.452 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_error_status_code": { 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_error_status_code", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "500", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "400", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "599", 2024-04-22T07:25:08.453 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "url_prefix": { 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "url_prefix", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.454 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "devicehealth", 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "enable_monitoring": { 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enable_monitoring", 2024-04-22T07:25:08.455 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "monitor device health metrics", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.456 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.457 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.458 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.459 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.460 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "mark_out_threshold": { 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mark_out_threshold", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.461 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "2419200", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "pool_name": { 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pool_name", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "device_health_metrics", 2024-04-22T07:25:08.462 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "retention_period": { 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "retention_period", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15552000", 2024-04-22T07:25:08.463 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long to retain device health metrics", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "scrape_frequency": { 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "scrape_frequency", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "86400", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.464 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "self_heal": { 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "self_heal", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.465 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.466 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_threshold": { 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_threshold", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "7257600", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.467 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "diskprediction_local", 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.468 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.469 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.470 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.471 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.472 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.473 INFO:teuthology.orchestra.run.smithi113.stdout: "predict_interval": { 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "predict_interval", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "86400", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.474 INFO:teuthology.orchestra.run.smithi113.stdout: "predictor_model": { 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "predictor_model", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "prophetstor", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.475 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.476 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "influx", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": false, 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "influxdb python module not found", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "batch_size": { 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "batch_size", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5000", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.477 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "database": { 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "database", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "ceph", 2024-04-22T07:25:08.478 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": { 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "hostname", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.479 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB server hostname", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "30", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "5", 2024-04-22T07:25:08.480 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.481 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.482 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.483 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.484 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "password": { 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "password", 2024-04-22T07:25:08.485 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "password of InfluxDB server user", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "port": { 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "port", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.486 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8086", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB server port", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl": { 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.487 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "false", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "threads": { 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "threads", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.488 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "32", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "username": { 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "username", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.489 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "username of InfluxDB server user", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "verify_ssl": { 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "verify_ssl", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "true", 2024-04-22T07:25:08.490 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "insights", 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.491 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.492 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.493 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.494 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.495 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "iostat", 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.496 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.497 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.498 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.499 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.500 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "k8sevents", 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.501 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_event_retention_days": { 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ceph_event_retention_days", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "7", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "config_check_secs": { 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_check_secs", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "10", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.503 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.504 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.505 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.506 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.507 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "localpool", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "failure_domain": { 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "failure_domain", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "host", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.509 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "failure domain for any created local pool", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.510 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.513 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.514 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "min_size": { 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "min_size", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.515 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default min_size for any created local pool", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "num_rep": { 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "num_rep", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.516 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default replica count for any created local pool", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_num": { 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pg_num", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "128", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.517 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "prefix": { 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prefix", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.518 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "name prefix for any created local pool", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "subtree": { 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "subtree", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "rack", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T07:25:08.519 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mds_autoscaler", 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.520 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.523 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.524 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mirroring", 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.526 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.527 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.528 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.529 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.530 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "nfs", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.531 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.532 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.533 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.535 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "orchestrator", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "fail_fs": { 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "fail_fs", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T07:25:08.536 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.537 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.538 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.539 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.541 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator": { 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "orchestrator", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "rook", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "test_orchestrator" 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Orchestrator backend", 2024-04-22T07:25:08.542 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "osd_perf_query", 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.543 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.544 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.546 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.547 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "osd_support", 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.548 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.549 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.550 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.551 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.552 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.553 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pg_autoscaler", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.554 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.555 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.556 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.557 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.558 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.559 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "threshold": { 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "threshold", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3.0", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1.0", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "scaling threshold", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.560 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "progress", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_pg_recovery_event": { 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_pg_recovery_event", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.561 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "enabled": { 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enabled", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.562 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.563 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.564 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.567 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.568 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.569 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "max_completed_events": { 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_completed_events", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "50", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.570 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "number of past completed events to remember", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.571 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long the module is going to sleep", 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prometheus", 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "cache": { 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cache", 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.572 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "exclude_perf_counters": { 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "exclude_perf_counters", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.573 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.574 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.575 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.576 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.577 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.578 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_stats_pools": { 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_stats_pools", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.579 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.580 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "300", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "scrape_interval": { 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "scrape_interval", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.581 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15.0", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.582 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "::", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "9283", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T07:25:08.584 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "stale_cache_strategy": { 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "stale_cache_strategy", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "log", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.585 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_behaviour": { 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_behaviour", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "default", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "default", 2024-04-22T07:25:08.586 INFO:teuthology.orchestra.run.smithi113.stdout: "error" 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_error_status_code": { 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_error_status_code", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "500", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "400", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "599", 2024-04-22T07:25:08.587 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_support", 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.588 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.589 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.590 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.591 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.592 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "max_concurrent_snap_create": { 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_concurrent_snap_create", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.593 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "mirror_snapshot_schedule": { 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.594 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "trash_purge_schedule": { 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "trash_purge_schedule", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.595 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "restful", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.596 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "enable_auth": { 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enable_auth", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "key_file": { 2024-04-22T07:25:08.597 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "key_file", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.598 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.599 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.600 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.601 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.602 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.603 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.604 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rgw", 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.605 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.606 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.607 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.608 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.609 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.610 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rook", 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.611 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.612 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.613 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.615 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "storage_class": { 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "storage_class", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "local", 2024-04-22T07:25:08.616 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "selftest", 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.617 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.618 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.619 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.620 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.621 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.622 INFO:teuthology.orchestra.run.smithi113.stdout: "roption1": { 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "roption1", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.623 INFO:teuthology.orchestra.run.smithi113.stdout: "roption2": { 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "roption2", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "xyz", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption1": { 2024-04-22T07:25:08.624 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption1", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption2": { 2024-04-22T07:25:08.625 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption2", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption3": { 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption3", 2024-04-22T07:25:08.626 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption4": { 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption4", 2024-04-22T07:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption5": { 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption5", 2024-04-22T07:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption6": { 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption6", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.629 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption7": { 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption7", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.630 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "42", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "testkey": { 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testkey", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.631 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "testlkey": { 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testlkey", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.632 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "testnewline": { 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testnewline", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.633 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snap_schedule", 2024-04-22T07:25:08.634 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_m_granularity": { 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_m_granularity", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T07:25:08.635 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "dump_on_update": { 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "dump_on_update", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "dump database to debug log on update", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.636 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.637 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.638 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.639 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.640 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.641 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "stats", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.642 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.643 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.644 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.645 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.646 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "status", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.647 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.648 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.649 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.650 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.651 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "telegraf", 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "address": { 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "address", 2024-04-22T07:25:08.652 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.653 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.654 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.655 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.656 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.657 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.658 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.659 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "telemetry", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_basic": { 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_basic", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.660 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_crash": { 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_crash", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T07:25:08.661 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_device": { 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_device", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T07:25:08.662 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_ident": { 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_ident", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T07:25:08.663 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_perf": { 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_perf", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "contact": { 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "contact", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "description": { 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "description", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.666 INFO:teuthology.orchestra.run.smithi113.stdout: "device_url": { 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_url", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "enabled": { 2024-04-22T07:25:08.667 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enabled", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.668 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "24", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "8", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "last_opt_revision": { 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "last_opt_revision", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.669 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "leaderboard": { 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "leaderboard", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.670 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "leaderboard_description": { 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "leaderboard_description", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.671 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.672 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.673 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.674 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.675 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.676 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "organization": { 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "organization", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.677 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "proxy": { 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "proxy", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.678 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "url": { 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "url", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.679 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "test_orchestrator", 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.680 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.681 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.682 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.683 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.684 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "volumes", 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.685 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.686 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.687 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.688 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.689 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "max_concurrent_clones": { 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_concurrent_clones", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.690 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "4", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "periodic_async_work": { 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "periodic_async_work", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.691 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Periodically check for async work", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "snapshot_clone_delay": { 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snapshot_clone_delay", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.692 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "snapshot_clone_no_wait": { 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.693 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix", 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.694 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "discovery_interval": { 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "discovery_interval", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "100", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.695 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "identifier": { 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "identifier", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.696 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.697 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.698 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.699 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.700 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.700 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.701 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.702 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.703 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_host": { 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_host", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.704 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_port": { 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_port", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10051", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.705 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_sender": { 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_sender", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.706 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "modules": [ 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm", 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "dashboard", 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "iostat", 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "nfs", 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "restful", 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "snap_schedule" 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.707 INFO:teuthology.orchestra.run.smithi113.stdout: "available_modules": [ 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "alerts", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.708 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.709 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.710 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.711 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.712 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_destination": { 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_destination", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Email address to send alerts to", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_from_name": { 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_from_name", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "Ceph", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Email From: name", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_host": { 2024-04-22T07:25:08.715 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_host", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP server", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_password": { 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_password", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.716 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Password to authenticate with", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_port": { 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_port", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.717 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "465", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP port", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_sender": { 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_sender", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.718 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SMTP envelope sender", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_ssl": { 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_ssl", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.719 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "smtp_user": { 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "smtp_user", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.720 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "User to authenticate as", 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "balancer", 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.721 INFO:teuthology.orchestra.run.smithi113.stdout: "active": { 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "active", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "begin_time": { 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "begin_time", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0000", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.723 INFO:teuthology.orchestra.run.smithi113.stdout: "begin_weekday": { 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "begin_weekday", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "6", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.724 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_max_iterations": { 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_max_iterations", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "25", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "250", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.725 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_metrics": { 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_metrics", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.726 INFO:teuthology.orchestra.run.smithi113.stdout: "crush_compat_step": { 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crush_compat_step", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0.5", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0.001", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "0.999", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "aggressiveness of optimization", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "end_time": { 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "end_time", 2024-04-22T07:25:08.727 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "2359", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "end_weekday": { 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "end_weekday", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.728 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "0", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "6", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.729 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.730 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.731 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.732 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.733 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "min_score": { 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "min_score", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.734 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": { 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mode", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.735 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "upmap", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "crush-compat", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "none", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "read", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap-read" 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Balancer mode", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.736 INFO:teuthology.orchestra.run.smithi113.stdout: "pool_ids": { 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pool_ids", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.737 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap_max_deviation": { 2024-04-22T07:25:08.738 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "upmap_max_deviation", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "upmap_max_optimizations": { 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "upmap_max_optimizations", 2024-04-22T07:25:08.739 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cephadm", 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.740 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_down_multiplier": { 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_down_multiplier", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3.0", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.741 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_refresh_rate": { 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_refresh_rate", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "20", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.742 INFO:teuthology.orchestra.run.smithi113.stdout: "agent_starting_port": { 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "agent_starting_port", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "4721", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_ptrace": { 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_ptrace", 2024-04-22T07:25:08.743 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "autotune_interval": { 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "autotune_interval", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.744 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "autotune_memory_target_ratio": { 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.745 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0.7", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm_log_destination": { 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cephadm_log_destination", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.746 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "file", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "file,syslog", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "syslog" 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "cgroups_split": { 2024-04-22T07:25:08.747 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cgroups_split", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "config_checks_enabled": { 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_checks_enabled", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.748 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "config_dashboard": { 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_dashboard", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.749 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_alertmanager": { 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_alertmanager", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.750 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_base": { 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_base", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.751 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Container image name, without the tag", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_elasticsearch": { 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_elasticsearch", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T07:25:08.752 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "elasticsearch container image", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_grafana": { 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_grafana", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.753 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_haproxy": { 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_haproxy", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.754 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "HAproxy container image", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_agent": { 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_agent", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.755 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger agent container image", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_collector": { 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_collector", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger collector container image", 2024-04-22T07:25:08.756 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_jaeger_query": { 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_jaeger_query", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Jaeger query container image", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.757 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_keepalived": { 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_keepalived", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Keepalived container image", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.758 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_loki": { 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_loki", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Loki container image", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.759 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_node_exporter": { 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_node_exporter", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.760 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_nvmeof": { 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_nvmeof", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Nvme-of container image", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_prometheus": { 2024-04-22T07:25:08.761 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_prometheus", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Prometheus container image", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_promtail": { 2024-04-22T07:25:08.762 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_promtail", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Promtail container image", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.763 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_samba": { 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_samba", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Samba/SMB container image", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.764 INFO:teuthology.orchestra.run.smithi113.stdout: "container_image_snmp_gateway": { 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_image_snmp_gateway", 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.765 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "SNMP Gateway container image", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "container_init": { 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "container_init", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T07:25:08.766 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "daemon_cache_timeout": { 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "daemon_cache_timeout", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T07:25:08.767 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "default_cephadm_command_timeout": { 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "900", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.768 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "default_registry": { 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "default_registry", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "docker.io", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.769 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "device_cache_timeout": { 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_cache_timeout", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1800", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache device inventory", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.770 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "device_enhanced_scan": { 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_enhanced_scan", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "facts_cache_timeout": { 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "facts_cache_timeout", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "seconds to cache host facts data", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "grafana_dashboards_path": { 2024-04-22T07:25:08.772 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "grafana_dashboards_path", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "host_check_interval": { 2024-04-22T07:25:08.773 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "host_check_interval", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to perform a host check", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "hw_monitoring": { 2024-04-22T07:25:08.774 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "hw_monitoring", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "inventory_list_all": { 2024-04-22T07:25:08.775 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "inventory_list_all", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.776 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.777 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "log_refresh_metadata": { 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_refresh_metadata", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T07:25:08.778 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.779 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.780 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.781 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.782 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "*", 2024-04-22T07:25:08.783 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "max_count_per_host": { 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_count_per_host", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.784 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "max number of daemons per service per host", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "max_osd_draining_count": { 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_osd_draining_count", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.785 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "migration_current": { 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "migration_current", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.786 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "internal - do not modify", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "mode": { 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mode", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "root", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.787 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm-package", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "root" 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "oob_default_addr": { 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "oob_default_addr", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "169.254.1.1", 2024-04-22T07:25:08.788 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "prometheus_alerts_path": { 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prometheus_alerts_path", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.789 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_insecure": { 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_insecure", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.790 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_password": { 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_password", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.791 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_url": { 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_url", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.792 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "registry_username": { 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "registry_username", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.793 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "secure_monitoring_stack": { 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "secure_monitoring_stack", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.794 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "service_discovery_port": { 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "service_discovery_port", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8765", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.795 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "cephadm service discovery port", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "ssh_config_file": { 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssh_config_file", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.796 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "use_agent": { 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "use_agent", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.797 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "use_repo_digest": { 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "use_repo_digest", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.798 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_failed_host_check": { 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_failed_host_check", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.799 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_stray_daemons": { 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_stray_daemons", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.800 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_on_stray_hosts": { 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_on_stray_hosts", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.801 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crash", 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.802 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.803 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.804 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.805 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.806 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "retain_interval": { 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "retain_interval", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.807 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "31536000", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_recent_interval": { 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_recent_interval", 2024-04-22T07:25:08.808 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1209600", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.809 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "dashboard", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.810 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.811 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.812 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "AUDIT_API_ENABLED": { 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.813 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.814 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.815 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.816 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T07:25:08.817 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T07:25:08.818 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T07:25:08.819 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T07:25:08.820 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T07:25:08.821 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T07:25:08.822 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.823 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.824 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_URL": { 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_URL", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.825 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T07:25:08.826 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.827 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.828 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.829 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.830 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.834 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "[]", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.835 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "{}", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.836 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.837 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.838 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.839 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.840 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.841 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.842 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.843 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.844 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.845 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.846 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.847 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.848 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.849 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "45", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.850 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.851 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "admin", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.852 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.853 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.854 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.855 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.856 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.857 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.858 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "cross_origin_url": { 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cross_origin_url", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.859 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "crt_file": { 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "crt_file", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.860 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "debug": { 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "debug", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Enable/disable debug options", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.861 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "jwt_token_ttl": { 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "jwt_token_ttl", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "28800", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.862 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "key_file": { 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "key_file", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.863 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.864 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.865 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.866 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.867 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "motd": { 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "motd", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.868 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "The message of the day", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "redirect_resolve_ip_addr": { 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.869 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.870 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "::", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.871 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8080", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl": { 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.872 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl_server_port": { 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl_server_port", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.873 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8443", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_behaviour": { 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_behaviour", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.874 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "redirect", 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "redirect" 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.875 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_error_status_code": { 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_error_status_code", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "500", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "400", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "599", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "url_prefix": { 2024-04-22T07:25:08.876 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "url_prefix", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.877 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "devicehealth", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "enable_monitoring": { 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enable_monitoring", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.878 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "monitor device health metrics", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.879 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.880 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.881 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.882 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.883 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "mark_out_threshold": { 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mark_out_threshold", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "2419200", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.884 INFO:teuthology.orchestra.run.smithi113.stdout: "pool_name": { 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pool_name", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "device_health_metrics", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.885 INFO:teuthology.orchestra.run.smithi113.stdout: "retention_period": { 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "retention_period", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15552000", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long to retain device health metrics", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.886 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "scrape_frequency": { 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "scrape_frequency", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "86400", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.887 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "self_heal": { 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "self_heal", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.888 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.889 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "warn_threshold": { 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "warn_threshold", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "7257600", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "diskprediction_local", 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.895 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "predict_interval": { 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "predict_interval", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "86400", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.896 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "predictor_model": { 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "predictor_model", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "prophetstor", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.897 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "600", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.898 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "influx", 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": false, 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "influxdb python module not found", 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "batch_size": { 2024-04-22T07:25:08.899 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "batch_size", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5000", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "database": { 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "database", 2024-04-22T07:25:08.900 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "ceph", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": { 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "hostname", 2024-04-22T07:25:08.901 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB server hostname", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:08.902 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "30", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "5", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.903 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.904 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.905 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.906 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.907 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "password": { 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "password", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.908 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "password of InfluxDB server user", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "port": { 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "port", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.909 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "8086", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "InfluxDB server port", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "ssl": { 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ssl", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.910 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "false", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "threads": { 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "threads", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.911 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "32", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "username": { 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "username", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.912 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "username of InfluxDB server user", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "verify_ssl": { 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "verify_ssl", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.913 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "true", 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "insights", 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "iostat", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "k8sevents", 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_event_retention_days": { 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "ceph_event_retention_days", 2024-04-22T07:25:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "7", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "config_check_secs": { 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "config_check_secs", 2024-04-22T07:25:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "10", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.928 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.930 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.931 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "localpool", 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "failure_domain": { 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "failure_domain", 2024-04-22T07:25:08.932 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "host", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "failure domain for any created local pool", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.933 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.934 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.935 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.936 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.937 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "min_size": { 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "min_size", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.938 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default min_size for any created local pool", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "num_rep": { 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "num_rep", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.939 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default replica count for any created local pool", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_num": { 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pg_num", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "128", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.940 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "prefix": { 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prefix", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.941 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "name prefix for any created local pool", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "subtree": { 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "subtree", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "rack", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.942 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mds_autoscaler", 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.943 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.944 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.945 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.946 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.947 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mirroring", 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.948 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.949 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.950 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.951 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.952 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.953 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "nfs", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.954 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.955 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.956 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.957 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.958 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "orchestrator", 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "fail_fs": { 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "fail_fs", 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.959 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.960 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.961 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.962 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.963 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.964 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.965 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator": { 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "orchestrator", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "cephadm", 2024-04-22T07:25:08.966 INFO:teuthology.orchestra.run.smithi113.stdout: "rook", 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "test_orchestrator" 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Orchestrator backend", 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "osd_perf_query", 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.967 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.969 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.970 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.971 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "osd_support", 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.972 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.973 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.974 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.975 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.977 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "pg_autoscaler", 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.978 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.979 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.980 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.981 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.982 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "threshold": { 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "threshold", 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "3.0", 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1.0", 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.984 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "scaling threshold", 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7f540a3d50 msgr2=0x7f7f540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7f540a3d50 0x7f7f540a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c06b0d0 tx=0x7f7f60008040 comp rx=0 tx=0).stop 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 msgr2=0x7f7f6c1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c1996e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005940 tx=0x7f7f5c004040 comp rx=0 tx=0).stop 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 shutdown_connections 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f7f540a3d50 0x7f7f540a6210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c19a160 0x7f7f6c1a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "progress", 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_pg_recovery_event": { 2024-04-22T07:25:08.986 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_pg_recovery_event", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "enabled": { 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enabled", 2024-04-22T07:25:08.987 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.988 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.989 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.990 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.991 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.992 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "max_completed_events": { 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_completed_events", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "50", 2024-04-22T07:25:08.993 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "number of past completed events to remember", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "sleep_interval": { 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "sleep_interval", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.994 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "5", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "how long the module is going to sleep", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "prometheus", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:08.995 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "cache": { 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "cache", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.996 INFO:teuthology.orchestra.run.smithi113.stdout: "exclude_perf_counters": { 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "exclude_perf_counters", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.997 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:08.998 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:08.999 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.000 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.001 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_stats_pools": { 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_stats_pools", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.002 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T07:25:09.003 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "300", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "scrape_interval": { 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "scrape_interval", 2024-04-22T07:25:09.004 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15.0", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:09.005 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "::", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:09.006 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "9283", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "stale_cache_strategy": { 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "stale_cache_strategy", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.007 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "log", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_behaviour": { 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_behaviour", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.008 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "default", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "default", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "error" 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.009 INFO:teuthology.orchestra.run.smithi113.stdout: "standby_error_status_code": { 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "standby_error_status_code", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "500", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "400", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "599", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.010 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_support", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.011 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.012 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.013 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.014 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.015 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "max_concurrent_snap_create": { 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_concurrent_snap_create", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.016 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "mirror_snapshot_schedule": { 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.017 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "trash_purge_schedule": { 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "trash_purge_schedule", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.018 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "restful", 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "enable_auth": { 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enable_auth", 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.019 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "key_file": { 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "key_file", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.020 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.021 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.023 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.024 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.025 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "server_addr": { 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_addr", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.026 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "server_port": { 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "server_port", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.027 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rgw", 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.028 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.029 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.030 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.031 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.032 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.033 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rook", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.034 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.035 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.036 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.037 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.038 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "storage_class": { 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "storage_class", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "local", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.039 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "selftest", 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.040 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.041 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.042 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.043 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.044 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "roption1": { 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "roption1", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.045 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "roption2": { 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "roption2", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "xyz", 2024-04-22T07:25:09.046 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption1": { 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption1", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.047 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption2": { 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption2", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.048 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption3": { 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption3", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "float", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.049 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption4": { 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption4", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.050 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption5": { 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption5", 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.051 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6c106160 0x7f7f6c199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 --2- 172.21.15.113:0/1038139317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f6c101090 0x7f7f6c1996e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 >> 172.21.15.113:0/1038139317 conn(0x7f7f6c0771d0 msgr2=0x7f7f6c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.052 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption6": { 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption6", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.053 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "rwoption7": { 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rwoption7", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "1", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "42", 2024-04-22T07:25:09.054 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "testkey": { 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testkey", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.055 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "testlkey": { 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testlkey", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.056 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "testnewline": { 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "testnewline", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snap_schedule", 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "allow_m_granularity": { 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "allow_m_granularity", 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.058 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "dump_on_update": { 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "dump_on_update", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.059 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "dump database to debug log on update", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.060 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.061 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.062 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.063 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "stats", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.065 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.066 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.067 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.068 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.069 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.070 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "status", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.071 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.072 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.073 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.074 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.075 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "telegraf", 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "address": { 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "address", 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.076 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.077 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "15", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.078 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.079 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.080 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.081 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.082 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "telemetry", 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_basic": { 2024-04-22T07:25:09.083 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_basic", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_crash": { 2024-04-22T07:25:09.084 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_crash", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_device": { 2024-04-22T07:25:09.085 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_device", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.086 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_ident": { 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_ident", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.087 INFO:teuthology.orchestra.run.smithi113.stdout: "channel_perf": { 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "channel_perf", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "contact": { 2024-04-22T07:25:09.088 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "contact", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "description": { 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "description", 2024-04-22T07:25:09.089 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "device_url": { 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "device_url", 2024-04-22T07:25:09.090 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "enabled": { 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "enabled", 2024-04-22T07:25:09.091 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:09.092 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "24", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "8", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "last_opt_revision": { 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "last_opt_revision", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.093 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "1", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "leaderboard": { 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "leaderboard", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.094 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "leaderboard_description": { 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "leaderboard_description", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.095 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.096 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.097 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.098 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.099 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.100 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "organization": { 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "organization", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.101 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "proxy": { 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "proxy", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.102 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "url": { 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "url", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.103 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "test_orchestrator", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.104 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.105 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.106 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.107 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.109 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "volumes", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.110 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.111 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.112 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.113 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.114 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "max_concurrent_clones": { 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "max_concurrent_clones", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "4", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T07:25:09.115 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "periodic_async_work": { 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "periodic_async_work", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Periodically check for async work", 2024-04-22T07:25:09.116 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "snapshot_clone_delay": { 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snapshot_clone_delay", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "0", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.117 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "snapshot_clone_no_wait": { 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "True", 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.118 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 shutdown_connections 2024-04-22T07:25:09.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:07.868+0000 7f7f7190e640 1 -- 172.21.15.113:0/1038139317 wait complete. 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix", 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "can_run": true, 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "error_string": "", 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "module_options": { 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "discovery_interval": { 2024-04-22T07:25:09.120 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "discovery_interval", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "uint", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "100", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "identifier": { 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "identifier", 2024-04-22T07:25:09.121 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "interval": { 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "interval", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "secs", 2024-04-22T07:25:09.122 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "60", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "log_level": { 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_level", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.123 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.124 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster": { 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.125 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_cluster_level": { 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_cluster_level", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "info", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [ 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "critical", 2024-04-22T07:25:09.126 INFO:teuthology.orchestra.run.smithi113.stdout: "debug", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "error", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "info", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "warning" 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "log_to_file": { 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "log_to_file", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "bool", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.127 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 1, 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "False", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_host": { 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_host", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.128 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_port": { 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_port", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "int", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "10051", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.129 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "zabbix_sender": { 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "zabbix_sender", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "str", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "level": "advanced", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "flags": 0, 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T07:25:09.130 INFO:teuthology.orchestra.run.smithi113.stdout: "min": "", 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "max": "", 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "enum_allowed": [], 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "desc": "", 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "long_desc": "", 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "tags": [], 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "see_also": [] 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "services": { 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "dashboard": "https://172.21.15.113:8443/" 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "always_on_modules": { 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "octopus": [ 2024-04-22T07:25:09.131 INFO:teuthology.orchestra.run.smithi113.stdout: "balancer", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "crash", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "devicehealth", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_autoscaler", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "progress", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_support", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "status", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "telemetry", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "volumes" 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "pacific": [ 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "balancer", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "crash", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "devicehealth", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator", 2024-04-22T07:25:09.132 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_autoscaler", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "progress", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_support", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "status", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "telemetry", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "volumes" 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "quincy": [ 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "balancer", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "crash", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "devicehealth", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_autoscaler", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "progress", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_support", 2024-04-22T07:25:09.133 INFO:teuthology.orchestra.run.smithi113.stdout: "status", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "telemetry", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "volumes" 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "reef": [ 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "balancer", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "crash", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "devicehealth", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_autoscaler", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "progress", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_support", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "status", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "telemetry", 2024-04-22T07:25:09.134 INFO:teuthology.orchestra.run.smithi113.stdout: "volumes" 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "squid": [ 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "balancer", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "crash", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "devicehealth", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "orchestrator", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "pg_autoscaler", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "progress", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "rbd_support", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "status", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "telemetry", 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: "volumes" 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "last_failure_osd_epoch": 89, 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "active_clients": [ 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "devicehealth", 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:0", 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2745034638 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "libcephsqlite", 2024-04-22T07:25:09.136 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:0", 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2901151916 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "rbd_support", 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:0", 2024-04-22T07:25:09.137 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 1411224302 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snap_schedule", 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:0", 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 505667314 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "snap_schedule", 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:0", 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 567144824 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "name": "volumes", 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "addrvec": [ 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v2", 2024-04-22T07:25:09.139 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "172.21.15.113:0", 2024-04-22T07:25:09.140 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 3351625758 2024-04-22T07:25:09.140 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.140 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.140 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:09.140 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-22T07:25:09.140 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:25:09.146 DEBUG:tasks.check_counter:Getting stats from a 2024-04-22T07:25:09.147 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-22T07:25:09.147 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:08 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1181629347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:25:09.148 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:08 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/1038139317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T07:25:09.148 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:08 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1181629347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:25:09.148 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:08 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/1038139317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T07:25:09.148 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:08 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1181629347' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:25:09.148 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:08 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/1038139317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T07:25:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:25:09.513 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:09 smithi113 ceph-mon[27933]: pgmap v325: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T07:25:09.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:09 smithi133 ceph-mon[31198]: pgmap v325: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T07:25:09.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:09 smithi154 ceph-mon[31862]: pgmap v325: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 4, 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 4, 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 3040, 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 448, 2024-04-22T07:25:10.998 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.001403421, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.000030839, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.000498346, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 3, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000663502, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000221167 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:10.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 153, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 320, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 183136, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.013043425, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.009110854, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.001860894, 2024-04-22T07:25:11.000 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 150, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.014795056, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000098633 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 183136 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.001 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 212, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 201, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1587200, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 123424, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 2, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.042384280, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.013485228, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.033024282, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.004225958, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 200, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.010131319, 2024-04-22T07:25:11.002 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000050656 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1587200, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 123424 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 0, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.003 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:11.004 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:11.005 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 149017, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 31, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:11.006 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 928, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 2, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-22T07:25:11.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 214, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 1557312, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 214, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 214, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1557312, 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-22T07:25:11.008 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:25:11.644 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:11 smithi113 ceph-mon[27933]: pgmap v326: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T07:25:11.669 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-22T07:25:11.669 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-22T07:25:11.669 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-22T07:25:11.669 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-22T07:25:11.670 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-22T07:25:11.670 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-22T07:25:11.670 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-22T07:25:11.670 DEBUG:tasks.check_counter:Getting stats from d 2024-04-22T07:25:11.670 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-22T07:25:11.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:11 smithi133 ceph-mon[31198]: pgmap v326: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T07:25:11.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:11 smithi154 ceph-mon[31862]: pgmap v326: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-22T07:25:11.907 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:25:13.529 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:13 smithi113 ceph-mon[27933]: pgmap v327: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 0, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 0, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 0, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.001628667, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.000395590, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:13.552 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:13.553 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 157, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 320, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 183840, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.013716069, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.008815455, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.002404629, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 154, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.011741948, 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000076246 2024-04-22T07:25:13.554 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 183840 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 221, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 208, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1591600, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 127072, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 2, 2024-04-22T07:25:13.555 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.045088889, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.013755845, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.036383783, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.004277117, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 208, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.011183796, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000053768 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.556 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1591600, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 127072 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:13.557 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:13.558 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:13.559 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 148984, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 30, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 0, 2024-04-22T07:25:13.560 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 928, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 2, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-22T07:25:13.561 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 219, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 1557962, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 219, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 219, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1557962, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:13.562 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:13.563 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:13.563 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:25:13.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:13 smithi133 ceph-mon[31198]: pgmap v327: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:13.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:13 smithi154 ceph-mon[31862]: pgmap v327: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:14.235 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-22T07:25:14.235 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-22T07:25:14.235 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-22T07:25:14.235 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-22T07:25:14.236 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-22T07:25:14.236 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-22T07:25:14.236 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-22T07:25:14.236 DEBUG:tasks.check_counter:Getting stats from g 2024-04-22T07:25:14.236 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-22T07:25:14.488 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:25:14.781 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:14 smithi113 ceph-mon[27933]: osdmap e103: 12 total, 12 up, 12 in 2024-04-22T07:25:14.781 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:14 smithi113 ceph-mon[27933]: pgmap v329: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:25:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:14 smithi133 ceph-mon[31198]: osdmap e103: 12 total, 12 up, 12 in 2024-04-22T07:25:14.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:14 smithi133 ceph-mon[31198]: pgmap v329: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:25:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:14 smithi154 ceph-mon[31862]: osdmap e103: 12 total, 12 up, 12 in 2024-04-22T07:25:14.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:14 smithi154 ceph-mon[31862]: pgmap v329: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-22T07:25:16.202 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 0, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 0, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 0, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.001192289, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.000231514, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:16.203 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 217, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 204, 2024-04-22T07:25:16.204 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1590512, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 124512, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 2, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.042195899, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.013763614, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.033846344, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.004188277, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 204, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.010946228, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000053657 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:16.205 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1590512, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 124512 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 2, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 155, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 320, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 183488, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:16.206 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.013971234, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.009416016, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.002031254, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 152, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.013724273, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000090291 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.207 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 183488 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:16.208 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:16.209 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:16.210 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 149017, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 31, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:16.211 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 928, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 2, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 215, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 1557442, 2024-04-22T07:25:16.212 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 215, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 215, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1557442, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:16.213 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:25:16.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:16 smithi113 ceph-mon[27933]: pgmap v330: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 222 B/s wr, 8 op/s 2024-04-22T07:25:16.915 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:16 smithi154 ceph-mon[31862]: pgmap v330: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 222 B/s wr, 8 op/s 2024-04-22T07:25:17.036 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-22T07:25:17.037 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-22T07:25:17.037 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-22T07:25:17.037 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-22T07:25:17.037 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-22T07:25:17.037 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-22T07:25:17.037 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-22T07:25:17.037 DEBUG:tasks.check_counter:Getting stats from j 2024-04-22T07:25:17.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-22T07:25:17.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:16 smithi133 ceph-mon[31198]: pgmap v330: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 222 B/s wr, 8 op/s 2024-04-22T07:25:17.277 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T07:25:18.654 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:18 smithi113 ceph-mon[27933]: pgmap v331: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:18.867 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-22T07:25:18.868 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:18.868 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 0, 2024-04-22T07:25:18.868 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 2, 2024-04-22T07:25:18.868 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 0, 2024-04-22T07:25:18.868 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 0, 2024-04-22T07:25:18.868 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 4, 2024-04-22T07:25:18.869 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:18.869 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.001072412, 2024-04-22T07:25:18.869 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T07:25:18.869 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.000270504, 2024-04-22T07:25:18.869 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:18.869 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:18.870 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:18.870 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:18.870 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:18.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.870 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:18.870 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 6, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 161, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 3360, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 184640, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:18.871 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.013421656, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.009029640, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.001818734, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 159, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.011365630, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000071481 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.872 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 184640 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_messages": 219, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages": 208, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_bytes": 1589104, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_bytes": 127904, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_created_connections": 2, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_total_time": 0.044140821, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_send_time": 0.013954558, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_recv_time": 0.034767741, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_running_fast_dispatch_time": 0.004513252, 2024-04-22T07:25:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 207, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.010541891, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000050927 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_recv_encrypted_bytes": 1589104, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "msgr_send_encrypted_bytes": 127904 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "cct": { 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "total_workers": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "unhealthy_workers": 0 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "mempool": { 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_items": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:18.877 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_bytes": 148985, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_anon_items": 30, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_bytes": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_items": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:18.878 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_bytes": 0, 2024-04-22T07:25:18.879 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_items": 0, 2024-04-22T07:25:18.879 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:18.879 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:18.879 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:18.879 INFO:teuthology.orchestra.run.smithi113.stdout: "pgmap_items": 0, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_bytes": 928, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_co_items": 2, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_1_items": 0, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "unittest_2_items": 0 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "val": 0, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "max": 104857600, 2024-04-22T07:25:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: "get_started": 0, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "get": 221, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "get_sum": 1558222, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "get_or_fail_success": 221, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "take": 0, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "take_sum": 0, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "put": 221, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "put_sum": 1558222, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "wait": { 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "avgcount": 0, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "sum": 0.000000000, 2024-04-22T07:25:18.884 INFO:teuthology.orchestra.run.smithi113.stdout: "avgtime": 0.000000000 2024-04-22T07:25:18.885 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:18.885 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-22T07:25:18.885 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-22T07:25:19.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:18 smithi133 ceph-mon[31198]: pgmap v331: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:19.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:18 smithi154 ceph-mon[31862]: pgmap v331: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:19.584 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-22T07:25:19.585 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-22T07:25:19.585 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-22T07:25:19.585 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-22T07:25:19.585 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-22T07:25:19.585 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-22T07:25:19.585 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-22T07:25:19.585 DEBUG:tasks.check_counter:Getting stats from b 2024-04-22T07:25:19.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-22T07:25:19.836 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:25:20.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:20 smithi113 ceph-mon[27933]: pgmap v332: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:20.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:20 smithi133 ceph-mon[31198]: pgmap v332: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:21.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:20 smithi154 ceph-mon[31862]: pgmap v332: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-22T07:25:21.648 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-22T07:25:21.648 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:21.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 456, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 456, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 4349477, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 144194, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.055324620, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.020706006, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.014613111, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.013537647, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 455, 2024-04-22T07:25:21.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.026172303, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000057521 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:21.650 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 453, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 546, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 112024, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 391466, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.047963954, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.013878991, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.012124227, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.017902454, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 544, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.028182786, 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000051806 2024-04-22T07:25:21.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 247664 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 655, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 644, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3586081, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 239225, 2024-04-22T07:25:21.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 18446744073709551611, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.062799423, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.019578010, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.023729827, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.015563588, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 643, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.022925595, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000035654 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.653 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 240176, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 77680 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2271, 2024-04-22T07:25:21.654 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.062915693, 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000027703 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-22T07:25:21.655 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-22T07:25:21.656 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 522, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 522, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 522, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-22T07:25:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-04-22T07:25:21.658 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T07:25:21.659 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-22T07:25:21.660 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-22T07:25:21.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 3432, 2024-04-22T07:25:21.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 5, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 11602306, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 11602194, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.663 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 3432 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 74, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 1871, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 1797, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 20, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 197, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 177, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 522, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 2159, 2024-04-22T07:25:21.664 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 1637, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 54372, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215416 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-22T07:25:21.665 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-22T07:25:21.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-22T07:25:21.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.672 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.675 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-22T07:25:21.676 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 6, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 4, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 5, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-22T07:25:21.677 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 3784, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:21.678 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:21.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 191913, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 120, 2024-04-22T07:25:21.680 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8832, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 197, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-22T07:25:21.681 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 599019, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 5376, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 1412, 2024-04-22T07:25:21.682 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 26, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 0, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 1412, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1412, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.841984216, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000596306 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1412, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1413 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 1412, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 1412, 2024-04-22T07:25:21.683 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 896, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 513, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-04-22T07:25:21.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 2, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-22T07:25:21.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-22T07:25:21.686 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 103, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 10, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 5, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 5, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-04-22T07:25:21.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-22T07:25:21.688 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 1560, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 7896639, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 1560, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:21.689 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1560, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 7896639, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:21.690 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 1412, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 7408012, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1412, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 7408012, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.691 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 1412, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 1412, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1412, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1412, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.692 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-22T07:25:21.693 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55782b470780": { 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:21.694 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:21.695 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-04-22T07:25:22.285 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T07:25:22.285 DEBUG:tasks.check_counter:Getting stats from e 2024-04-22T07:25:22.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-22T07:25:22.524 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:25:22.877 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:22 smithi133 ceph-mon[31198]: pgmap v333: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:25:23.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:22 smithi154 ceph-mon[31862]: pgmap v333: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:25:23.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:22 smithi113 ceph-mon[27933]: pgmap v333: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:25:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-22T07:25:24.153 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:24.153 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1599, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1142, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 6399448, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 489652, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.195382584, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.054658246, 2024-04-22T07:25:24.154 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.084821440, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.052993069, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1141, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.043574705, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000038189 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 681, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000120849, 2024-04-22T07:25:24.155 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000177 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1982320, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 234208 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 13537, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 8608, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 7112100, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 4300202, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 122, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 8, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.646568340, 2024-04-22T07:25:24.156 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.253146569, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.438126563, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.047425469, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8602, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.184188284, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000021412 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 20755, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.002445770, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000117 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.157 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 443984 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1845, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1716, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1792969, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 2662115, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.125983767, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.041522182, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.054464723, 2024-04-22T07:25:24.158 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.030289633, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1712, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.051192985, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000029902 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1949, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000318181, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000163 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T07:25:24.159 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2126, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.463866751, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000218187 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-22T07:25:24.160 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 529, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.047140736, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000089112 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 769, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 665, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 665, 2024-04-22T07:25:24.161 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.967020020, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001454165 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 104, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 12, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 8, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 9, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 1, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 268, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 7, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 24, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 237, 2024-04-22T07:25:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 257, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 1525, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 3, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 4, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 4, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 2912, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 1699, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 104, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 798, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-22T07:25:24.163 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 25, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 382, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 8, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 120, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 158, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 158, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 94, 2024-04-22T07:25:24.164 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 13, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 81, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 64, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 4, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 111, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 360, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 200, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 101, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 21, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 548 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.165 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 47, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 2195, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 1493, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 1344, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 62, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 8, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 279, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 271, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-22T07:25:24.166 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 13, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 25, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 1, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-22T07:25:24.167 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 932, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 3432, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 1, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 5, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 11602306, 2024-04-22T07:25:24.168 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 9513142, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 195, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.504852834, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002588988 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 2500 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 93, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 2478, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 2385, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 39, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 910, 2024-04-22T07:25:24.169 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 871, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 268, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 3251, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 2983, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 4, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 310, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 306, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 65156, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 248184 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 1173, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 68, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 769, 2024-04-22T07:25:24.170 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 11, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 419, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.005553751, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000555375 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 80, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.014056372, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000175704 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 240, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.076779126, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000319913 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.173 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-22T07:25:24.174 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.174 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.174 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.174 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.174 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-22T07:25:24.174 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001424137, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000178017 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.175 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.168764856, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.021095607 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 35, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.024134484, 2024-04-22T07:25:24.176 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000689556 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.177 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.131023852, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.013102385 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.023007277, 2024-04-22T07:25:24.178 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004601455 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.179 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.181 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 269, 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.522276165, 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001941547 2024-04-22T07:25:24.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.183 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 6, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 4, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 5, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 33, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 16, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 3512, 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-22T07:25:24.184 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:24.185 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:24.186 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:24.187 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 385933, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 207, 2024-04-22T07:25:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 88, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 1, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:24.189 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8832, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 197, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 520147, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 5059, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:24.190 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 1749, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 2256643, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 4, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 1749, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1749, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.227958125, 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001273846 2024-04-22T07:25:24.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1749, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1842 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 1749, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 1153, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 596, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 564, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-04-22T07:25:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 394, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 227, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 29, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 2, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 1, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 300, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 193, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-22T07:25:24.193 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-22T07:25:24.194 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 103, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 17, 2024-04-22T07:25:24.195 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 67, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 44, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 21 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-22T07:25:24.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 20, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 290, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 271, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 16977, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 13732858, 2024-04-22T07:25:24.197 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 16977, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 16977, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 13732858, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:24.198 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 1749, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 7621493, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1749, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 7621493, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.199 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 1749, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 1749, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1749, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1749, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.200 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 271, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 29723, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 271, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 32, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 29723, 2024-04-22T07:25:24.201 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55fbe4b446e0": { 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 932, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 2072619, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:24.202 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 932, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 195, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 2072619, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:24.203 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-22T07:25:24.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:24 smithi133 ceph-mon[31198]: pgmap v334: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:25:24.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:24 smithi113 ceph-mon[27933]: pgmap v334: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds.exported=25 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds.imported=8 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=47 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=2195 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=8 2024-04-22T07:25:24.942 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=5 2024-04-22T07:25:24.942 DEBUG:tasks.check_counter:Getting stats from h 2024-04-22T07:25:24.943 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-22T07:25:25.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:24 smithi154 ceph-mon[31862]: pgmap v334: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-22T07:25:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:25:26.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:26 smithi154 ceph-mon[31862]: pgmap v335: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:25:26.859 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:26 smithi133 ceph-mon[31198]: pgmap v335: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:25:26.859 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-22T07:25:26.859 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 11254, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 16673, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 27999312, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 24428631, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 1.069814847, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.448191365, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.386803628, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.228157848, 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:26.860 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 16671, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.602607952, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000036147 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 18046, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.003505230, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000194 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.861 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 33426, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 66113, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 32334263, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 56968193, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 36, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 7, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 2.156805346, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.979453648, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 1.755402269, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.184135673, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 66113, 2024-04-22T07:25:26.862 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.633552382, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000024708 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 84329, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.011088211, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000131 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 433200, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 102304 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 3798, 2024-04-22T07:25:26.863 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 3930, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 13548649, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 31892929, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.531864299, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.205314480, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.094752084, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.206235124, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3928, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.317970207, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000080949 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.864 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 26, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000007725, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000297 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1104, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 299088 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-22T07:25:26.865 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3404, 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 14.962063557, 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004395435 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11891, 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.787268535, 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000234401 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.866 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 17487, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 17426, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 17426, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 34.933968580, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002004703 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 60, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 11, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 2, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 64, 2024-04-22T07:25:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 7, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 1, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 13774, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 114, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 8420, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 5240, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 13609, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 402, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 24, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 20829, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 20026, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 4, 2024-04-22T07:25:26.868 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 20, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 35, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 356, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 55, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 768, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 1376, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-22T07:25:26.869 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 1376, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 89, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 89, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 1287, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 102, 2024-04-22T07:25:26.870 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 1202, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 14072, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 6385, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 5227, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 73, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 10141 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.871 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 1276, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 64, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 42, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 45, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 2518, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 339, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 8335, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 7996, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-22T07:25:26.872 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 35, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 8, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-22T07:25:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 27250, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 51569, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 37, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 69, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 6, 2024-04-22T07:25:26.874 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 138434446, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 62936586, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1071, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 63.267570892, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.059073362 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 24319 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 6945, 2024-04-22T07:25:26.875 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 18092, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 11147, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 435, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 789, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 354, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 13774, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 26057, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 12283, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 8689, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 8689, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 119952, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 239992 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.876 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 21372, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 608, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 17487, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 46, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 1450, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5629, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 8.481122464, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001506683 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 27, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.034792604, 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001288614 2024-04-22T07:25:26.877 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2490, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.090736308, 2024-04-22T07:25:26.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000438046 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.879 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.880 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 307, 2024-04-22T07:25:26.881 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.238803791, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000777862 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-22T07:25:26.882 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.003894800, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003894800 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 635, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.500350987, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000787954 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.883 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 587, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 10.750971072, 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.018315112 2024-04-22T07:25:26.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-04-22T07:25:26.885 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.102920068, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.051460034 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.886 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.887 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7748, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 13.730376486, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001772118 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.888 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 4, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 4, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 13178, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 6589, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 3005, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:26.889 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-22T07:25:26.891 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 2947152, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 7941, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 792, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 9, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8832, 2024-04-22T07:25:26.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 197, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 27124165, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 379713, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-22T07:25:26.893 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 11465, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 78196857, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 0, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 11465, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11465, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 30.581424661, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002667372 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11465, 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 11637 2024-04-22T07:25:26.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 11465, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 2021, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 9444, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 320, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 309, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 1207, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 54, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 2, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 7, 2024-04-22T07:25:26.895 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 8065, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1390, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-22T07:25:26.896 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-22T07:25:26.897 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 103, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 12, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 210, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 30, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 43 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-22T07:25:26.898 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 18, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 8033, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 7996, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 48474, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 69506139, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 48474, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 48474, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 69506139, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 11465, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 137806391, 2024-04-22T07:25:26.901 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 11465, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 137806391, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-22T07:25:26.902 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 11465, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 11465, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 11465, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 11465, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-22T07:25:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 7996, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 857612, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 7996, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 136, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 857612, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:26.904 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x56464a600780": { 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 27250, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 75494727, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 27250, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1071, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 75494727, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:26.905 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:26.906 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:26.906 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.906 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:26.906 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-22T07:25:27.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:26 smithi113 ceph-mon[27933]: pgmap v335: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T07:25:27.624 INFO:tasks.check_counter:Daemon mds.h mds.exported=35 2024-04-22T07:25:27.625 INFO:tasks.check_counter:Daemon mds.h mds.imported=55 2024-04-22T07:25:27.625 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1276 2024-04-22T07:25:27.625 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=64 2024-04-22T07:25:27.625 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-22T07:25:27.625 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-22T07:25:27.625 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T07:25:27.625 DEBUG:tasks.check_counter:Getting stats from k 2024-04-22T07:25:27.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-22T07:25:27.862 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T07:25:28.979 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:28 smithi133 ceph-mon[31198]: pgmap v336: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:29.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:28 smithi154 ceph-mon[31862]: pgmap v336: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:29.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:28 smithi113 ceph-mon[27933]: pgmap v336: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2024-04-22T07:25:29.527 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.001044061, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000246871, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.528 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 226, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 213, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1592960, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 130272, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.047259173, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.013819655, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.038201528, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.005390128, 2024-04-22T07:25:29.529 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 213, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.012769033, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000059948 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1592960, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 130272 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:29.530 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 162, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 184720, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.010484377, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.006768629, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.001514943, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 159, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.018249532, 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000114776 2024-04-22T07:25:29.531 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 184720 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.532 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:29.533 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:29.534 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148995, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 31, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-22T07:25:29.535 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:29.536 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 224, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1558612, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 224, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-22T07:25:29.537 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 224, 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1558612, 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-22T07:25:29.538 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-22T07:25:30.200 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-22T07:25:30.200 DEBUG:tasks.check_counter:Getting stats from c 2024-04-22T07:25:30.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-22T07:25:30.460 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:25:30.925 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:30 smithi154 ceph-mon[31862]: pgmap v337: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:31.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:30 smithi133 ceph-mon[31198]: pgmap v337: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:31.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:30 smithi113 ceph-mon[27933]: pgmap v337: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 954, 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 956, 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 59026493, 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 303379, 2024-04-22T07:25:32.109 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 8, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.141581336, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.031239237, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.064486878, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.035352529, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 954, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.041843003, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000043860 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.110 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 471, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 565, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 37878174, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 402210, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 6, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.082408132, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.019483991, 2024-04-22T07:25:32.111 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.041417912, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.015493703, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 563, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.027350039, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000048579 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 252688 2024-04-22T07:25:32.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 251, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 239, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 38017598, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 110568, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.048040638, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.010600960, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.033517821, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.005239209, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 239, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.012058826, 2024-04-22T07:25:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000050455 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 242528, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 78768 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.114 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2354, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.074829518, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000031788 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.115 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-04-22T07:25:32.116 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 22955, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 7, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 22948, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 22945, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 1, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-04-22T07:25:32.117 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-22T07:25:32.118 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T07:25:32.119 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-04-22T07:25:32.120 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-22T07:25:32.121 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 51569, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 69, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 6, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-04-22T07:25:32.122 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 138434446, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 138431885, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 51569 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 6944, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 18068, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 11124, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 424, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 763, 2024-04-22T07:25:32.123 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 339, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 22955, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 26021, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 3066, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 96872, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215416 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-04-22T07:25:32.124 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.125 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-22T07:25:32.126 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.127 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-22T07:25:32.128 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-22T07:25:32.129 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.130 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.132 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.134 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 2, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 4, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 2, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 4, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-22T07:25:32.135 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 3065, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:32.136 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:32.137 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 506949, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 7564, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:32.138 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 8832, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 197, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 33294404, 2024-04-22T07:25:32.139 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 391948, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 1524, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 21, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 0, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 1524, 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-22T07:25:32.140 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1524, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.201968267, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000788693 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1524, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1525 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 1524, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1524, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 991, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-04-22T07:25:32.141 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 531, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 1, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-22T07:25:32.142 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-22T07:25:32.143 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 103, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 10, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 10, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 10, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-04-22T07:25:32.144 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-22T07:25:32.145 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1672, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 134761242, 2024-04-22T07:25:32.146 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1672, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1672, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 134761242, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:32.147 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1524, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 134247487, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1524, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 134247487, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:32.148 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1524, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 1524, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1524, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1524, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-22T07:25:32.149 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.150 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55a2ea7d85a0": { 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-22T07:25:32.151 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:32.152 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:32.152 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:32.152 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:32.152 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.152 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:32.152 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-22T07:25:32.816 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:32 smithi154 ceph-mon[31862]: pgmap v338: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-22T07:25:32.865 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T07:25:32.865 DEBUG:tasks.check_counter:Getting stats from f 2024-04-22T07:25:32.866 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-22T07:25:32.886 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:32 smithi113 ceph-mon[27933]: pgmap v338: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:33.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:32 smithi133 ceph-mon[31198]: pgmap v338: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:33.111 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:25:34.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:34 smithi154 ceph-mon[31862]: pgmap v339: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:34.882 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-22T07:25:34.882 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:34.882 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 626, 2024-04-22T07:25:34.882 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 614, 2024-04-22T07:25:34.882 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1397729, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 229982, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.071826430, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.024117603, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.026653234, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.018648251, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 614, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.027843692, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000045348 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.883 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 243136, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 79568 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 4, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 99, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1508, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 255445, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.006510671, 2024-04-22T07:25:34.884 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.003895971, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001635551, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000055335, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 97, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.006321629, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000065171 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-22T07:25:34.885 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 254480 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 939, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 941, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 4467645, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 298660, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 4, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.107175179, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.029827485, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.030787953, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.037815740, 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:34.886 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 939, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.039172534, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000041717 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-22T07:25:34.887 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2275, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.075942926, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000033381 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-22T07:25:34.888 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 0, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 0, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 688, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 688, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 0, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 688, 2024-04-22T07:25:34.890 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 1, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 0, 2024-04-22T07:25:34.891 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T07:25:34.892 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 0, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 0 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-22T07:25:34.893 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-22T07:25:34.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-22T07:25:34.895 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 2494, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 4, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 9420027, 2024-04-22T07:25:34.896 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 9417466, 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 2494 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 78, 2024-04-22T07:25:34.897 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 961, 2024-04-22T07:25:34.898 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 883, 2024-04-22T07:25:34.898 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 10, 2024-04-22T07:25:34.898 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 135, 2024-04-22T07:25:34.898 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 125, 2024-04-22T07:25:34.898 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 688, 2024-04-22T07:25:34.898 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 1377, 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 689, 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 0, 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 0, 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 54172, 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 215416 2024-04-22T07:25:34.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 0, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 0, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 0, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 0, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 0, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-22T07:25:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.905 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.906 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.907 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.908 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.909 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-22T07:25:34.910 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 3, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 2, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 3, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 0, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 0, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 4376, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.912 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:34.913 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:34.914 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 200390, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 135, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:34.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 8832, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 197, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 692266, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 6262, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-22T07:25:34.916 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 1415, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 21, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 0, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 1415, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1415, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.213967976, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000857927 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1415, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1416 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 1415, 2024-04-22T07:25:34.917 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1415, 2024-04-22T07:25:34.918 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 0, 2024-04-22T07:25:34.918 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-22T07:25:34.918 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-22T07:25:34.918 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 920, 2024-04-22T07:25:34.921 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-04-22T07:25:34.921 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 493, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-22T07:25:34.922 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 1, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-22T07:25:34.923 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 103, 2024-04-22T07:25:34.924 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 10, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 6, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 6, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 2, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 0 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.925 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:34.926 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1565, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 5715385, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1565, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1565, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 5715385, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.927 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1415, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 5225728, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1415, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 5225728, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.928 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1415, 2024-04-22T07:25:34.929 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 1415, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1415, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1415, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:34.930 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55bd413845a0": { 2024-04-22T07:25:34.931 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:34.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.933 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:34.933 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-22T07:25:35.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:34 smithi133 ceph-mon[31198]: pgmap v339: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:35.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:34 smithi113 ceph-mon[27933]: pgmap v339: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-22T07:25:35.573 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T07:25:35.573 DEBUG:tasks.check_counter:Getting stats from i 2024-04-22T07:25:35.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-22T07:25:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:25:36.870 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:36 smithi154 ceph-mon[31862]: pgmap v340: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:37.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:36 smithi133 ceph-mon[31198]: pgmap v340: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:37.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:36 smithi113 ceph-mon[27933]: pgmap v340: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:37.393 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-22T07:25:37.393 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:37.393 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1847, 2024-04-22T07:25:37.393 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2993, 2024-04-22T07:25:37.393 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 2513093, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 4947099, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 38, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 7, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.139461062, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.055285915, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.068433919, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.015545038, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2989, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.071040680, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000023767 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:37.394 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2940, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000449439, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000152 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 1108, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 1362, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 395000, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 908758, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 8, 2024-04-22T07:25:37.395 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.090195016, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.040175283, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.042865107, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.008418552, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1362, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.047399569, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000034801 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1936, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000334927, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000172 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.396 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 297760 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 12404, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 7127, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 7209992, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 1925958, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 8, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.517073996, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.203457054, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.407110579, 2024-04-22T07:25:37.397 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.018653156, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 7127, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.130051069, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000018247 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 18201, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.002147021, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000117 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 434352, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 104368 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.398 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1112, 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.486586836, 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000437578 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-04-22T07:25:37.399 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 730, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.072561569, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000099399 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 844, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 729, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 729, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 4.372848290, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005998420 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.400 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 115, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 11, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 9, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 9, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 1, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 288, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 7, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 10, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 271, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 274, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 1533, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 4, 2024-04-22T07:25:37.401 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 3033, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 1742, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 113, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 793, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 1, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 32, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 404, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 29, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 254, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 44, 2024-04-22T07:25:37.402 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 4, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 234, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 44, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 4, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 4, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 40, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 4, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 202, 2024-04-22T07:25:37.403 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 501, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 332, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 149, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 24, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 727 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 75, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 2108, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 1434, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 1300, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 109, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 7, 2024-04-22T07:25:37.404 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 396, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 389, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 32, 2024-04-22T07:25:37.405 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 1, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 1333, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 2494, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 1, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-04-22T07:25:37.406 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 4, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 2, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 9420027, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 6173649, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 181, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.485488464, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002682256 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 1161 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.407 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 102, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 1528, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 1426, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 35, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 811, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 776, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 288, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 2431, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 2143, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 425, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 425, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 60528, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 239992 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.408 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 1345, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 68, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 844, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 6, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 486, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.001015591, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001015591 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.409 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 281, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 3.307712679, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.011771219 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.410 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-04-22T07:25:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-04-22T07:25:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.413 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 51, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.037706639, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000739345 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.414 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.415 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 20, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.315300162, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.015765008 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-04-22T07:25:37.416 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.417 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.418 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 376, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.711113219, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001891258 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-04-22T07:25:37.419 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 3, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 2, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 3, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 608, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 608, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 4143, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:37.420 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:37.421 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 332852, 2024-04-22T07:25:37.422 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 195, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 8832, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 197, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 535559, 2024-04-22T07:25:37.423 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 5328, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 1166, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 3516845, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 0, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 1166, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1166, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.915949363, 2024-04-22T07:25:37.424 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001643181 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1166, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1267 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 1166, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 483, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 683, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 236, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 0, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 180, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 217, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 23, 2024-04-22T07:25:37.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 2, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 1, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 400, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 65, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-04-22T07:25:37.426 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-04-22T07:25:37.427 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 103, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 12, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 77, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 44, 2024-04-22T07:25:37.428 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 22 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 0, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "osd-slow-ops": { 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 0 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 10, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-04-22T07:25:37.429 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 10, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 389, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 389, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 15355, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 8722757, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 15355, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:37.430 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 15355, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 8722757, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:37.431 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1166, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 5538273, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1166, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 5538273, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-04-22T07:25:37.432 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 1166, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 1166, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1166, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1166, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-22T07:25:37.433 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 389, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 41753, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 389, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 36, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 41753, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x558a1bc1c780": { 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1333, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 3245804, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1333, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 181, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 3245804, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:37.435 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:37.436 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.436 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:37.436 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-22T07:25:38.156 INFO:tasks.check_counter:Daemon mds.i mds.exported=32 2024-04-22T07:25:38.156 INFO:tasks.check_counter:Daemon mds.i mds.imported=29 2024-04-22T07:25:38.156 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=75 2024-04-22T07:25:38.156 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2108 2024-04-22T07:25:38.156 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-22T07:25:38.156 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-22T07:25:38.157 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T07:25:38.157 DEBUG:tasks.check_counter:Getting stats from l 2024-04-22T07:25:38.157 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697 shell --fsid 81035712-0076-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-22T07:25:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T07:25:38.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:38 smithi154 ceph-mon[31862]: pgmap v341: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:39.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:38 smithi133 ceph-mon[31198]: pgmap v341: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:39.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:38 smithi113 ceph-mon[27933]: pgmap v341: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:40.188 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-04-22T07:25:40.188 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T07:25:40.188 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 0, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 2, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 0, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 0, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 4, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 0, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.000936771, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.000166749, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:40.189 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:40.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.190 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:40.190 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:40.190 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:40.190 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:40.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 227, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 216, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 1591344, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 132544, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 2, 2024-04-22T07:25:40.191 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 1, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.046045941, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.013892471, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.035819367, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.005422599, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 215, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.012185500, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000056676 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:40.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1591344, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 132544 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 6, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 166, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 3360, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 185520, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 5, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 3, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 0.011352787, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 0.007386323, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 0.001694436, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T07:25:40.193 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 164, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.014592189, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000088976 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 185520 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-04-22T07:25:40.194 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T07:25:40.195 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-04-22T07:25:40.196 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 148967, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 29, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-04-22T07:25:40.197 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 928, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 2, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-04-22T07:25:40.198 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 229, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 1559326, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 229, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 229, 2024-04-22T07:25:40.199 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 1559326, 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-04-22T07:25:40.200 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-04-22T07:25:40.936 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:40 smithi154 ceph-mon[31862]: pgmap v342: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:40.964 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-22T07:25:40.964 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-22T07:25:40.964 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-22T07:25:40.965 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-22T07:25:40.965 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-22T07:25:40.965 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-22T07:25:40.965 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-22T07:25:40.965 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-22T07:25:41.173 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-22T07:25:41.173 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:25:41.173 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:25:41.175 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:40 smithi113 ceph-mon[27933]: pgmap v342: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:41.175 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:40 smithi133 ceph-mon[31198]: pgmap v342: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:41.196 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:25:41.196 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:25:41.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:25:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 -- 172.21.15.113:0/1114589046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 msgr2=0x7f4b7810dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/1114589046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b7810dd00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009cd0 tx=0x7f4b6802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 -- 172.21.15.113:0/1114589046 shutdown_connections 2024-04-22T07:25:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/1114589046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b7810dd00 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/1114589046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b781011a0 0x7f4b781015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/1114589046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b78105ed0 0x7f4b781062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 -- 172.21.15.113:0/1114589046 >> 172.21.15.113:0/1114589046 conn(0x7f4b78077e30 msgr2=0x7f4b78078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 -- 172.21.15.113:0/1114589046 shutdown_connections 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.412+0000 7f4b7f34b640 1 -- 172.21.15.113:0/1114589046 wait complete. 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 Processor -- start 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 -- start start 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b781011a0 0x7f4b781995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b78199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b78105ed0 0x7f4b781a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78111200 con 0x7f4b78101b20 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78111080 con 0x7f4b78105ed0 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7f34b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78111380 con 0x7f4b781011a0 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7c8e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b78199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7c8e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b78199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55014/0 (socket says 172.21.15.113:55014) 2024-04-22T07:25:41.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7c8e6640 1 -- 172.21.15.113:0/3067774618 learned_addr learned my addr 172.21.15.113:0/3067774618 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7d8e8640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b78105ed0 0x7f4b781a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.413+0000 7f4b7d0e7640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b781011a0 0x7f4b781995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7c8e6640 1 -- 172.21.15.113:0/3067774618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b781011a0 msgr2=0x7f4b781995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7c8e6640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b781011a0 0x7f4b781995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7c8e6640 1 -- 172.21.15.113:0/3067774618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b78105ed0 msgr2=0x7f4b781a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7c8e6640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b78105ed0 0x7f4b781a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7c8e6640 1 -- 172.21.15.113:0/3067774618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b781a4b80 con 0x7f4b78101b20 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7c8e6640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b78199b00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00d770 tx=0x7f4b6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c017070 con 0x7f4b78101b20 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4b6c0054f0 con 0x7f4b78101b20 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b781a4d50 con 0x7f4b78101b20 2024-04-22T07:25:41.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c004de0 con 0x7f4b78101b20 2024-04-22T07:25:41.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.414+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b781a51f0 con 0x7f4b78101b20 2024-04-22T07:25:41.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.415+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b781025c0 con 0x7f4b78101b20 2024-04-22T07:25:41.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.416+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f4b6c004f80 con 0x7f4b78101b20 2024-04-22T07:25:41.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.416+0000 7f4b767fc640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4b600a3d20 0x7f4b600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.417+0000 7f4b7d0e7640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4b600a3d20 0x7f4b600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.417+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7f4b6c0d08d0 con 0x7f4b78101b20 2024-04-22T07:25:41.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.417+0000 7f4b7d0e7640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4b600a3d20 0x7f4b600a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b70005e00 tx=0x7f4b70005c50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:41.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.420+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b6c09cdd0 con 0x7f4b78101b20 2024-04-22T07:25:41.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.601+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b78069230 con 0x7f4b78101b20 2024-04-22T07:25:41.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.603+0000 7f4b767fc640 1 -- 172.21.15.113:0/3067774618 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23827 (secure 0 0 0) 0x7f4b78069230 con 0x7f4b78101b20 2024-04-22T07:25:41.600 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:25:13.493086+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1615298521":"2024-04-23T07:17:49.535084+0000","172.21.15.154:6836/1615298521":"2024-04-23T07:17:49.535084+0000","172.21.15.154:6834/217490889":"2024-04-23T07:17:49.535032+0000","172.21.15.133:6838/982201245":"2024-04-23T07:17:48.307009+0000","172.21.15.154:6839/3404626603":"2024-04-23T07:15:51.118540+0000","172.21.15.154:6838/3404626603":"2024-04-23T07:15:51.118540+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.154:6835/217490889":"2024-04-23T07:17:49.535032+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.133:6839/982201245":"2024-04-23T07:17:48.307009+0000","172.21.15.133:6837/65907218":"2024-04-23T07:15:51.098366+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.133:6836/65907218":"2024-04-23T07:15:51.098366+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4b600a3d20 msgr2=0x7f4b600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4b600a3d20 0x7f4b600a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b70005e00 tx=0x7f4b70005c50 comp rx=0 tx=0).stop 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 msgr2=0x7f4b78199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b78199b00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00d770 tx=0x7f4b6c00dc30 comp rx=0 tx=0).stop 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 shutdown_connections 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f4b600a3d20 0x7f4b600a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b78105ed0 0x7f4b781a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b78101b20 0x7f4b78199b00 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 --2- 172.21.15.113:0/3067774618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b781011a0 0x7f4b781995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 >> 172.21.15.113:0/3067774618 conn(0x7f4b78077e30 msgr2=0x7f4b780fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:41.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 shutdown_connections 2024-04-22T07:25:41.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.605+0000 7f4b7f34b640 1 -- 172.21.15.113:0/3067774618 wait complete. 2024-04-22T07:25:41.613 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:08:19.077214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:12:52.335272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:12:53.444188+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:25:41.614 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 -- 172.21.15.113:0/1261747302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06401011a0 msgr2=0x7f0640101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 --2- 172.21.15.113:0/1261747302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06401011a0 0x7f0640101600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0630009930 tx=0x7f063002bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 -- 172.21.15.113:0/1261747302 shutdown_connections 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 --2- 172.21.15.113:0/1261747302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0640101b40 0x7f064010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 --2- 172.21.15.113:0/1261747302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06401011a0 0x7f0640101600 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 --2- 172.21.15.113:0/1261747302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0640107030 0x7f0640107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 -- 172.21.15.113:0/1261747302 >> 172.21.15.113:0/1261747302 conn(0x7f0640076c50 msgr2=0x7f0640077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 -- 172.21.15.113:0/1261747302 shutdown_connections 2024-04-22T07:25:41.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 -- 172.21.15.113:0/1261747302 wait complete. 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.718+0000 7f0645c43640 1 Processor -- start 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 -- start start 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06401011a0 0x7f064019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 0x7f064019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0640107030 0x7f06401986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0640112380 con 0x7f0640101b40 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0640112200 con 0x7f0640107030 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f0645c43640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0640112500 con 0x7f06401011a0 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 0x7f064019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 0x7f064019eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55030/0 (socket says 172.21.15.113:55030) 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 -- 172.21.15.113:0/2411177386 learned_addr learned my addr 172.21.15.113:0/2411177386 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063f7fe640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06401011a0 0x7f064019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063ffff640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0640107030 0x7f06401986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 -- 172.21.15.113:0/2411177386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06401011a0 msgr2=0x7f064019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06401011a0 0x7f064019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 -- 172.21.15.113:0/2411177386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0640107030 msgr2=0x7f06401986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0640107030 0x7f06401986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 -- 172.21.15.113:0/2411177386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0640198f80 con 0x7f0640101b40 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063f7fe640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06401011a0 0x7f064019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.719+0000 7f063effd640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 0x7f064019eaf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0630005ec0 tx=0x7f063002cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.720+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630041070 con 0x7f0640101b40 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.720+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0640199210 con 0x7f0640101b40 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.720+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f06300052f0 con 0x7f0640101b40 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.720+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630003e20 con 0x7f0640101b40 2024-04-22T07:25:41.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.720+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06401a5530 con 0x7f0640101b40 2024-04-22T07:25:41.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.721+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0640069da0 con 0x7f0640101b40 2024-04-22T07:25:41.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.722+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f0630044070 con 0x7f0640101b40 2024-04-22T07:25:41.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.722+0000 7f063cff9640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f06280a3d20 0x7f06280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:41.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.722+0000 7f063f7fe640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f06280a3d20 0x7f06280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:41.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.722+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7f06300f23c0 con 0x7f0640101b40 2024-04-22T07:25:41.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.723+0000 7f063f7fe640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f06280a3d20 0x7f06280a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06340058c0 tx=0x7f0634005850 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:41.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.727+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06300be9d0 con 0x7f0640101b40 2024-04-22T07:25:41.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.906+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06400691c0 con 0x7f0640101b40 2024-04-22T07:25:41.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.906+0000 7f063cff9640 1 -- 172.21.15.113:0/2411177386 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f06300be2d0 con 0x7f0640101b40 2024-04-22T07:25:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-04-22T07:25:41.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.908+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f06280a3d20 msgr2=0x7f06280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.908+0000 7f0645c43640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f06280a3d20 0x7f06280a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06340058c0 tx=0x7f0634005850 comp rx=0 tx=0).stop 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 msgr2=0x7f064019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 0x7f064019eaf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0630005ec0 tx=0x7f063002cdf0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 shutdown_connections 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f06280a3d20 0x7f06280a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0640107030 0x7f06401986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0640101b40 0x7f064019eaf0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 --2- 172.21.15.113:0/2411177386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06401011a0 0x7f064019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 >> 172.21.15.113:0/2411177386 conn(0x7f0640076c50 msgr2=0x7f06400ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 shutdown_connections 2024-04-22T07:25:41.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:41.909+0000 7f0645c43640 1 -- 172.21.15.113:0/2411177386 wait complete. 2024-04-22T07:25:41.916 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:41 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/3067774618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:41.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:25:42.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 -- 172.21.15.113:0/1179757398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f81011a0 msgr2=0x7f79f8101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 --2- 172.21.15.113:0/1179757398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f81011a0 0x7f79f8101600 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009930 tx=0x7f79e802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 -- 172.21.15.113:0/1179757398 shutdown_connections 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 --2- 172.21.15.113:0/1179757398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f8101b40 0x7f79f810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 --2- 172.21.15.113:0/1179757398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f81011a0 0x7f79f8101600 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 --2- 172.21.15.113:0/1179757398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79f8107030 0x7f79f8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 -- 172.21.15.113:0/1179757398 >> 172.21.15.113:0/1179757398 conn(0x7f79f8076c50 msgr2=0x7f79f8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 -- 172.21.15.113:0/1179757398 shutdown_connections 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 -- 172.21.15.113:0/1179757398 wait complete. 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.020+0000 7f79fd693640 1 Processor -- start 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 -- start start 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f81011a0 0x7f79f819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 0x7f79f819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79f8107030 0x7f79f81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f79f8111260 con 0x7f79f8101b40 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81110e0 con 0x7f79f8107030 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f79f81113e0 con 0x7f79f81011a0 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 0x7f79f819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 0x7f79f819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55054/0 (socket says 172.21.15.113:55054) 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 -- 172.21.15.113:0/631643914 learned_addr learned my addr 172.21.15.113:0/631643914 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f77fe640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79f8107030 0x7f79f81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f6ffd640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f81011a0 0x7f79f819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 -- 172.21.15.113:0/631643914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f81011a0 msgr2=0x7f79f819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f81011a0 0x7f79f819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 -- 172.21.15.113:0/631643914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79f8107030 msgr2=0x7f79f81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79f8107030 0x7f79f81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 -- 172.21.15.113:0/631643914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f8198fa0 con 0x7f79f8101b40 2024-04-22T07:25:42.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f6ffd640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f81011a0 0x7f79f819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:25:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79f67fc640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 0x7f79f819eb60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005ec0 tx=0x7f79e802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8041070 con 0x7f79f8101b40 2024-04-22T07:25:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f79e80052f0 con 0x7f79f8101b40 2024-04-22T07:25:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f8199230 con 0x7f79f8101b40 2024-04-22T07:25:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.021+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8003e20 con 0x7f79f8101b40 2024-04-22T07:25:42.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.022+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f81a55a0 con 0x7f79f8101b40 2024-04-22T07:25:42.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.023+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f81995a0 con 0x7f79f8101b40 2024-04-22T07:25:42.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.023+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f79e8044070 con 0x7f79f8101b40 2024-04-22T07:25:42.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.024+0000 7f79dffff640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f79e00a3d20 0x7f79e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.024+0000 7f79f6ffd640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f79e00a3d20 0x7f79e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.024+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7f79e80f2470 con 0x7f79f8101b40 2024-04-22T07:25:42.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.024+0000 7f79f6ffd640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f79e00a3d20 0x7f79e00a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009ab0 tx=0x7f79ec0094b0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.028+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79e80bea80 con 0x7f79f8101b40 2024-04-22T07:25:42.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:41 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/3067774618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:42.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:41 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/3067774618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:42.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.205+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79f8199c10 con 0x7f79f8101b40 2024-04-22T07:25:42.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.205+0000 7f79dffff640 1 -- 172.21.15.113:0/631643914 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f79e80be380 con 0x7f79f8101b40 2024-04-22T07:25:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-04-22T07:25:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f79e00a3d20 msgr2=0x7f79e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f79e00a3d20 0x7f79e00a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009ab0 tx=0x7f79ec0094b0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 msgr2=0x7f79f819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 0x7f79f819eb60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005ec0 tx=0x7f79e802cdf0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 shutdown_connections 2024-04-22T07:25:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f79e00a3d20 0x7f79e00a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79f8107030 0x7f79f81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f8101b40 0x7f79f819eb60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 --2- 172.21.15.113:0/631643914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79f81011a0 0x7f79f819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.207+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 >> 172.21.15.113:0/631643914 conn(0x7f79f8076c50 msgr2=0x7f79f8100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:42.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.208+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 shutdown_connections 2024-04-22T07:25:42.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.208+0000 7f79fd693640 1 -- 172.21.15.113:0/631643914 wait complete. 2024-04-22T07:25:42.216 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:25:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2244721879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 msgr2=0x7fae78107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2244721879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78107310 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fae68009930 tx=0x7fae6802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2244721879 shutdown_connections 2024-04-22T07:25:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2244721879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae78100e10 0x7fae7810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2244721879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae78100450 0x7fae781008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2244721879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78107310 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2244721879 >> 172.21.15.113:0/2244721879 conn(0x7fae78077410 msgr2=0x7fae780fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2244721879 shutdown_connections 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.319+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2244721879 wait complete. 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 Processor -- start 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 -- start start 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae78100450 0x7fae7819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae78100e10 0x7fae7819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fae78112450 con 0x7fae78106f30 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae781122d0 con 0x7fae78100e10 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7c9b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fae781125d0 con 0x7fae78100450 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae76d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae76d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55070/0 (socket says 172.21.15.113:55070) 2024-04-22T07:25:42.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae76d9d640 1 -- 172.21.15.113:0/2333033668 learned_addr learned my addr 172.21.15.113:0/2333033668 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae6ffff640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae78100e10 0x7fae7819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.320+0000 7fae7659c640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae78100450 0x7fae7819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae76d9d640 1 -- 172.21.15.113:0/2333033668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae78100450 msgr2=0x7fae7819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae76d9d640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae78100450 0x7fae7819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae76d9d640 1 -- 172.21.15.113:0/2333033668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae78100e10 msgr2=0x7fae7819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae76d9d640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae78100e10 0x7fae7819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae76d9d640 1 -- 172.21.15.113:0/2333033668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae78199030 con 0x7fae78106f30 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae76d9d640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78198770 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b970 tx=0x7fae7000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70019070 con 0x7fae78106f30 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae78199320 con 0x7fae78106f30 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fae7000cb40 con 0x7fae78106f30 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae781a5630 con 0x7fae78106f30 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.321+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70007670 con 0x7fae78106f30 2024-04-22T07:25:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.322+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae78103770 con 0x7fae78106f30 2024-04-22T07:25:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.323+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fae7001c070 con 0x7fae78106f30 2024-04-22T07:25:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.324+0000 7fae6f7fe640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fae640a3d20 0x7fae640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.324+0000 7fae7659c640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fae640a3d20 0x7fae640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.324+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7fae700d0290 con 0x7fae78106f30 2024-04-22T07:25:42.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.324+0000 7fae7659c640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fae640a3d20 0x7fae640a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae68009900 tx=0x7fae68000bc0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:42.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.327+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae7009c250 con 0x7fae78106f30 2024-04-22T07:25:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.504+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae7810a410 con 0x7fae78106f30 2024-04-22T07:25:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.504+0000 7fae6f7fe640 1 -- 172.21.15.113:0/2333033668 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7fae700a0550 con 0x7fae78106f30 2024-04-22T07:25:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fae640a3d20 msgr2=0x7fae640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fae640a3d20 0x7fae640a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae68009900 tx=0x7fae68000bc0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 msgr2=0x7fae78198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78198770 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b970 tx=0x7fae7000be30 comp rx=0 tx=0).stop 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 shutdown_connections 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fae640a3d20 0x7fae640a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae78106f30 0x7fae78198770 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae78100e10 0x7fae7819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 --2- 172.21.15.113:0/2333033668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae78100450 0x7fae7819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 >> 172.21.15.113:0/2333033668 conn(0x7fae78077410 msgr2=0x7fae78105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 shutdown_connections 2024-04-22T07:25:42.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.506+0000 7fae7c9b6640 1 -- 172.21.15.113:0/2333033668 wait complete. 2024-04-22T07:25:42.514 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 -- 172.21.15.113:0/2742100664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd890106160 msgr2=0x7fd89010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 --2- 172.21.15.113:0/2742100664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd890106160 0x7fd89010dd30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ca0 tx=0x7fd88802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 -- 172.21.15.113:0/2742100664 shutdown_connections 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 --2- 172.21.15.113:0/2742100664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd890106160 0x7fd89010dd30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 --2- 172.21.15.113:0/2742100664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890101a40 0x7fd890105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 --2- 172.21.15.113:0/2742100664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 0x7fd890101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 -- 172.21.15.113:0/2742100664 >> 172.21.15.113:0/2742100664 conn(0x7fd890077f50 msgr2=0x7fd890078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 -- 172.21.15.113:0/2742100664 shutdown_connections 2024-04-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.616+0000 7fd894975640 1 -- 172.21.15.113:0/2742100664 wait complete. 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 Processor -- start 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 -- start start 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 0x7fd890198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890106160 0x7fd890198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 0x7fd89019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd890112300 con 0x7fd89019cfe0 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd890112180 con 0x7fd890101090 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd894975640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd890112480 con 0x7fd890106160 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 0x7fd89019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 0x7fd89019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55096/0 (socket says 172.21.15.113:55096) 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 -- 172.21.15.113:0/861733663 learned_addr learned my addr 172.21.15.113:0/861733663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88e59c640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 0x7fd890198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88dd9b640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890106160 0x7fd890198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 -- 172.21.15.113:0/861733663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890106160 msgr2=0x7fd890198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890106160 0x7fd890198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 -- 172.21.15.113:0/861733663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 msgr2=0x7fd890198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 0x7fd890198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88ed9d640 1 -- 172.21.15.113:0/861733663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89019dad0 con 0x7fd89019cfe0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.617+0000 7fd88e59c640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 0x7fd890198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.618+0000 7fd88ed9d640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 0x7fd89019d490 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd888037040 tx=0x7fd888035d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.618+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888039040 con 0x7fd89019cfe0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.618+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89019dd60 con 0x7fd89019cfe0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.618+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd888044070 con 0x7fd89019cfe0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.618+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8880042a0 con 0x7fd89019cfe0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.618+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89010e8e0 con 0x7fd89019cfe0 2024-04-22T07:25:42.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.619+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd890068d70 con 0x7fd89019cfe0 2024-04-22T07:25:42.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.620+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fd888004440 con 0x7fd89019cfe0 2024-04-22T07:25:42.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.620+0000 7fd87f7fe640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd8780a3d20 0x7fd8780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:42.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.620+0000 7fd88e59c640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd8780a3d20 0x7fd8780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:42.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.620+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7fd8880f3520 con 0x7fd89019cfe0 2024-04-22T07:25:42.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.621+0000 7fd88e59c640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd8780a3d20 0x7fd8780a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c20 tx=0x7fd880005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:42.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.624+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8880bfb30 con 0x7fd89019cfe0 2024-04-22T07:25:42.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.814+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd890109420 con 0x7fd89019cfe0 2024-04-22T07:25:42.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.815+0000 7fd87f7fe640 1 -- 172.21.15.113:0/861733663 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 26 entries v103) ==== 81+0+1572 (secure 0 0 0) 0x7fd8880bf430 con 0x7fd89019cfe0 2024-04-22T07:25:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.154:6837/1615298521 2024-04-23T07:17:49.535084+0000 2024-04-22T07:25:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.154:6836/1615298521 2024-04-23T07:17:49.535084+0000 2024-04-22T07:25:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.154:6834/217490889 2024-04-23T07:17:49.535032+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.133:6838/982201245 2024-04-23T07:17:48.307009+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.154:6839/3404626603 2024-04-23T07:15:51.118540+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.154:6838/3404626603 2024-04-23T07:15:51.118540+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/2266395984 2024-04-23T07:05:54.991164+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6801/1303444723 2024-04-23T07:05:28.459936+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/2185314274 2024-04-23T07:05:28.459936+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/3528402889 2024-04-23T07:05:54.991164+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/1824685298 2024-04-23T07:14:49.346691+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/2368945695 2024-04-23T07:05:54.991164+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6800/1807162111 2024-04-23T07:05:54.991164+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.154:6835/217490889 2024-04-23T07:17:49.535032+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/4206830927 2024-04-23T07:05:28.459936+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6801/1807162111 2024-04-23T07:05:54.991164+0000 2024-04-22T07:25:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/3223226441 2024-04-23T07:14:49.346691+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.133:6839/982201245 2024-04-23T07:17:48.307009+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.133:6837/65907218 2024-04-23T07:15:51.098366+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6800/1303444723 2024-04-23T07:05:28.459936+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/3098383517 2024-04-23T07:14:49.346691+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/1679614079 2024-04-23T07:05:28.459936+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:0/3347479031 2024-04-23T07:14:49.346691+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.133:6836/65907218 2024-04-23T07:15:51.098366+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6800/610623186 2024-04-23T07:14:49.346691+0000 2024-04-22T07:25:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6801/610623186 2024-04-23T07:14:49.346691+0000 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:listed 26 entries 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd8780a3d20 msgr2=0x7fd8780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd8780a3d20 0x7fd8780a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c20 tx=0x7fd880005f90 comp rx=0 tx=0).stop 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 msgr2=0x7fd89019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 0x7fd89019d490 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd888037040 tx=0x7fd888035d00 comp rx=0 tx=0).stop 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 shutdown_connections 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fd8780a3d20 0x7fd8780a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd89019cfe0 0x7fd89019d490 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890106160 0x7fd890198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 --2- 172.21.15.113:0/861733663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd890101090 0x7fd890198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:42.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 >> 172.21.15.113:0/861733663 conn(0x7fd890077f50 msgr2=0x7fd8900fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:42.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 shutdown_connections 2024-04-22T07:25:42.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:42.817+0000 7fd894975640 1 -- 172.21.15.113:0/861733663 wait complete. 2024-04-22T07:25:42.824 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:42 smithi113 ceph-mon[27933]: pgmap v343: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:42.824 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:42 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2411177386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:25:42.824 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:42 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/631643914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:25:42.824 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:42 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2333033668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:25:42.825 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-22T07:25:42.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:42.825 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-22T07:25:42.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:25:42.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:42.866 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T07:25:42.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:42.945 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-22T07:25:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T07:25:43.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:43.001 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:25:43.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:42 smithi133 ceph-mon[31198]: pgmap v343: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:43.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:42 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2411177386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:25:43.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:42 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/631643914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:25:43.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:42 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2333033668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:25:43.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:43.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-22T07:25:43.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:42 smithi154 ceph-mon[31862]: pgmap v343: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-22T07:25:43.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:42 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2411177386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T07:25:43.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:42 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/631643914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T07:25:43.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:42 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2333033668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T07:25:43.164 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-22T07:25:43.164 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-22T07:25:43.164 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:25:43.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056098,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1213 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/39c2c85c4072c396bfc77ab2dbf583b9a246896611e8b9b817829c8aaf76b6fe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/39c2c85c4072c396bfc77ab2dbf583b9a246896611e8b9b817829c8aaf76b6fe/diff,workdir=/var/lib/containers/storage/overlay/39c2c85c4072c396bfc77ab2dbf583b9a246896611e8b9b817829c8aaf76b6fe/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e268346fb2295bd4fcca35a869bc415c110839901d956d769171630f234cb981/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/e268346fb2295bd4fcca35a869bc415c110839901d956d769171630f234cb981/diff,workdir=/var/lib/containers/storage/overlay/e268346fb2295bd4fcca35a869bc415c110839901d956d769171630f234cb981/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/bc955340ee32f405e8564d3d0eff8f2ca201522b21846f1286f579dbe8e53070/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/bc955340ee32f405e8564d3d0eff8f2ca201522b21846f1286f579dbe8e53070/diff,workdir=/var/lib/containers/storage/overlay/bc955340ee32f405e8564d3d0eff8f2ca201522b21846f1286f579dbe8e53070/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/606acadfcc77980520953a92c539f44ff3fe6c4725b32836d9f097493bcf0a5b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/606acadfcc77980520953a92c539f44ff3fe6c4725b32836d9f097493bcf0a5b/diff,workdir=/var/lib/containers/storage/overlay/606acadfcc77980520953a92c539f44ff3fe6c4725b32836d9f097493bcf0a5b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a7f6163499e5a1930e6ab682176d9805ae081ccf465fed462dfe2c199c2a3661/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/a7f6163499e5a1930e6ab682176d9805ae081ccf465fed462dfe2c199c2a3661/diff,workdir=/var/lib/containers/storage/overlay/a7f6163499e5a1930e6ab682176d9805ae081ccf465fed462dfe2c199c2a3661/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6d704c4a1fc5f91c48cf540d7695089c56feb8cb56a9f458147f83cb691ed0dc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/6d704c4a1fc5f91c48cf540d7695089c56feb8cb56a9f458147f83cb691ed0dc/diff,workdir=/var/lib/containers/storage/overlay/6d704c4a1fc5f91c48cf540d7695089c56feb8cb56a9f458147f83cb691ed0dc/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db949b45e8d1527852c58d034d154e2e882f50fc1926ea1f2d4d3028526a320e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/db949b45e8d1527852c58d034d154e2e882f50fc1926ea1f2d4d3028526a320e/diff,workdir=/var/lib/containers/storage/overlay/db949b45e8d1527852c58d034d154e2e882f50fc1926ea1f2d4d3028526a320e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/898aba7108fe31f5d3cb61062a621e7c9ce6abcd71717dce6c303ca03f0493bb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/898aba7108fe31f5d3cb61062a621e7c9ce6abcd71717dce6c303ca03f0493bb/diff,workdir=/var/lib/containers/storage/overlay/898aba7108fe31f5d3cb61062a621e7c9ce6abcd71717dce6c303ca03f0493bb/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/86bcdf4388b424f036b7ccbb5f1acee5a952b1eb1c4583ddf72a950d48f61a8a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/86bcdf4388b424f036b7ccbb5f1acee5a952b1eb1c4583ddf72a950d48f61a8a/diff,workdir=/var/lib/containers/storage/overlay/86bcdf4388b424f036b7ccbb5f1acee5a952b1eb1c4583ddf72a950d48f61a8a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f03092e2dbe755d950ee522be16f6e1469cd6975955e514edd4755db99b17d5b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/XWIZDDWCFEXSZ2XO3HBQTYPDC7:/var/lib/containers/storage/overlay/l/FV7DL6M6UXNMCJMJNOVZBSEZBA,upperdir=/var/lib/containers/storage/overlay/f03092e2dbe755d950ee522be16f6e1469cd6975955e514edd4755db99b17d5b/diff,workdir=/var/lib/containers/storage/overlay/f03092e2dbe755d950ee522be16f6e1469cd6975955e514edd4755db99b17d5b/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-22T07:25:43.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:43.181 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-04-22T07:25:43.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:25:43.236 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-04-22T07:25:43.303 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-04-22T07:25:43.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:25:43.305 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-22T07:25:43.316 INFO:tasks.cephadm:Teardown begin 2024-04-22T07:25:43.316 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-22T07:25:43.317 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:25:43.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 -- 172.21.15.113:0/2510191329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006a8b0 msgr2=0x7fe28006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 --2- 172.21.15.113:0/2510191329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006a8b0 0x7fe28006ac90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe274009900 tx=0x7fe27402bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 -- 172.21.15.113:0/2510191329 shutdown_connections 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 --2- 172.21.15.113:0/2510191329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe280107340 0x7fe28010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 --2- 172.21.15.113:0/2510191329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28006b260 0x7fe280106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 --2- 172.21.15.113:0/2510191329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006a8b0 0x7fe28006ac90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 -- 172.21.15.113:0/2510191329 >> 172.21.15.113:0/2510191329 conn(0x7fe280077e30 msgr2=0x7fe280078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 -- 172.21.15.113:0/2510191329 shutdown_connections 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.446+0000 7fe286a11640 1 -- 172.21.15.113:0/2510191329 wait complete. 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 Processor -- start 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 -- start start 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28006a8b0 0x7fe28019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 0x7fe28019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2801986f0 0x7fe280198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe280112390 con 0x7fe280107340 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe280112210 con 0x7fe28006a8b0 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe286a11640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe280112510 con 0x7fe2801986f0 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 0x7fe28019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 0x7fe28019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55106/0 (socket says 172.21.15.113:55106) 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 -- 172.21.15.113:0/717484454 learned_addr learned my addr 172.21.15.113:0/717484454 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27ffff640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28006a8b0 0x7fe28019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe284fae640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2801986f0 0x7fe280198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 -- 172.21.15.113:0/717484454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2801986f0 msgr2=0x7fe280198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2801986f0 0x7fe280198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 -- 172.21.15.113:0/717484454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28006a8b0 msgr2=0x7fe28019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28006a8b0 0x7fe28019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.447+0000 7fe27f7fe640 1 -- 172.21.15.113:0/717484454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2801992f0 con 0x7fe280107340 2024-04-22T07:25:43.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.448+0000 7fe27f7fe640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 0x7fe28019eb70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe27000d740 tx=0x7fe27000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:43.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.448+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270017070 con 0x7fe280107340 2024-04-22T07:25:43.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.448+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe27000de50 con 0x7fe280107340 2024-04-22T07:25:43.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.448+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2801a53d0 con 0x7fe280107340 2024-04-22T07:25:43.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.448+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27000f620 con 0x7fe280107340 2024-04-22T07:25:43.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.448+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2801a57d0 con 0x7fe280107340 2024-04-22T07:25:43.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.449+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2801036f0 con 0x7fe280107340 2024-04-22T07:25:43.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.450+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fe27001a070 con 0x7fe280107340 2024-04-22T07:25:43.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.450+0000 7fe27d7fa640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe2680a3d20 0x7fe2680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.451+0000 7fe27ffff640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe2680a3d20 0x7fe2680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.451+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7fe2700d0280 con 0x7fe280107340 2024-04-22T07:25:43.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.451+0000 7fe27ffff640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe2680a3d20 0x7fe2680a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe274009230 tx=0x7fe274005fd0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:43.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.454+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe27009c130 con 0x7fe280107340 2024-04-22T07:25:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.673+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe280069230 con 0x7fe280107340 2024-04-22T07:25:43.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.674+0000 7fe27d7fa640 1 -- 172.21.15.113:0/717484454 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7fe2700a0430 con 0x7fe280107340 2024-04-22T07:25:43.672 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:25:43.672 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe2680a3d20 msgr2=0x7fe2680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe2680a3d20 0x7fe2680a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe274009230 tx=0x7fe274005fd0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 msgr2=0x7fe28019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 0x7fe28019eb70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe27000d740 tx=0x7fe27000dc00 comp rx=0 tx=0).stop 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 shutdown_connections 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fe2680a3d20 0x7fe2680a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2801986f0 0x7fe280198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe280107340 0x7fe28019eb70 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 --2- 172.21.15.113:0/717484454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28006a8b0 0x7fe28019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 >> 172.21.15.113:0/717484454 conn(0x7fe280077e30 msgr2=0x7fe2800ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:43.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 shutdown_connections 2024-04-22T07:25:43.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.676+0000 7fe286a11640 1 -- 172.21.15.113:0/717484454 wait complete. 2024-04-22T07:25:43.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 -- 172.21.15.113:0/3011727996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0101b60 msgr2=0x7fabd010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 --2- 172.21.15.113:0/3011727996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0101b60 0x7fabd010dc90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009ca0 tx=0x7fabc802c140 comp rx=0 tx=0).stop 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 -- 172.21.15.113:0/3011727996 shutdown_connections 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 --2- 172.21.15.113:0/3011727996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0101b60 0x7fabd010dc90 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 --2- 172.21.15.113:0/3011727996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 0x7fabd0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 --2- 172.21.15.113:0/3011727996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd0105ee0 0x7fabd01062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 -- 172.21.15.113:0/3011727996 >> 172.21.15.113:0/3011727996 conn(0x7fabd0077c20 msgr2=0x7fabd0078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 -- 172.21.15.113:0/3011727996 shutdown_connections 2024-04-22T07:25:43.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.784+0000 7fabd8159640 1 -- 172.21.15.113:0/3011727996 wait complete. 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 Processor -- start 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 -- start start 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 0x7fabd00feb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd0101b60 0x7fabd00ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 0x7fabd00ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0111270 con 0x7fabd0105ee0 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01110f0 con 0x7fabd01011a0 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd8159640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01113f0 con 0x7fabd0101b60 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 0x7fabd00ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 0x7fabd00ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55118/0 (socket says 172.21.15.113:55118) 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 -- 172.21.15.113:0/2736538571 learned_addr learned my addr 172.21.15.113:0/2736538571 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd56f4640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd0101b60 0x7fabd00ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd5ef5640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 0x7fabd00feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 -- 172.21.15.113:0/2736538571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd0101b60 msgr2=0x7fabd00ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd0101b60 0x7fabd00ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 -- 172.21.15.113:0/2736538571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 msgr2=0x7fabd00feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 0x7fabd00feb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd66f6640 1 -- 172.21.15.113:0/2736538571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd01a6cb0 con 0x7fabd0105ee0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.785+0000 7fabd5ef5640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 0x7fabd00feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.786+0000 7fabd66f6640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 0x7fabd00ff5e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fabc802c750 tx=0x7fabc8035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.786+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8039040 con 0x7fabd0105ee0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.786+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fabc8044070 con 0x7fabd0105ee0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.786+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd01a6f40 con 0x7fabd0105ee0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.786+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004460 con 0x7fabd0105ee0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.786+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01a7400 con 0x7fabd0105ee0 2024-04-22T07:25:43.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.787+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd0069230 con 0x7fabd0105ee0 2024-04-22T07:25:43.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.788+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fabc80327a0 con 0x7fabd0105ee0 2024-04-22T07:25:43.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.788+0000 7fabc6ffd640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fabbc0a3d20 0x7fabbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:43.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.788+0000 7fabd5ef5640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fabbc0a3d20 0x7fabbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:43.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.788+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7fabc80f34a0 con 0x7fabd0105ee0 2024-04-22T07:25:43.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.789+0000 7fabd5ef5640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fabbc0a3d20 0x7fabbc0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fabcc005fd0 tx=0x7fabcc00b5d0 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:43.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:43.792+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabc80bfab0 con 0x7fabd0105ee0 2024-04-22T07:25:43.815 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:43 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/861733663' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T07:25:44.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.006+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabd0100090 con 0x7fabd0105ee0 2024-04-22T07:25:44.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.007+0000 7fabc6ffd640 1 -- 172.21.15.113:0/2736538571 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11885 (secure 0 0 0) 0x7fabc80bf3b0 con 0x7fabd0105ee0 2024-04-22T07:25:44.005 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/127167986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":127167986},{"type":"v1","addr":"172.21.15.113:6835","nonce":127167986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/918732402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":918732402},{"type":"v1","addr":"172.21.15.113:6837","nonce":918732402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/1495688995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":1495688995},{"type":"v1","addr":"172.21.15.113:6839","nonce":1495688995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6841/1868487444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6840","nonce":1868487444},{"type":"v1","addr":"172.21.15.113:6841","nonce":1868487444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/819873552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":819873552},{"type":"v1","addr":"172.21.15.154:6833","nonce":819873552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/4012646459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":4012646459},{"type":"v1","addr":"172.21.15.133:6833","nonce":4012646459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T07:12:54.514841+0000","modified":"2024-04-22T07:21:21.125672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24446,"mds_1":25367,"mds_2":25258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24446":{"gid":24446,"name":"e","rank":0,"incarnation":36,"state":"up:active","state_seq":84,"addr":"172.21.15.133:6835/3935683574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3935683574},{"type":"v1","addr":"172.21.15.133:6835","nonce":3935683574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25258":{"gid":25258,"name":"i","rank":2,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.154:6839/3008705527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":3008705527},{"type":"v1","addr":"172.21.15.154:6839","nonce":3008705527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25367":{"gid":25367,"name":"h","rank":1,"incarnation":38,"state":"up:active","state_seq":34,"addr":"172.21.15.133:6837/3335686802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3335686802},{"type":"v1","addr":"172.21.15.133:6837","nonce":3335686802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25576":{"gid":25576,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6835/2460958291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2460958291},{"type":"v1","addr":"172.21.15.154:6835","nonce":2460958291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25582":{"gid":25582,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.154:6837/2537342418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2537342418},{"type":"v1","addr":"172.21.15.154:6837","nonce":2537342418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25718":{"gid":25718,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6839/22002757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":22002757},{"type":"v1","addr":"172.21.15.133:6839","nonce":22002757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24446,"qdb_cluster":[25367,25258,24446]},"id":1}]} 2024-04-22T07:25:44.005 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fabbc0a3d20 msgr2=0x7fabbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fabbc0a3d20 0x7fabbc0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fabcc005fd0 tx=0x7fabcc00b5d0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 msgr2=0x7fabd00ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 0x7fabd00ff5e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fabc802c750 tx=0x7fabc8035d60 comp rx=0 tx=0).stop 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 shutdown_connections 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fabbc0a3d20 0x7fabbc0a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd0105ee0 0x7fabd00ff5e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd0101b60 0x7fabd00ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 --2- 172.21.15.113:0/2736538571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabd01011a0 0x7fabd00feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 >> 172.21.15.113:0/2736538571 conn(0x7fabd0077c20 msgr2=0x7fabd006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:44.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 shutdown_connections 2024-04-22T07:25:44.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.009+0000 7fabd8159640 1 -- 172.21.15.113:0/2736538571 wait complete. 2024-04-22T07:25:44.016 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:43 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/861733663' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T07:25:44.018 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:25:44.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:43 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/861733663' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T07:25:44.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 -- 172.21.15.113:0/1619136922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff841011a0 msgr2=0x7eff84101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 --2- 172.21.15.113:0/1619136922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff841011a0 0x7eff84101620 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7eff78009930 tx=0x7eff7802bc30 comp rx=0 tx=0).stop 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 -- 172.21.15.113:0/1619136922 shutdown_connections 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 --2- 172.21.15.113:0/1619136922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff84101b60 0x7eff8410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 --2- 172.21.15.113:0/1619136922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff841011a0 0x7eff84101620 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 --2- 172.21.15.113:0/1619136922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff84105e30 0x7eff84106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 -- 172.21.15.113:0/1619136922 >> 172.21.15.113:0/1619136922 conn(0x7eff84077e30 msgr2=0x7eff84078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 -- 172.21.15.113:0/1619136922 shutdown_connections 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.118+0000 7eff8a752640 1 -- 172.21.15.113:0/1619136922 wait complete. 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 Processor -- start 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 -- start start 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff841011a0 0x7eff841996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff84101b60 0x7eff84199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 0x7eff841a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff841112a0 con 0x7eff84105e30 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff84111120 con 0x7eff84101b60 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff8a752640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7eff84111420 con 0x7eff841011a0 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff88cef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 0x7eff841a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff88cef640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 0x7eff841a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55140/0 (socket says 172.21.15.113:55140) 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff88cef640 1 -- 172.21.15.113:0/626216008 learned_addr learned my addr 172.21.15.113:0/626216008 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff83fff640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff841011a0 0x7eff841996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.119+0000 7eff837fe640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff84101b60 0x7eff84199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff88cef640 1 -- 172.21.15.113:0/626216008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff841011a0 msgr2=0x7eff841996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff88cef640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff841011a0 0x7eff841996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff88cef640 1 -- 172.21.15.113:0/626216008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff84101b60 msgr2=0x7eff84199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff88cef640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff84101b60 0x7eff84199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff88cef640 1 -- 172.21.15.113:0/626216008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841a4c80 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff88cef640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 0x7eff841a4740 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00cad0 tx=0x7eff7c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00ad60 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7eff7c005020 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff841a4e50 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00f690 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.120+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841a5320 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.121+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff84068de0 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.122+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7eff7c0051d0 con 0x7eff84105e30 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.123+0000 7eff817fa640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.123+0000 7eff83fff640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.123+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7eff7c0d0c90 con 0x7eff84105e30 2024-04-22T07:25:44.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.123+0000 7eff83fff640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7eff74009990 tx=0x7eff74009110 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:44.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.126+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff7c014030 con 0x7eff84105e30 2024-04-22T07:25:44.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.303+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff840696f0 con 0x7eff84105e30 2024-04-22T07:25:44.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.305+0000 7eff817fa640 1 -- 172.21.15.113:0/626216008 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23827 (secure 0 0 0) 0x7eff840696f0 con 0x7eff84105e30 2024-04-22T07:25:44.303 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-22T07:25:44.303 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"fsid":"81035712-0076-11ef-bc93-c7b262605968","created":"2024-04-22T07:04:58.384709+0000","modified":"2024-04-22T07:25:13.493086+0000","last_up_change":"2024-04-22T07:12:04.580925+0000","last_in_change":"2024-04-22T07:11:43.724537+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:08:19.077214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:12:52.335272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:12:53.444188+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99b292ee-908e-46a7-8b87-733e426fe61b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6803","nonce":1926824887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6805","nonce":1926824887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6809","nonce":1926824887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1926824887},{"type":"v1","addr":"172.21.15.113:6807","nonce":1926824887}]},"public_addr":"172.21.15.113:6803/1926824887","cluster_addr":"172.21.15.113:6805/1926824887","heartbeat_back_addr":"172.21.15.113:6809/1926824887","heartbeat_front_addr":"172.21.15.113:6807/1926824887","state":["exists","up"]},{"osd":1,"uuid":"fed6e4a7-b083-483a-b8ec-2b567c9f6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6801","nonce":283944628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6803","nonce":283944628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6807","nonce":283944628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":283944628},{"type":"v1","addr":"172.21.15.133:6805","nonce":283944628}]},"public_addr":"172.21.15.133:6801/283944628","cluster_addr":"172.21.15.133:6803/283944628","heartbeat_back_addr":"172.21.15.133:6807/283944628","heartbeat_front_addr":"172.21.15.133:6805/283944628","state":["exists","up"]},{"osd":2,"uuid":"ed142201-490b-4bd4-8fe4-5aeaa334c18a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6801","nonce":343492984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6803","nonce":343492984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6807","nonce":343492984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":343492984},{"type":"v1","addr":"172.21.15.154:6805","nonce":343492984}]},"public_addr":"172.21.15.154:6801/343492984","cluster_addr":"172.21.15.154:6803/343492984","heartbeat_back_addr":"172.21.15.154:6807/343492984","heartbeat_front_addr":"172.21.15.154:6805/343492984","state":["exists","up"]},{"osd":3,"uuid":"06e87876-83f8-46b3-8daf-3ebf810d8b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6811","nonce":1273649373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6813","nonce":1273649373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6817","nonce":1273649373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":1273649373},{"type":"v1","addr":"172.21.15.113:6815","nonce":1273649373}]},"public_addr":"172.21.15.113:6811/1273649373","cluster_addr":"172.21.15.113:6813/1273649373","heartbeat_back_addr":"172.21.15.113:6817/1273649373","heartbeat_front_addr":"172.21.15.113:6815/1273649373","state":["exists","up"]},{"osd":4,"uuid":"35753be4-5e96-47bf-8527-b7c4baeb873e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6809","nonce":2027989696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6811","nonce":2027989696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6815","nonce":2027989696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2027989696},{"type":"v1","addr":"172.21.15.133:6813","nonce":2027989696}]},"public_addr":"172.21.15.133:6809/2027989696","cluster_addr":"172.21.15.133:6811/2027989696","heartbeat_back_addr":"172.21.15.133:6815/2027989696","heartbeat_front_addr":"172.21.15.133:6813/2027989696","state":["exists","up"]},{"osd":5,"uuid":"ed9d2dcb-3e4a-4a70-8d14-f2e00ec3f724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6809","nonce":3828843548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6811","nonce":3828843548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6815","nonce":3828843548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":3828843548},{"type":"v1","addr":"172.21.15.154:6813","nonce":3828843548}]},"public_addr":"172.21.15.154:6809/3828843548","cluster_addr":"172.21.15.154:6811/3828843548","heartbeat_back_addr":"172.21.15.154:6815/3828843548","heartbeat_front_addr":"172.21.15.154:6813/3828843548","state":["exists","up"]},{"osd":6,"uuid":"6196d5c6-a3a0-407c-af4a-3ca38ca997fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6819","nonce":499388993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6821","nonce":499388993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6825","nonce":499388993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":499388993},{"type":"v1","addr":"172.21.15.113:6823","nonce":499388993}]},"public_addr":"172.21.15.113:6819/499388993","cluster_addr":"172.21.15.113:6821/499388993","heartbeat_back_addr":"172.21.15.113:6825/499388993","heartbeat_front_addr":"172.21.15.113:6823/499388993","state":["exists","up"]},{"osd":7,"uuid":"da26f96f-5fe9-4081-8152-7396392c2754","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6817","nonce":3422116555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6819","nonce":3422116555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6823","nonce":3422116555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3422116555},{"type":"v1","addr":"172.21.15.133:6821","nonce":3422116555}]},"public_addr":"172.21.15.133:6817/3422116555","cluster_addr":"172.21.15.133:6819/3422116555","heartbeat_back_addr":"172.21.15.133:6823/3422116555","heartbeat_front_addr":"172.21.15.133:6821/3422116555","state":["exists","up"]},{"osd":8,"uuid":"c4f5d70d-b3b8-447c-8d18-60e6ac682dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6817","nonce":3748797627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6819","nonce":3748797627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6823","nonce":3748797627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":3748797627},{"type":"v1","addr":"172.21.15.154:6821","nonce":3748797627}]},"public_addr":"172.21.15.154:6817/3748797627","cluster_addr":"172.21.15.154:6819/3748797627","heartbeat_back_addr":"172.21.15.154:6823/3748797627","heartbeat_front_addr":"172.21.15.154:6821/3748797627","state":["exists","up"]},{"osd":9,"uuid":"c9a6649c-b736-4f95-986a-2ff0463c0995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6827","nonce":744209950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6829","nonce":744209950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6833","nonce":744209950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":744209950},{"type":"v1","addr":"172.21.15.113:6831","nonce":744209950}]},"public_addr":"172.21.15.113:6827/744209950","cluster_addr":"172.21.15.113:6829/744209950","heartbeat_back_addr":"172.21.15.113:6833/744209950","heartbeat_front_addr":"172.21.15.113:6831/744209950","state":["exists","up"]},{"osd":10,"uuid":"513726aa-29df-4ae7-bebc-db08c423daf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6825","nonce":1753665247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6827","nonce":1753665247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6831","nonce":1753665247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1753665247},{"type":"v1","addr":"172.21.15.133:6829","nonce":1753665247}]},"public_addr":"172.21.15.133:6825/1753665247","cluster_addr":"172.21.15.133:6827/1753665247","heartbeat_back_addr":"172.21.15.133:6831/1753665247","heartbeat_front_addr":"172.21.15.133:6829/1753665247","state":["exists","up"]},{"osd":11,"uuid":"0998e208-1905-409f-8e8d-7e8b7d1703dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6825","nonce":3813220428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6827","nonce":3813220428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6831","nonce":3813220428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":3813220428},{"type":"v1","addr":"172.21.15.154:6829","nonce":3813220428}]},"public_addr":"172.21.15.154:6825/3813220428","cluster_addr":"172.21.15.154:6827/3813220428","heartbeat_back_addr":"172.21.15.154:6831/3813220428","heartbeat_front_addr":"172.21.15.154:6829/3813220428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:27.856980+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:07:52.114045+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:16.553524+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:08:41.535224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:05.817334+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:30.575289+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:09:55.461642+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:20.471686+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:10:45.881020+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:11.500034+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:11:36.765435+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:12:02.100244+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1615298521":"2024-04-23T07:17:49.535084+0000","172.21.15.154:6836/1615298521":"2024-04-23T07:17:49.535084+0000","172.21.15.154:6834/217490889":"2024-04-23T07:17:49.535032+0000","172.21.15.133:6838/982201245":"2024-04-23T07:17:48.307009+0000","172.21.15.154:6839/3404626603":"2024-04-23T07:15:51.118540+0000","172.21.15.154:6838/3404626603":"2024-04-23T07:15:51.118540+0000","172.21.15.113:0/2266395984":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6801/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/2185314274":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3528402889":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/1824685298":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/2368945695":"2024-04-23T07:05:54.991164+0000","172.21.15.113:6800/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.154:6835/217490889":"2024-04-23T07:17:49.535032+0000","172.21.15.113:0/4206830927":"2024-04-23T07:05:28.459936+0000","172.21.15.113:6801/1807162111":"2024-04-23T07:05:54.991164+0000","172.21.15.113:0/3223226441":"2024-04-23T07:14:49.346691+0000","172.21.15.133:6839/982201245":"2024-04-23T07:17:48.307009+0000","172.21.15.133:6837/65907218":"2024-04-23T07:15:51.098366+0000","172.21.15.113:6800/1303444723":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3098383517":"2024-04-23T07:14:49.346691+0000","172.21.15.113:0/1679614079":"2024-04-23T07:05:28.459936+0000","172.21.15.113:0/3347479031":"2024-04-23T07:14:49.346691+0000","172.21.15.133:6836/65907218":"2024-04-23T07:15:51.098366+0000","172.21.15.113:6800/610623186":"2024-04-23T07:14:49.346691+0000","172.21.15.113:6801/610623186":"2024-04-23T07:14:49.346691+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7eff6c0a3d20 msgr2=0x7eff6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7eff74009990 tx=0x7eff74009110 comp rx=0 tx=0).stop 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 msgr2=0x7eff841a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 0x7eff841a4740 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00cad0 tx=0x7eff7c00ced0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 shutdown_connections 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff84105e30 0x7eff841a4740 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff84101b60 0x7eff84199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 --2- 172.21.15.113:0/626216008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff841011a0 0x7eff841996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 >> 172.21.15.113:0/626216008 conn(0x7eff84077e30 msgr2=0x7eff840fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:44.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 shutdown_connections 2024-04-22T07:25:44.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.307+0000 7eff8a752640 1 -- 172.21.15.113:0/626216008 wait complete. 2024-04-22T07:25:44.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-22T07:25:44.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 -- 172.21.15.113:0/1980113405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570101090 msgr2=0x7f0570101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 --2- 172.21.15.113:0/1980113405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570101090 0x7f0570101470 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ca0 tx=0x7f056802c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 -- 172.21.15.113:0/1980113405 shutdown_connections 2024-04-22T07:25:44.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 --2- 172.21.15.113:0/1980113405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570106160 0x7f057010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 --2- 172.21.15.113:0/1980113405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0570101a40 0x7f0570105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 --2- 172.21.15.113:0/1980113405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570101090 0x7f0570101470 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 -- 172.21.15.113:0/1980113405 >> 172.21.15.113:0/1980113405 conn(0x7f0570077f50 msgr2=0x7f0570078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 -- 172.21.15.113:0/1980113405 shutdown_connections 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 -- 172.21.15.113:0/1980113405 wait complete. 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 Processor -- start 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.421+0000 7f0576e55640 1 -- start start 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570101090 0x7f0570199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 0x7f0570199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057019a130 0x7f05701a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0570111200 con 0x7f0570106160 2024-04-22T07:25:44.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0570111080 con 0x7f0570101090 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0570111380 con 0x7f057019a130 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 0x7f0570199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 0x7f0570199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55150/0 (socket says 172.21.15.113:55150) 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 -- 172.21.15.113:0/120411075 learned_addr learned my addr 172.21.15.113:0/120411075 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f05753f2640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057019a130 0x7f05701a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0574bf1640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570101090 0x7f0570199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 -- 172.21.15.113:0/120411075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057019a130 msgr2=0x7f05701a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057019a130 0x7f05701a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 -- 172.21.15.113:0/120411075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570101090 msgr2=0x7f0570199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570101090 0x7f0570199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 -- 172.21.15.113:0/120411075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05701a4ee0 con 0x7f0570106160 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056ffff640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 0x7f0570199ba0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f056000c970 tx=0x7f056000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560007bb0 con 0x7f0570106160 2024-04-22T07:25:44.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05701a51d0 con 0x7f0570106160 2024-04-22T07:25:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0560007d50 con 0x7f0570106160 2024-04-22T07:25:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.422+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560018780 con 0x7f0570106160 2024-04-22T07:25:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.423+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05701a5720 con 0x7f0570106160 2024-04-22T07:25:44.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.424+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0570068d70 con 0x7f0570106160 2024-04-22T07:25:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.425+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f056000fa40 con 0x7f0570106160 2024-04-22T07:25:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.425+0000 7f056dffb640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f055c0a3d20 0x7f055c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.425+0000 7f0574bf1640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f055c0a3d20 0x7f055c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.426+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8943+0+0 (secure 0 0 0) 0x7f05600d0a00 con 0x7f0570106160 2024-04-22T07:25:44.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.426+0000 7f0574bf1640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f055c0a3d20 0x7f055c0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0568000f80 tx=0x7f0568005c30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:44.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.430+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0560014030 con 0x7f0570106160 2024-04-22T07:25:44.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.663+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0570109390 con 0x7f0570106160 2024-04-22T07:25:44.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.666+0000 7f056dffb640 1 -- 172.21.15.113:0/120411075 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) ==== 161+0+0 (secure 0 0 0) 0x7f056009cf00 con 0x7f0570106160 2024-04-22T07:25:44.663 INFO:teuthology.orchestra.run.smithi113.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-22T07:25:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.668+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f055c0a3d20 msgr2=0x7f055c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.668+0000 7f0576e55640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f055c0a3d20 0x7f055c0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0568000f80 tx=0x7f0568005c30 comp rx=0 tx=0).stop 2024-04-22T07:25:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.668+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 msgr2=0x7f0570199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.668+0000 7f0576e55640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 0x7f0570199ba0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f056000c970 tx=0x7f056000ce30 comp rx=0 tx=0).stop 2024-04-22T07:25:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 shutdown_connections 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f055c0a3d20 0x7f055c0a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f057019a130 0x7f05701a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0570106160 0x7f0570199ba0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 --2- 172.21.15.113:0/120411075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570101090 0x7f0570199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 >> 172.21.15.113:0/120411075 conn(0x7f0570077f50 msgr2=0x7f05700ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 shutdown_connections 2024-04-22T07:25:44.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.669+0000 7f0576e55640 1 -- 172.21.15.113:0/120411075 wait complete. 2024-04-22T07:25:44.677 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-22T07:25:44.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 -- 172.21.15.113:0/2940235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f0102b50 msgr2=0x7f03f0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2940235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f0102b50 0x7f03f0106b90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009ca0 tx=0x7f03e402c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 -- 172.21.15.113:0/2940235706 shutdown_connections 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2940235706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f0107270 0x7f03f010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2940235706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f0102b50 0x7f03f0106b90 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2940235706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03f01021a0 0x7f03f0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 -- 172.21.15.113:0/2940235706 >> 172.21.15.113:0/2940235706 conn(0x7f03f00793e0 msgr2=0x7f03f00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 -- 172.21.15.113:0/2940235706 shutdown_connections 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.780+0000 7f03f4c43640 1 -- 172.21.15.113:0/2940235706 wait complete. 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 Processor -- start 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 -- start start 2024-04-22T07:25:44.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 0x7f03f019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f0107270 0x7f03f019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03f0198650 0x7f03f0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f03f01122d0 con 0x7f03f01021a0 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0112150 con 0x7f03f0107270 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03f4c43640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0112450 con 0x7f03f0198650 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03ee59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 0x7f03f019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03ee59c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 0x7f03f019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55162/0 (socket says 172.21.15.113:55162) 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.781+0000 7f03ee59c640 1 -- 172.21.15.113:0/2016593889 learned_addr learned my addr 172.21.15.113:0/2016593889 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03eed9d640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03f0198650 0x7f03f0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03edd9b640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f0107270 0x7f03f019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03ee59c640 1 -- 172.21.15.113:0/2016593889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03f0198650 msgr2=0x7f03f0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03ee59c640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03f0198650 0x7f03f0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03ee59c640 1 -- 172.21.15.113:0/2016593889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f0107270 msgr2=0x7f03f019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03ee59c640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f0107270 0x7f03f019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03ee59c640 1 -- 172.21.15.113:0/2016593889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f0199250 con 0x7f03f01021a0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03ee59c640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 0x7f03f019e590 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c890 tx=0x7f03e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8004220 con 0x7f03f01021a0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f03e80043c0 con 0x7f03f01021a0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f01a5330 con 0x7f03f01021a0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e80049c0 con 0x7f03f01021a0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.782+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f01a56e0 con 0x7f03f01021a0 2024-04-22T07:25:44.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.783+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f0103670 con 0x7f03f01021a0 2024-04-22T07:25:44.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.784+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f03e8005140 con 0x7f03f01021a0 2024-04-22T07:25:44.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.785+0000 7f03df7fe640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f03d80a3d20 0x7f03d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:44.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.785+0000 7f03edd9b640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f03d80a3d20 0x7f03d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:44.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.785+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9157+0+0 (secure 0 0 0) 0x7f03e80d0ac0 con 0x7f03f01021a0 2024-04-22T07:25:44.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.785+0000 7f03edd9b640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f03d80a3d20 0x7f03d80a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f03e4005cc0 tx=0x7f03e4005c50 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:44.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:44.789+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03e8014030 con 0x7f03f01021a0 2024-04-22T07:25:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:44 smithi113 ceph-mon[27933]: pgmap v344: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:44 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/717484454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:44 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2736538571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:44 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/626216008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.834 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a[27929]: 2024-04-22T07:25:44.664+0000 7fcdb4209700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T07:25:44.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mon[31198]: pgmap v344: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:44.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/717484454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2736538571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.965 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/626216008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[65475]: ignoring --setuser ceph since I am not root 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[65475]: ignoring --setgroup ceph since I am not root 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mds[65479]: -- 172.21.15.133:0/2528416994 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55baa1187a40 con 0x55baa115a800 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[65475]: starting mds.b at 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[64880]: ignoring --setuser ceph since I am not root 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[64880]: ignoring --setgroup ceph since I am not root 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mds[64885]: -- 172.21.15.133:0/4136871589 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ef98ada40 con 0x560ef9880c00 2024-04-22T07:25:44.966 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[64880]: starting mds.h at 2024-04-22T07:25:44.967 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: ignoring --setuser ceph since I am not root 2024-04-22T07:25:44.967 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: ignoring --setgroup ceph since I am not root 2024-04-22T07:25:44.967 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-mds[61275]: -- 172.21.15.133:0/3021123924 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d877fda40 con 0x559d877d1000 2024-04-22T07:25:44.967 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:25:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: starting mds.e at 2024-04-22T07:25:44.985 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mon[31862]: pgmap v344: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/717484454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2736538571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/626216008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[67312]: ignoring --setuser ceph since I am not root 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[67312]: ignoring --setgroup ceph since I am not root 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mds[67336]: -- 172.21.15.154:0/640544713 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4c6089a40 con 0x55a4c605d000 2024-04-22T07:25:44.986 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[67312]: starting mds.c at 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[67307]: ignoring --setuser ceph since I am not root 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[67307]: ignoring --setgroup ceph since I am not root 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mds[67316]: -- 172.21.15.154:0/4030347441 <== mon.0 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0aaa51a40 con 0x55b0aaa24800 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[67307]: starting mds.f at 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[66673]: ignoring --setuser ceph since I am not root 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[66673]: ignoring --setgroup ceph since I am not root 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-mds[66677]: -- 172.21.15.154:0/1279103931 <== mon.1 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55765f605a40 con 0x55765f5d9000 2024-04-22T07:25:44.987 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:25:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[66673]: starting mds.i at 2024-04-22T07:25:45.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.002+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f03f010a560 con 0x7f03f01021a0 2024-04-22T07:25:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.669+0000 7f03df7fe640 1 -- 172.21.15.113:0/2016593889 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f03e809c890 con 0x7f03f01021a0 2024-04-22T07:25:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f03d80a3d20 msgr2=0x7f03d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f03d80a3d20 0x7f03d80a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f03e4005cc0 tx=0x7f03e4005c50 comp rx=0 tx=0).stop 2024-04-22T07:25:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 msgr2=0x7f03f019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:45.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 0x7f03f019e590 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c890 tx=0x7f03e800cd50 comp rx=0 tx=0).stop 2024-04-22T07:25:45.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 shutdown_connections 2024-04-22T07:25:45.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f03d80a3d20 0x7f03d80a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03f0198650 0x7f03f0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03f0107270 0x7f03f019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 --2- 172.21.15.113:0/2016593889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03f01021a0 0x7f03f019e590 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 >> 172.21.15.113:0/2016593889 conn(0x7f03f00793e0 msgr2=0x7f03f00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 shutdown_connections 2024-04-22T07:25:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.672+0000 7f03f4c43640 1 -- 172.21.15.113:0/2016593889 wait complete. 2024-04-22T07:25:45.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 -- 172.21.15.113:0/2313843488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94107240 msgr2=0x7fad9410ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 --2- 172.21.15.113:0/2313843488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94107240 0x7fad9410ee10 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009ca0 tx=0x7fad8c02c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 -- 172.21.15.113:0/2313843488 shutdown_connections 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 --2- 172.21.15.113:0/2313843488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94107240 0x7fad9410ee10 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 --2- 172.21.15.113:0/2313843488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94102b20 0x7fad94106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 --2- 172.21.15.113:0/2313843488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad94102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 -- 172.21.15.113:0/2313843488 >> 172.21.15.113:0/2313843488 conn(0x7fad940fda10 msgr2=0x7fad940ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 -- 172.21.15.113:0/2313843488 shutdown_connections 2024-04-22T07:25:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.791+0000 7fad9a425640 1 -- 172.21.15.113:0/2313843488 wait complete. 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 Processor -- start 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 -- start start 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad9419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94107240 0x7fad9419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 0x7fad94198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fad941122e0 con 0x7fad94198600 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112160 con 0x7fad94107240 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad9a425640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112460 con 0x7fad94102170 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad989c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 0x7fad94198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad93fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad9419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad93fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad9419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.113:41206/0 (socket says 172.21.15.113:41206) 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad989c2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 0x7fad94198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36886/0 (socket says 172.21.15.113:36886) 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.792+0000 7fad93fff640 1 -- 172.21.15.113:0/4230289908 learned_addr learned my addr 172.21.15.113:0/4230289908 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad937fe640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94107240 0x7fad9419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad989c2640 1 -- 172.21.15.113:0/4230289908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 msgr2=0x7fad9419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad989c2640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad9419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad989c2640 1 -- 172.21.15.113:0/4230289908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94107240 msgr2=0x7fad9419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad989c2640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94107240 0x7fad9419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad989c2640 1 -- 172.21.15.113:0/4230289908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad94199120 con 0x7fad94198600 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad93fff640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad9419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad989c2640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 0x7fad94198ab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fad8c02c700 tx=0x7fad8c035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c039040 con 0x7fad94198600 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fad8c044070 con 0x7fad94198600 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad941993b0 con 0x7fad94198600 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004330 con 0x7fad94198600 2024-04-22T07:25:45.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.793+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad941a55a0 con 0x7fad94198600 2024-04-22T07:25:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.794+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad94068d70 con 0x7fad94198600 2024-04-22T07:25:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.795+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7fad8c0044d0 con 0x7fad94198600 2024-04-22T07:25:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.796+0000 7fad917fa640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad7c0a3d20 0x7fad7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:45.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.796+0000 7fad93fff640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad7c0a3d20 0x7fad7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:45.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.796+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9157+0+0 (secure 0 0 0) 0x7fad8c0f2e10 con 0x7fad94198600 2024-04-22T07:25:45.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.796+0000 7fad93fff640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad7c0a3d20 0x7fad7c0a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fad84011500 tx=0x7fad84005f90 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.801+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad8c0bf340 con 0x7fad94198600 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/120411075' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/120411075' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: osdmap e104: 12 total, 12 up, 12 in 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-22T07:25:45.919 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:45 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2016593889' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:25:45.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:45.978+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fad94102b20 con 0x7fad94198600 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/120411075' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/120411075' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: osdmap e104: 12 total, 12 up, 12 in 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-22T07:25:45.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:45 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2016593889' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/120411075' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/120411075' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: osdmap e104: 12 total, 12 up, 12 in 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-22T07:25:46.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:45 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2016593889' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T07:25:46.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.671+0000 7fad917fa640 1 -- 172.21.15.113:0/4230289908 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7fad8c0bec40 con 0x7fad94198600 2024-04-22T07:25:46.670 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' removed 2024-04-22T07:25:46.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad7c0a3d20 msgr2=0x7fad7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad7c0a3d20 0x7fad7c0a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fad84011500 tx=0x7fad84005f90 comp rx=0 tx=0).stop 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 msgr2=0x7fad94198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 0x7fad94198ab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fad8c02c700 tx=0x7fad8c035d80 comp rx=0 tx=0).stop 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 shutdown_connections 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7fad7c0a3d20 0x7fad7c0a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad94198600 0x7fad94198ab0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94107240 0x7fad9419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 --2- 172.21.15.113:0/4230289908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad94102170 0x7fad9419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 >> 172.21.15.113:0/4230289908 conn(0x7fad940fda10 msgr2=0x7fad940ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 shutdown_connections 2024-04-22T07:25:46.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.675+0000 7fad9a425640 1 -- 172.21.15.113:0/4230289908 wait complete. 2024-04-22T07:25:46.690 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 -- 172.21.15.113:0/495130666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84341010a0 msgr2=0x7f8434101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 --2- 172.21.15.113:0/495130666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84341010a0 0x7f8434101480 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8430009ca0 tx=0x7f843002c0f0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 -- 172.21.15.113:0/495130666 shutdown_connections 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 --2- 172.21.15.113:0/495130666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8434106170 0x7f843410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 --2- 172.21.15.113:0/495130666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8434101a50 0x7f8434105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 --2- 172.21.15.113:0/495130666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84341010a0 0x7f8434101480 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 -- 172.21.15.113:0/495130666 >> 172.21.15.113:0/495130666 conn(0x7f8434077eb0 msgr2=0x7f84340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 -- 172.21.15.113:0/495130666 shutdown_connections 2024-04-22T07:25:46.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.798+0000 7f843b469640 1 -- 172.21.15.113:0/495130666 wait complete. 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 Processor -- start 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 -- start start 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84341010a0 0x7f8434199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8434106170 0x7f8434199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 0x7f84341a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8434111270 con 0x7f843419a170 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f84341110f0 con 0x7f8434106170 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f843b469640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f84341113f0 con 0x7f84341010a0 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 0x7f84341a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 0x7f84341a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36894/0 (socket says 172.21.15.113:36894) 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 -- 172.21.15.113:0/2313895491 learned_addr learned my addr 172.21.15.113:0/2313895491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8438a04640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8434106170 0x7f8434199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439205640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84341010a0 0x7f8434199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 -- 172.21.15.113:0/2313895491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84341010a0 msgr2=0x7f8434199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:46.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84341010a0 0x7f8434199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 -- 172.21.15.113:0/2313895491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8434106170 msgr2=0x7f8434199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8434106170 0x7f8434199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 -- 172.21.15.113:0/2313895491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84341a4ef0 con 0x7f843419a170 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.799+0000 7f8439a06640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 0x7f84341a46d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f842c009fd0 tx=0x7f842c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.800+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c015020 con 0x7f843419a170 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.800+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341a51e0 con 0x7f843419a170 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.800+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f842c011d00 con 0x7f843419a170 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.800+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c010e20 con 0x7f843419a170 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.800+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a5730 con 0x7f843419a170 2024-04-22T07:25:46.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.801+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84340691c0 con 0x7f843419a170 2024-04-22T07:25:46.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.802+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 26) ==== 138778+0+0 (secure 0 0 0) 0x7f842c018020 con 0x7f843419a170 2024-04-22T07:25:46.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.802+0000 7f842a7fc640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f84200a3d20 0x7f84200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:25:46.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.802+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8603+0+0 (secure 0 0 0) 0x7f842c0cff80 con 0x7f843419a170 2024-04-22T07:25:46.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.802+0000 7f8439205640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f84200a3d20 0x7f84200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:25:46.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.803+0000 7f8439205640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f84200a3d20 0x7f84200a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8430000f80 tx=0x7f8430005c30 comp rx=0 tx=0).ready entity=mgr.15279 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:25:46.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.807+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f842c01f020 con 0x7f843419a170 2024-04-22T07:25:46.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:46.987+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f84341093a0 con 0x7f843419a170 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: pgmap v346: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 819 B/s wr, 7 op/s 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: Cluster is now healthy 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2016593889' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6834/3948608632,v1:172.21.15.133:6835/3948608632] up:boot 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6834/1272448502,v1:172.21.15.154:6837/1272448502] up:boot 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6836/1398470483,v1:172.21.15.154:6838/1398470483] up:boot 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.154:6835/3586061668,v1:172.21.15.154:6839/3586061668] up:boot 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6836/1726056752,v1:172.21.15.133:6837/1726056752] up:boot 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: mds.? [v2:172.21.15.133:6838/3953477663,v1:172.21.15.133:6839/3953477663] up:boot 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: fsmap 12 up:standby 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:25:47.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:25:47.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:25:47.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:25:47.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:25:47.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:25:47.065 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:46 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4230289908' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T07:25:47.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: pgmap v346: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 819 B/s wr, 7 op/s 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: Cluster is now healthy 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2016593889' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6834/3948608632,v1:172.21.15.133:6835/3948608632] up:boot 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6834/1272448502,v1:172.21.15.154:6837/1272448502] up:boot 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6836/1398470483,v1:172.21.15.154:6838/1398470483] up:boot 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.154:6835/3586061668,v1:172.21.15.154:6839/3586061668] up:boot 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6836/1726056752,v1:172.21.15.133:6837/1726056752] up:boot 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: mds.? [v2:172.21.15.133:6838/3953477663,v1:172.21.15.133:6839/3953477663] up:boot 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: fsmap 12 up:standby 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:25:47.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:25:47.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:25:47.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:25:47.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:25:47.067 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:46 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4230289908' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T07:25:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: pgmap v346: 97 pgs: 97 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 819 B/s wr, 7 op/s 2024-04-22T07:25:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T07:25:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T07:25:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: Cluster is now healthy 2024-04-22T07:25:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2016593889' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T07:25:47.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6834/3948608632,v1:172.21.15.133:6835/3948608632] up:boot 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6834/1272448502,v1:172.21.15.154:6837/1272448502] up:boot 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6836/1398470483,v1:172.21.15.154:6838/1398470483] up:boot 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.154:6835/3586061668,v1:172.21.15.154:6839/3586061668] up:boot 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6836/1726056752,v1:172.21.15.133:6837/1726056752] up:boot 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: mds.? [v2:172.21.15.133:6838/3953477663,v1:172.21.15.133:6839/3953477663] up:boot 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: fsmap 12 up:standby 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='mgr.15279 172.21.15.113:0/521586513' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T07:25:47.085 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:46 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4230289908' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T07:25:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.677+0000 7f842a7fc640 1 -- 172.21.15.113:0/2313895491 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f842c09c6e0 con 0x7f843419a170 2024-04-22T07:25:47.675 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' removed 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f84200a3d20 msgr2=0x7f84200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f84200a3d20 0x7f84200a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8430000f80 tx=0x7f8430005c30 comp rx=0 tx=0).stop 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 msgr2=0x7f84341a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 0x7f84341a46d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f842c009fd0 tx=0x7f842c00b820 comp rx=0 tx=0).stop 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 shutdown_connections 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:6800/2258977739,v1:172.21.15.113:6801/2258977739] conn(0x7f84200a3d20 0x7f84200a61e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843419a170 0x7f84341a46d0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8434106170 0x7f8434199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 --2- 172.21.15.113:0/2313895491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84341010a0 0x7f8434199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 >> 172.21.15.113:0/2313895491 conn(0x7f8434077eb0 msgr2=0x7f84340fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 shutdown_connections 2024-04-22T07:25:47.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-22T07:25:47.680+0000 7f843b469640 1 -- 172.21.15.113:0/2313895491 wait complete. 2024-04-22T07:25:47.694 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:25:47.722 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:25:47.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:25:47.781 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-22T07:25:47.781 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T07:25:47.796 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T07:25:47.810 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T07:25:47.837 INFO:tasks.cephadm:Stopping all daemons... 2024-04-22T07:25:47.837 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-22T07:25:47.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a 2024-04-22T07:25:48.063 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:47 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/4230289908' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T07:25:48.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:47 smithi133 ceph-mon[31198]: osdmap e105: 12 total, 12 up, 12 in 2024-04-22T07:25:48.064 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:47 smithi133 ceph-mon[31198]: from='client.? 172.21.15.113:0/2313895491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T07:25:48.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:47 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/4230289908' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T07:25:48.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:47 smithi154 ceph-mon[31862]: osdmap e105: 12 total, 12 up, 12 in 2024-04-22T07:25:48.065 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:47 smithi154 ceph-mon[31862]: from='client.? 172.21.15.113:0/2313895491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T07:25:48.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:47 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/4230289908' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T07:25:48.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:47 smithi113 ceph-mon[27933]: osdmap e105: 12 total, 12 up, 12 in 2024-04-22T07:25:48.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:47 smithi113 ceph-mon[27933]: from='client.? 172.21.15.113:0/2313895491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T07:25:48.084 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:47 smithi113 systemd[1]: Stopping Ceph mon.a for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a[27929]: 2024-04-22T07:25:48.117+0000 7fcdb9a14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:25:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a[27929]: 2024-04-22T07:25:48.117+0000 7fcdb9a14700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-22T07:25:48.584 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:48 smithi113 podman[93377]: 2024-04-22 07:25:48.333359399 +0000 UTC m=+0.299461210 container died b65780c4ca1b883f1ebc5aa14aededfbe33e3627d91b330268067768f794ed97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS) 2024-04-22T07:25:48.967 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:48 smithi113 podman[93377]: 2024-04-22 07:25:48.718919511 +0000 UTC m=+0.685021320 container cleanup b65780c4ca1b883f1ebc5aa14aededfbe33e3627d91b330268067768f794ed97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:25:48.967 INFO:journalctl@ceph.mon.a.smithi113.stdout:Apr 22 07:25:48 smithi113 bash[93377]: ceph-81035712-0076-11ef-bc93-c7b262605968-mon-a 2024-04-22T07:25:49.132 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mon.a.service' 2024-04-22T07:25:49.178 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:25:49.178 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-22T07:25:49.178 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-22T07:25:49.179 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mon.b 2024-04-22T07:25:49.509 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:49 smithi133 systemd[1]: Stopping Ceph mon.b for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:49.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-b[31194]: 2024-04-22T07:25:49.511+0000 7fbcb2046700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:25:49.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-b[31194]: 2024-04-22T07:25:49.511+0000 7fbcb2046700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-22T07:25:49.814 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:49 smithi133 podman[67699]: 2024-04-22 07:25:49.572944989 +0000 UTC m=+0.181238009 container died ba8f14fe86288af8cc7c3e99976f14ca9d56b45cca8ade96621d9207fba1766a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, ceph=True) 2024-04-22T07:25:50.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:49 smithi133 podman[67699]: 2024-04-22 07:25:49.937128251 +0000 UTC m=+0.545421280 container cleanup ba8f14fe86288af8cc7c3e99976f14ca9d56b45cca8ade96621d9207fba1766a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-22T07:25:50.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Apr 22 07:25:49 smithi133 bash[67699]: ceph-81035712-0076-11ef-bc93-c7b262605968-mon-b 2024-04-22T07:25:50.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mon.b.service' 2024-04-22T07:25:50.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:25:50.403 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-22T07:25:50.404 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-22T07:25:50.404 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mon.c 2024-04-22T07:25:50.739 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:50 smithi154 systemd[1]: Stopping Ceph mon.c for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:51.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-c[31858]: 2024-04-22T07:25:50.741+0000 7fac818f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:25:51.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mon-c[31858]: 2024-04-22T07:25:50.741+0000 7fac818f4700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-22T07:25:51.066 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:50 smithi154 podman[69566]: 2024-04-22 07:25:50.926004334 +0000 UTC m=+0.325101956 container died 83e618809653fc5cb3b2855da565a0d71e43fdd0ce9efb9313765bd68f9e9541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T07:25:51.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:51 smithi154 podman[69566]: 2024-04-22 07:25:51.375322314 +0000 UTC m=+0.774419934 container cleanup 83e618809653fc5cb3b2855da565a0d71e43fdd0ce9efb9313765bd68f9e9541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-22T07:25:51.565 INFO:journalctl@ceph.mon.c.smithi154.stdout:Apr 22 07:25:51 smithi154 bash[69566]: ceph-81035712-0076-11ef-bc93-c7b262605968-mon-c 2024-04-22T07:25:51.790 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mon.c.service' 2024-04-22T07:25:51.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:25:51.855 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-22T07:25:51.855 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-22T07:25:51.855 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x 2024-04-22T07:25:52.154 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:25:51 smithi113 systemd[1]: Stopping Ceph mgr.x for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:52.404 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:25:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x[28170]: 2024-04-22T07:25:52.154+0000 7f14467fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:25:52.404 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:25:52 smithi113 podman[93499]: 2024-04-22 07:25:52.187206539 +0000 UTC m=+0.139768613 container died 8768c6742f4334d7a36c4b3cf5f7a5317dde72ccd1ee8cde30b8eea46ee717c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:25:52.755 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:25:52 smithi113 podman[93499]: 2024-04-22 07:25:52.473002064 +0000 UTC m=+0.425564140 container cleanup 8768c6742f4334d7a36c4b3cf5f7a5317dde72ccd1ee8cde30b8eea46ee717c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:86f7587a5a09af35f0895e1a2d08527638fae697, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-22T07:25:52.755 INFO:journalctl@ceph.mgr.x.smithi113.stdout:Apr 22 07:25:52 smithi113 bash[93499]: ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-x 2024-04-22T07:25:52.898 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-22T07:25:52.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:25:52.932 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-22T07:25:52.932 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-22T07:25:52.932 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.y 2024-04-22T07:25:53.276 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:25:52 smithi133 systemd[1]: Stopping Ceph mgr.y for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:53.539 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:25:53 smithi133 podman[67822]: 2024-04-22 07:25:53.279779845 +0000 UTC m=+0.150148899 container died 2e7396d59a130640786ff1cda5b1495df59cd106369799d7998644d26457873c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T07:25:53.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:25:53 smithi133 podman[67822]: 2024-04-22 07:25:53.741960505 +0000 UTC m=+0.612329560 container cleanup 2e7396d59a130640786ff1cda5b1495df59cd106369799d7998644d26457873c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:25:53.814 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Apr 22 07:25:53 smithi133 bash[67822]: ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-y 2024-04-22T07:25:54.168 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-22T07:25:54.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:25:54.202 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-22T07:25:54.202 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-22T07:25:54.202 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.z 2024-04-22T07:25:54.523 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:25:54 smithi154 systemd[1]: Stopping Ceph mgr.z for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:54.803 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:25:54 smithi154 podman[69687]: 2024-04-22 07:25:54.526124149 +0000 UTC m=+0.129556234 container died 45a3c77739e9f55fdb2d9dc8421be8b0f517f3f627872bcc3b1d2bd066776095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-22T07:25:55.065 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:25:54 smithi154 podman[69687]: 2024-04-22 07:25:54.871769641 +0000 UTC m=+0.475201725 container cleanup 45a3c77739e9f55fdb2d9dc8421be8b0f517f3f627872bcc3b1d2bd066776095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416) 2024-04-22T07:25:55.066 INFO:journalctl@ceph.mgr.z.smithi154.stdout:Apr 22 07:25:54 smithi154 bash[69687]: ceph-81035712-0076-11ef-bc93-c7b262605968-mgr-z 2024-04-22T07:25:55.293 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-22T07:25:55.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:25:55.327 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-22T07:25:55.327 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-22T07:25:55.328 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.0 2024-04-22T07:25:55.615 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:25:55 smithi113 systemd[1]: Stopping Ceph osd.0 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:25:56.084 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:25:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0[40027]: 2024-04-22T07:25:55.618+0000 7f9614275700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:25:56.084 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:25:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0[40027]: 2024-04-22T07:25:55.618+0000 7f9614275700 -1 osd.0 106 *** Got signal Terminated *** 2024-04-22T07:25:56.084 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:25:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0[40027]: 2024-04-22T07:25:55.618+0000 7f9614275700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:01.359 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:01 smithi113 podman[93622]: 2024-04-22 07:26:01.100923959 +0000 UTC m=+5.587676413 container died c2f278c3b5da2c88cbabce2cb1bccbebefc158f3f1c959850173ea9fec885842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-22T07:26:01.627 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:01 smithi113 podman[93622]: 2024-04-22 07:26:01.443339141 +0000 UTC m=+5.930091593 container cleanup c2f278c3b5da2c88cbabce2cb1bccbebefc158f3f1c959850173ea9fec885842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-22T07:26:01.628 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:01 smithi113 bash[93622]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0 2024-04-22T07:26:02.058 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:01 smithi113 podman[93639]: 2024-04-22 07:26:01.628025945 +0000 UTC m=+0.515983837 container remove c2f278c3b5da2c88cbabce2cb1bccbebefc158f3f1c959850173ea9fec885842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-22T07:26:02.315 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:02 smithi113 podman[93730]: 2024-04-22 07:26:01.964637198 +0000 UTC m=+0.017892562 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:02.315 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:02 smithi113 podman[93730]: 2024-04-22 07:26:02.079002473 +0000 UTC m=+0.132257835 container create 8c8c64ed4e762421b105cabb7790e8bbfb7cc8e3d96e406c6f173efc7331da92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:02.315 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:02 smithi113 podman[93730]: 2024-04-22 07:26:02.314621118 +0000 UTC m=+0.367876480 container init 8c8c64ed4e762421b105cabb7790e8bbfb7cc8e3d96e406c6f173efc7331da92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:02.584 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:02 smithi113 podman[93730]: 2024-04-22 07:26:02.317788034 +0000 UTC m=+0.371043405 container start 8c8c64ed4e762421b105cabb7790e8bbfb7cc8e3d96e406c6f173efc7331da92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-22T07:26:02.584 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:02 smithi113 podman[93730]: 2024-04-22 07:26:02.423110044 +0000 UTC m=+0.476365424 container attach 8c8c64ed4e762421b105cabb7790e8bbfb7cc8e3d96e406c6f173efc7331da92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-22T07:26:02.923 INFO:journalctl@ceph.osd.0.smithi113.stdout:Apr 22 07:26:02 smithi113 podman[93730]: 2024-04-22 07:26:02.614689366 +0000 UTC m=+0.667944729 container died 8c8c64ed4e762421b105cabb7790e8bbfb7cc8e3d96e406c6f173efc7331da92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True) 2024-04-22T07:26:03.226 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.0.service' 2024-04-22T07:26:03.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:03.266 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-22T07:26:03.266 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-22T07:26:03.266 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.1 2024-04-22T07:26:03.564 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:03 smithi133 systemd[1]: Stopping Ceph osd.1 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:04.064 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1[35803]: 2024-04-22T07:26:03.596+0000 7ff38eecb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:04.064 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1[35803]: 2024-04-22T07:26:03.596+0000 7ff38eecb700 -1 osd.1 106 *** Got signal Terminated *** 2024-04-22T07:26:04.064 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1[35803]: 2024-04-22T07:26:03.596+0000 7ff38eecb700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:09.212 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:08 smithi133 podman[67947]: 2024-04-22 07:26:08.9370774 +0000 UTC m=+5.468940463 container died 7df7f5889aa0a7b1f02ee78ab1f6f0be2f4bfbeee254ea6af0136a5e1e14707a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T07:26:09.563 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:09 smithi133 podman[67947]: 2024-04-22 07:26:09.305236164 +0000 UTC m=+5.837099226 container cleanup 7df7f5889aa0a7b1f02ee78ab1f6f0be2f4bfbeee254ea6af0136a5e1e14707a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-22T07:26:09.564 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:09 smithi133 bash[67947]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1 2024-04-22T07:26:09.906 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:09 smithi133 podman[67964]: 2024-04-22 07:26:09.609027707 +0000 UTC m=+0.671244185 container remove 7df7f5889aa0a7b1f02ee78ab1f6f0be2f4bfbeee254ea6af0136a5e1e14707a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-22T07:26:10.193 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:09 smithi133 podman[68059]: 2024-04-22 07:26:09.812317613 +0000 UTC m=+0.017968968 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:10.193 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:09 smithi133 podman[68059]: 2024-04-22 07:26:09.909633311 +0000 UTC m=+0.115284659 container create bd652a30b57e4f289bda377df2e04a656797c692bcde89dcbfb7935272c19a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:10.194 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:10 smithi133 podman[68059]: 2024-04-22 07:26:10.095533349 +0000 UTC m=+0.301184694 container init bd652a30b57e4f289bda377df2e04a656797c692bcde89dcbfb7935272c19a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-22T07:26:10.194 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:10 smithi133 podman[68059]: 2024-04-22 07:26:10.098585509 +0000 UTC m=+0.304236855 container start bd652a30b57e4f289bda377df2e04a656797c692bcde89dcbfb7935272c19a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T07:26:10.194 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:10 smithi133 podman[68059]: 2024-04-22 07:26:10.196749164 +0000 UTC m=+0.402400527 container attach bd652a30b57e4f289bda377df2e04a656797c692bcde89dcbfb7935272c19a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-22T07:26:10.564 INFO:journalctl@ceph.osd.1.smithi133.stdout:Apr 22 07:26:10 smithi133 podman[68059]: 2024-04-22 07:26:10.400440753 +0000 UTC m=+0.606092098 container died bd652a30b57e4f289bda377df2e04a656797c692bcde89dcbfb7935272c19a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS) 2024-04-22T07:26:11.044 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.1.service' 2024-04-22T07:26:11.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:11.083 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-22T07:26:11.084 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-22T07:26:11.084 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.2 2024-04-22T07:26:11.397 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:11 smithi154 systemd[1]: Stopping Ceph osd.2 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:11.815 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2[36997]: 2024-04-22T07:26:11.399+0000 7f4d66235700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:11.816 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2[36997]: 2024-04-22T07:26:11.399+0000 7f4d66235700 -1 osd.2 106 *** Got signal Terminated *** 2024-04-22T07:26:11.816 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2[36997]: 2024-04-22T07:26:11.399+0000 7f4d66235700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:17.147 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:16 smithi154 podman[69815]: 2024-04-22 07:26:16.843190082 +0000 UTC m=+5.559880356 container died de7c281fd5933191da60f209fc04912c77b90d3a7c78760271a7677239f72e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:26:17.422 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:17 smithi154 podman[69815]: 2024-04-22 07:26:17.249474935 +0000 UTC m=+5.966165216 container cleanup de7c281fd5933191da60f209fc04912c77b90d3a7c78760271a7677239f72e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T07:26:17.423 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:17 smithi154 bash[69815]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2 2024-04-22T07:26:17.806 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:17 smithi154 podman[69832]: 2024-04-22 07:26:17.425703115 +0000 UTC m=+0.579433908 container remove de7c281fd5933191da60f209fc04912c77b90d3a7c78760271a7677239f72e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:18.065 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:17 smithi154 podman[69927]: 2024-04-22 07:26:17.809811872 +0000 UTC m=+0.092500317 container create 189d1292ac1eddb3410a65535e5b1454e2e2823378fe8a339a6470f4fa0ead76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS) 2024-04-22T07:26:18.066 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:17 smithi154 podman[69927]: 2024-04-22 07:26:17.735538931 +0000 UTC m=+0.018227404 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:18.066 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:18 smithi154 podman[69927]: 2024-04-22 07:26:18.029331444 +0000 UTC m=+0.312019890 container init 189d1292ac1eddb3410a65535e5b1454e2e2823378fe8a339a6470f4fa0ead76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:26:18.066 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:18 smithi154 podman[69927]: 2024-04-22 07:26:18.032880731 +0000 UTC m=+0.315569184 container start 189d1292ac1eddb3410a65535e5b1454e2e2823378fe8a339a6470f4fa0ead76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-22T07:26:18.327 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:18 smithi154 podman[69927]: 2024-04-22 07:26:18.121066393 +0000 UTC m=+0.403754842 container attach 189d1292ac1eddb3410a65535e5b1454e2e2823378fe8a339a6470f4fa0ead76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:18.670 INFO:journalctl@ceph.osd.2.smithi154.stdout:Apr 22 07:26:18 smithi154 podman[69927]: 2024-04-22 07:26:18.329941733 +0000 UTC m=+0.612630188 container died 189d1292ac1eddb3410a65535e5b1454e2e2823378fe8a339a6470f4fa0ead76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-22T07:26:18.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.2.service' 2024-04-22T07:26:19.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:19.018 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-22T07:26:19.018 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-22T07:26:19.018 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.3 2024-04-22T07:26:19.334 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:19 smithi113 systemd[1]: Stopping Ceph osd.3 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:19.584 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:19 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:26:19.335+0000 7f269cc0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:19.584 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:19 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:26:19.335+0000 7f269cc0b700 -1 osd.3 106 *** Got signal Terminated *** 2024-04-22T07:26:19.584 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:19 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:26:19.335+0000 7f269cc0b700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:22.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:22 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:22.369+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:22.834 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:22 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:26:22.455+0000 7f26991ff700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.309974+0000 front 2024-04-22T07:25:58.310010+0000 (oldest deadline 2024-04-22T07:26:22.410131+0000) 2024-04-22T07:26:23.315 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:22.885+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:23.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:23 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:23.060+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:23.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:23 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:23.359+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:23.834 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:23 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3[46739]: 2024-04-22T07:26:23.449+0000 7f26991ff700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.309974+0000 front 2024-04-22T07:25:58.310010+0000 (oldest deadline 2024-04-22T07:26:22.410131+0000) 2024-04-22T07:26:23.834 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:23 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:23.598+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:24.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:23.855+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:24.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:23.765+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:24.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:23.973+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:24.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:24.023+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:24.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:24 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:24.382+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:25.008 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:24 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:24.621+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:25.009 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:24 smithi113 podman[93851]: 2024-04-22 07:26:24.825674592 +0000 UTC m=+5.614475766 container died b10e4c7aa2eabdf7cff9f44598b190ffdf90dbcbc15b254b32fa1e001b25dc61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T07:26:25.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:24.979+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:25.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:24.748+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:25.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:24.812+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:25.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:25 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:25.011+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:25.334 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:25 smithi113 podman[93851]: 2024-04-22 07:26:25.177506432 +0000 UTC m=+5.966307614 container cleanup b10e4c7aa2eabdf7cff9f44598b190ffdf90dbcbc15b254b32fa1e001b25dc61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-22T07:26:25.334 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:25 smithi113 bash[93851]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3 2024-04-22T07:26:25.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:25 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:25.410+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:25.834 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:25 smithi113 podman[93868]: 2024-04-22 07:26:25.495580896 +0000 UTC m=+0.671072291 container remove b10e4c7aa2eabdf7cff9f44598b190ffdf90dbcbc15b254b32fa1e001b25dc61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-22T07:26:25.834 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:25 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:25.604+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:26.065 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:25.774+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:26.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:25.836+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:26.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:25.999+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:26.145 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:25 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:25.966+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:26.145 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:25 smithi113 podman[93958]: 2024-04-22 07:26:25.758524459 +0000 UTC m=+0.026459677 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:26.145 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:25 smithi113 podman[93958]: 2024-04-22 07:26:25.8798946 +0000 UTC m=+0.147829884 container create 3d0d8071145c3eb2a48873f63755bafba73baa257b4292a7c8e017817d66f008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True) 2024-04-22T07:26:26.146 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:26 smithi113 podman[93958]: 2024-04-22 07:26:26.148726576 +0000 UTC m=+0.416661794 container init 3d0d8071145c3eb2a48873f63755bafba73baa257b4292a7c8e017817d66f008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:26:26.441 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:26 smithi113 podman[93958]: 2024-04-22 07:26:26.152075083 +0000 UTC m=+0.420010301 container start 3d0d8071145c3eb2a48873f63755bafba73baa257b4292a7c8e017817d66f008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=) 2024-04-22T07:26:26.441 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:26 smithi113 podman[93958]: 2024-04-22 07:26:26.232093696 +0000 UTC m=+0.500028926 container attach 3d0d8071145c3eb2a48873f63755bafba73baa257b4292a7c8e017817d66f008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-22T07:26:26.732 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:26 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:26.604+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:26.732 INFO:journalctl@ceph.osd.3.smithi113.stdout:Apr 22 07:26:26 smithi113 podman[93958]: 2024-04-22 07:26:26.444699627 +0000 UTC m=+0.712634855 container died 3d0d8071145c3eb2a48873f63755bafba73baa257b4292a7c8e017817d66f008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:26:26.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:26 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:26.449+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:27.017 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.3.service' 2024-04-22T07:26:27.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:27.057 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-22T07:26:27.057 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-22T07:26:27.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.4 2024-04-22T07:26:27.267 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:26 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:26.997+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:27.315 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:26.823+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:27.316 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:26.869+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:27.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:27 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:27.021+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:27.373 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:27 smithi133 systemd[1]: Stopping Ceph osd.4 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:27.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:27 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:27.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:27.814 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:27 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:26:27.375+0000 7f49640f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:27.814 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:27 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:26:27.375+0000 7f49640f0700 -1 osd.4 106 *** Got signal Terminated *** 2024-04-22T07:26:27.814 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:27 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:26:27.375+0000 7f49640f0700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:27.960 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:27.652+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:28.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:27 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:27.988+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:28.316 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:27 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:27.836+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:28.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:27 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:27.824+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:28.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:27 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:27.962+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:28.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:28 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:28.519+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:29.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:28 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:28.919+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:29.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:28 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:28.677+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:29.316 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:28 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:28.865+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:29.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:29 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:29.015+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:29.317 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:28 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:28.837+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:30.029 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:29 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:29.872+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:30.029 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:29 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:29.872+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:30.030 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:29 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:29.899+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:30.030 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:29 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:29.899+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:30.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:29 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:29.567+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:30.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:29 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:29.929+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:30.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:29 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:29.702+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:30.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:29 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:29.702+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:30.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:30 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:30.031+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:30.563 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:30 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:26:30.188+0000 7f49606e4700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.303784+0000 front 2024-04-22T07:26:07.303795+0000 (oldest deadline 2024-04-22T07:26:29.603901+0000) 2024-04-22T07:26:31.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:30 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:30.610+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:31.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:30 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:30.905+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:31.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:30 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:30.657+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:31.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:30 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:30.657+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:31.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:30 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:30.991+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:31.316 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:30 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:30.924+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:31.316 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:30 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:30.924+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:31.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:30 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:30.825+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:31.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:30 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:30.825+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:31.563 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:31 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:26:31.192+0000 7f49606e4700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.303784+0000 front 2024-04-22T07:26:07.303795+0000 (oldest deadline 2024-04-22T07:26:29.603901+0000) 2024-04-22T07:26:31.948 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:31 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:31.610+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:31.948 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:31 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:31.610+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:32.063 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:31 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:31.652+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:32.068 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:31.964+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:32.068 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:31.786+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:32.068 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:31.786+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:32.068 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:31.891+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:32.068 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:31 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:31.891+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:32.336 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:31 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:31.950+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:32.564 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:32 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4[42726]: 2024-04-22T07:26:32.221+0000 7f49606e4700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.303784+0000 front 2024-04-22T07:26:07.303795+0000 (oldest deadline 2024-04-22T07:26:29.603901+0000) 2024-04-22T07:26:32.933 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:32 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:32.631+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:32.933 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:32 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:32.631+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:32.999 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:32 smithi133 podman[68176]: 2024-04-22 07:26:32.742065943 +0000 UTC m=+5.475860782 container died 65bb9d160d6dc8f06f34c7d8ce13f309acaf908d0ab182f72d144ec18d40bfa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux ) 2024-04-22T07:26:32.999 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:32 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:32.787+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:32.999 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:32 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:32.644+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:32.999 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:32 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:32.644+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:33.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:33.007+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:33.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:32 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:32.885+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:33.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:32 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:32.885+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:33.067 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:32 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:32.799+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:33.067 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:32 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:32.799+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:33.314 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:33 smithi133 podman[68176]: 2024-04-22 07:26:33.100444649 +0000 UTC m=+5.834239483 container cleanup 65bb9d160d6dc8f06f34c7d8ce13f309acaf908d0ab182f72d144ec18d40bfa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T07:26:33.314 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:33 smithi133 bash[68176]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4 2024-04-22T07:26:33.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:32 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:32.934+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:33.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:32 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:32.934+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:33.782 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:33 smithi133 podman[68194]: 2024-04-22 07:26:33.493660645 +0000 UTC m=+0.753352879 container remove 65bb9d160d6dc8f06f34c7d8ce13f309acaf908d0ab182f72d144ec18d40bfa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T07:26:33.783 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:33 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:33.620+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:33.783 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:33 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:33.621+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:33.919 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:33.634+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:33.919 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:33.634+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:34.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:33 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:33.784+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:34.064 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:33 smithi133 podman[68283]: 2024-04-22 07:26:33.770997611 +0000 UTC m=+0.017875818 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:34.064 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:33 smithi133 podman[68283]: 2024-04-22 07:26:33.886058394 +0000 UTC m=+0.132936632 container create 7ef462da0747388ab09dbc0b9973347d24028c4e8a7e69a0a943d6836b65e446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-22T07:26:34.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:33.975+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:34.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:33.773+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:34.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:33.773+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:34.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:33.858+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:34.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:33 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:33.858+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:34.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:33.921+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:34.334 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:33 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:33.921+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:34.382 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:34 smithi133 podman[68283]: 2024-04-22 07:26:34.089888016 +0000 UTC m=+0.336766220 container init 7ef462da0747388ab09dbc0b9973347d24028c4e8a7e69a0a943d6836b65e446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:34.383 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:34 smithi133 podman[68283]: 2024-04-22 07:26:34.09334684 +0000 UTC m=+0.340225041 container start 7ef462da0747388ab09dbc0b9973347d24028c4e8a7e69a0a943d6836b65e446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS) 2024-04-22T07:26:34.383 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:34 smithi133 podman[68283]: 2024-04-22 07:26:34.163521112 +0000 UTC m=+0.410399331 container attach 7ef462da0747388ab09dbc0b9973347d24028c4e8a7e69a0a943d6836b65e446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-22T07:26:34.679 INFO:journalctl@ceph.osd.4.smithi133.stdout:Apr 22 07:26:34 smithi133 podman[68283]: 2024-04-22 07:26:34.38610699 +0000 UTC m=+0.632985182 container died 7ef462da0747388ab09dbc0b9973347d24028c4e8a7e69a0a943d6836b65e446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-22T07:26:34.680 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:34 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:34.597+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:34.680 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:34 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:34.597+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:34.980 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:34 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:34.793+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:35.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.4.service' 2024-04-22T07:26:35.004 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:34 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:34.878+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:35.004 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:34 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:34.878+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:35.005 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:34 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:34.734+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:35.005 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:34 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:34.734+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:35.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:35.040 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-22T07:26:35.040 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-22T07:26:35.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.5 2024-04-22T07:26:35.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:34 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:34.873+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:35.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:34 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:34.873+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:35.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:34 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:34.661+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:35.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:34 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:34.661+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:35.316 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:35 smithi154 systemd[1]: Stopping Ceph osd.5 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:35.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:35.007+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:35.706 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:35.346+0000 7f82ed3c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:35.706 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:35.346+0000 7f82ed3c7700 -1 osd.5 106 *** Got signal Terminated *** 2024-04-22T07:26:35.707 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:35.346+0000 7f82ed3c7700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:35.965 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:35.709+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:35.965 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:35.709+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:35.965 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:35.871+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:35.965 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:35.871+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:36.063 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:35 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:35.816+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:36.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:35 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:35.643+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:36.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:35 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:35.643+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:36.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:35 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:35.874+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:36.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:35 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:35.874+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:36.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:35 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:35.681+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:36.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:35 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:35.681+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:36.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:35 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:35.967+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:36.986 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:36.871+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:36.986 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:36.871+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:36.986 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:36.708+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:36.986 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:36.708+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:37.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:36.799+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:37.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:36.674+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:37.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:36 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:36.674+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:37.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:36 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:36.846+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:37.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:36 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:36.846+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:37.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:36 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:36.691+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:37.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:36 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:36.691+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:37.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:36 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:36.988+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:37.972 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:37 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:37.847+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:37.972 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:37 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:37.847+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:37.972 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:37 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:37.712+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:37.972 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:37 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:37.712+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:38.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:37 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:37.788+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:38.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:37 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:37.788+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:38.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:37 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:37.660+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:38.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:37 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:37.660+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:38.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:37 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:37.889+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:38.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:37 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:37.889+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:38.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:37 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:37.680+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:38.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:37 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:37.680+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:38.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:37 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:37.974+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:39.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:38 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:38.790+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:39.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:38 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:38.790+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:39.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:38 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:38.618+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:39.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:38 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:38.618+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:39.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:38 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:38.932+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:39.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:38 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:38.932+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:39.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:38 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:38.815+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:39.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:38 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:38.815+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:39.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:38 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:38.759+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:39.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:38 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:38.759+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:39.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:38 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:38.881+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:39.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:38 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:38.881+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:39.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:38 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:38.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:39.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:38 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:38.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:39.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:38 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:38.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:39.834 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:39.576+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:39.834 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:39.576+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:40.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:39.853+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:56.804383+0000 front 2024-04-22T07:25:56.804476+0000 (oldest deadline 2024-04-22T07:26:22.704497+0000) 2024-04-22T07:26:40.066 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5[43954]: 2024-04-22T07:26:39.853+0000 7f82e99bb700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.306238+0000 front 2024-04-22T07:26:07.306179+0000 (oldest deadline 2024-04-22T07:26:29.006274+0000) 2024-04-22T07:26:40.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:39.783+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:40.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:39.783+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:40.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:39.896+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:40.067 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:39 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:39.896+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:40.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:39 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:39.892+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:40.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:39 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:39.892+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:40.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:39 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:39.741+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:40.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:39 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:39.741+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:40.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:39 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:39.741+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:40.314 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:39.837+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:40.314 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:39 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:39.837+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:40.923 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:40 smithi154 podman[70044]: 2024-04-22 07:26:40.670890619 +0000 UTC m=+5.431853546 container died e07172a6ec870abd8bb40dec7876e547826e19f36d8baa7497ba8b2396f7ddc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:26:40.924 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:40 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:40.875+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:40.924 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:40 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:40.875+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:40.924 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:40 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:40.813+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:40.924 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:40 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:40.813+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:40.924 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:40 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:40.813+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:41.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:40 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:40.813+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:41.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:40 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:40.813+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:41.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:40 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:40.588+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:41.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:40 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:40.588+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:41.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:40.895+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:41.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:40.895+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:41.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:40.895+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:41.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:40.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:41.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:40.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:41.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:40.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:41.291 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 podman[70044]: 2024-04-22 07:26:41.026206784 +0000 UTC m=+5.787169710 container cleanup e07172a6ec870abd8bb40dec7876e547826e19f36d8baa7497ba8b2396f7ddc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T07:26:41.291 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 bash[70044]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5 2024-04-22T07:26:41.565 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 podman[70059]: 2024-04-22 07:26:41.294114539 +0000 UTC m=+0.624003757 container remove e07172a6ec870abd8bb40dec7876e547826e19f36d8baa7497ba8b2396f7ddc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux ) 2024-04-22T07:26:41.905 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 podman[70153]: 2024-04-22 07:26:41.548582073 +0000 UTC m=+0.018264106 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:41.906 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 podman[70153]: 2024-04-22 07:26:41.686842585 +0000 UTC m=+0.156524647 container create d6e77a61427b2e2711aa235186e4b266dbe3784b8f35d3102165a4c85bb771ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T07:26:41.906 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:41 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:41.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:41.906 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:41 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:41.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:41.906 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:41 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:41.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:42.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:41 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:41.821+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:42.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:41 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:41.821+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:42.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:41 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:41.631+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:42.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:41 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:41.631+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:42.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:41 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:41.631+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:42.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:41 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:41.891+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:42.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:41 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:41.891+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:42.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:41 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:41.891+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:42.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:41 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:41.690+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:42.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:41 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:41.690+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:42.085 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:41 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:41.690+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:42.224 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:41 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:41.908+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:42.224 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:41 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:41.908+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:42.224 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 podman[70153]: 2024-04-22 07:26:41.930098119 +0000 UTC m=+0.399780139 container init d6e77a61427b2e2711aa235186e4b266dbe3784b8f35d3102165a4c85bb771ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:26:42.224 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:41 smithi154 podman[70153]: 2024-04-22 07:26:41.933248834 +0000 UTC m=+0.402930864 container start d6e77a61427b2e2711aa235186e4b266dbe3784b8f35d3102165a4c85bb771ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:26:42.225 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:42 smithi154 podman[70153]: 2024-04-22 07:26:42.022577205 +0000 UTC m=+0.492259226 container attach d6e77a61427b2e2711aa235186e4b266dbe3784b8f35d3102165a4c85bb771ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T07:26:42.514 INFO:journalctl@ceph.osd.5.smithi154.stdout:Apr 22 07:26:42 smithi154 podman[70153]: 2024-04-22 07:26:42.227135672 +0000 UTC m=+0.696817706 container died d6e77a61427b2e2711aa235186e4b266dbe3784b8f35d3102165a4c85bb771ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-22T07:26:42.804 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.5.service' 2024-04-22T07:26:42.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:42.841 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-22T07:26:42.841 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-22T07:26:42.842 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.6 2024-04-22T07:26:43.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:42.831+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:43.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:42.831+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:43.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:42.612+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:43.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:42.612+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:43.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:42 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:42.612+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:43.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:42.922+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:43.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:42.922+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:43.065 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:42.840+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:43.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:42.840+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:43.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:42 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:42.840+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:42.696+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:42.696+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:42.696+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:42.850+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:42.850+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:42 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:42.850+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:43.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:42 smithi113 systemd[1]: Stopping Ceph osd.6 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:43.584 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:43.136+0000 7f425fbd8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:43.584 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:43.136+0000 7f425fbd8700 -1 osd.6 106 *** Got signal Terminated *** 2024-04-22T07:26:43.584 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:43.136+0000 7f425fbd8700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:44.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:43 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:43.789+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:44.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:43 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:43.789+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:44.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:43 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:43.604+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:44.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:43 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:43.604+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:44.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:43 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:43.604+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:44.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:43.841+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:44.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:43.841+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:44.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:43.841+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:44.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:43.721+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:44.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:43.721+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:44.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:43 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:43.721+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:44.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:43 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:43.911+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:44.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:43 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:43.911+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:44.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:43 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:43.861+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:44.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:43 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:43.861+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:44.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:43 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:43.861+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:45.063 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:44.742+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:45.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:44.742+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:45.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:44.581+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:45.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:44.581+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:45.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:44 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:44.581+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:45.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:44.917+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:45.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:44.917+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:45.065 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:44.817+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:45.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:44.817+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:45.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:44 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:44.817+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:45.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:44.875+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:45.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:44.875+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:45.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:44.875+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:45.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:44.747+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:45.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:44.747+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:45.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:44 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:44.747+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:45.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:45.781+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:45.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:45.781+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:45.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:45.781+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:45.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:45.562+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:45.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:45.562+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:45.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:45 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:45.562+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:46.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:45.767+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:46.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:45.767+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:46.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:45.767+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:46.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:45.850+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:46.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:45.850+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:46.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:45.850+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:46.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:45.882+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:46.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:45.882+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:46.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:45.882+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:46.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:45.836+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:46.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:45.836+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:46.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:45 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:45.836+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:46.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:46.789+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:46.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:46.789+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:46.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:46.789+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:46.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:46.549+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:46.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:46.549+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:46.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:46 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:46.549+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:47.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:46.721+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:47.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:46.721+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:47.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:46.721+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:47.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:46.812+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:47.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:46.812+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:47.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:46.812+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:47.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:46 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:46.812+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.393083+0000 front 2024-04-22T07:26:21.393063+0000 (oldest deadline 2024-04-22T07:26:46.693247+0000) 2024-04-22T07:26:47.217 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:46.923+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:47.217 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:46.923+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:47.217 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:46.923+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:47.218 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:46.853+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:47.218 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:46.853+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:47.218 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:46.853+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:47.218 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:46 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:46.853+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:48.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:47.828+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:48.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:47.828+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:48.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:47.828+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:48.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:47.592+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:48.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:47.592+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:48.065 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:47.592+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:48.065 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:47 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:47.592+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:48.065 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:47.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:48.065 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:47.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:48.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:47.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:48.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:47.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:48.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:47.939+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:48.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:47.939+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:48.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:47 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:47.939+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:48.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:47.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:48.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:47.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:48.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:47.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:48.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:47.773+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.489119+0000 front 2024-04-22T07:25:57.489153+0000 (oldest deadline 2024-04-22T07:26:22.789280+0000) 2024-04-22T07:26:48.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:47.773+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:06.890317+0000 front 2024-04-22T07:26:06.890336+0000 (oldest deadline 2024-04-22T07:26:32.190489+0000) 2024-04-22T07:26:48.084 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:47.773+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.192137+0000 front 2024-04-22T07:26:16.192210+0000 (oldest deadline 2024-04-22T07:26:40.892043+0000) 2024-04-22T07:26:48.085 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:47 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6[53398]: 2024-04-22T07:26:47.773+0000 7f42581c4700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.393083+0000 front 2024-04-22T07:26:21.393063+0000 (oldest deadline 2024-04-22T07:26:46.693247+0000) 2024-04-22T07:26:48.718 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:48 smithi113 podman[94075]: 2024-04-22 07:26:48.469657463 +0000 UTC m=+5.440017611 container died 2fe84bf5c71860320fd73260bd319edaff740b9a290c89cc117844e3a40eb45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-22T07:26:48.718 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:48.686+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:48.719 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:48.686+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:48.719 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:48 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:48.686+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:49.000 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:48 smithi113 podman[94075]: 2024-04-22 07:26:48.794103888 +0000 UTC m=+5.764464040 container cleanup 2fe84bf5c71860320fd73260bd319edaff740b9a290c89cc117844e3a40eb45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-22T07:26:49.001 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:48 smithi113 bash[94075]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:48.819+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:48.819+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:48.819+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:48.585+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:48.585+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:48.585+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:49.064 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:48 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:48.585+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:49.315 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:48.838+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:49.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:48.838+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:49.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:48.838+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:49.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:48.838+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:49.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:48.908+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:49.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:48.908+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:49.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:48 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:48.908+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:49.333 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94093]: 2024-04-22 07:26:49.003712826 +0000 UTC m=+0.528962961 container remove 2fe84bf5c71860320fd73260bd319edaff740b9a290c89cc117844e3a40eb45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-22T07:26:49.672 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94183]: 2024-04-22 07:26:49.313670631 +0000 UTC m=+0.018337978 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:49.672 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94183]: 2024-04-22 07:26:49.43785122 +0000 UTC m=+0.142518561 container create dd7cf9353c60d5ab03b653624422ffd05f0dc2e6a31b3df8f8ae3115b14a25d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T07:26:49.813 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:49.778+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:49.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:49.778+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:49.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:49.778+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:49.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:49.536+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:49.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:49.536+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:49.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:49.536+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:49.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:49 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:49.536+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:49.968 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:49.700+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:49.968 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:49.700+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:49.968 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:49 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:49.700+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:49.968 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94183]: 2024-04-22 07:26:49.674412048 +0000 UTC m=+0.379079391 container init dd7cf9353c60d5ab03b653624422ffd05f0dc2e6a31b3df8f8ae3115b14a25d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:49.969 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94183]: 2024-04-22 07:26:49.677593989 +0000 UTC m=+0.382261333 container start dd7cf9353c60d5ab03b653624422ffd05f0dc2e6a31b3df8f8ae3115b14a25d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-22T07:26:49.969 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94183]: 2024-04-22 07:26:49.773759553 +0000 UTC m=+0.478426902 container attach dd7cf9353c60d5ab03b653624422ffd05f0dc2e6a31b3df8f8ae3115b14a25d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-22T07:26:50.248 INFO:journalctl@ceph.osd.6.smithi113.stdout:Apr 22 07:26:49 smithi113 podman[94183]: 2024-04-22 07:26:49.971612414 +0000 UTC m=+0.676279764 container died dd7cf9353c60d5ab03b653624422ffd05f0dc2e6a31b3df8f8ae3115b14a25d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:49.884+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:49.884+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:49.884+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:49.825+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:49.825+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:49.825+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:50.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:49 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:49.825+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:50.524 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.6.service' 2024-04-22T07:26:50.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:50.562 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-22T07:26:50.562 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-22T07:26:50.563 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.7 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 systemd[1]: Stopping Ceph osd.7 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:50.774+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:50.774+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:50.774+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:50.535+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:50.535+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:50.535+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:50.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:50.535+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:50.834 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:50.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:50.834 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:50.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:50.834 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:50.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:50.834 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:50 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:50.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:51.314 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:50.860+0000 7efeb58f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:51.314 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:50.860+0000 7efeb58f2700 -1 osd.7 106 *** Got signal Terminated *** 2024-04-22T07:26:51.314 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:50.860+0000 7efeb58f2700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:51.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:50.919+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:51.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:50.919+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:51.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:50.919+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:51.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:50.820+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:51.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:50.820+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:51.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:50.820+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:51.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:50 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:50.820+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:51.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:51.520+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:51.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:51.520+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:51.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:51.520+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:51.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:51.520+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:52.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:51.772+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:52.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:51.772+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:52.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:51 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:51.772+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:52.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:51.882+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:52.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:51.882+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:52.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:51.882+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:52.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:51.781+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:52.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:51.781+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:52.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:51.781+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:52.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:51 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:51.781+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:52.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:51.751+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:52.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:51.751+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:52.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:51.751+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:52.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:51 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:51.751+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:52.734 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:52.484+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:52.734 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:52.484+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:52.734 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:52.484+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:52.734 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:52.484+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:53.012 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:52.736+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:53.013 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:52.736+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:53.013 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:52 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:52.736+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:52.892+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:52.892+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:52.892+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:52.736+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:52.736+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:52.736+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:53.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:52 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:52.736+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:53.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:52.734+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:53.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:52.734+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:53.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:52.734+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:53.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:52 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:52.734+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:53.724 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:53.452+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:53.724 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:53.452+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:53.724 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:53.452+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:53.724 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:53.452+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:54.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:53.726+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:54.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:53.726+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:54.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:53.726+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:54.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:53.726+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:28.343720+0000 front 2024-04-22T07:26:28.343638+0000 (oldest deadline 2024-04-22T07:26:53.643706+0000) 2024-04-22T07:26:54.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:53.901+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:54.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:53.901+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:54.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:53.901+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:54.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:53.748+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:54.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:53.748+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:54.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:53.748+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:54.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:53 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:53.748+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:54.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:53.728+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:54.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:53.728+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:54.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:53.728+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:54.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:53 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:53.728+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:54.768 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:54.423+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:54.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:54.423+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:54.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:54.423+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:54.769 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:54.423+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:55.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:54.770+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:55.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:54.770+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:55.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:54.770+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:55.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:54 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:54.770+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:28.343720+0000 front 2024-04-22T07:26:28.343638+0000 (oldest deadline 2024-04-22T07:26:53.643706+0000) 2024-04-22T07:26:55.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:54.893+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:55.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:54.893+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:55.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:54.893+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:55.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:54.783+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:55.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:54.783+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:55.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:54.783+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:55.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:54 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:54.783+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:55.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:54.680+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:55.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:54.680+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:55.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:54.680+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:55.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:54 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:54.680+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:55.810 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:55.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:55.810 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:55.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:55.810 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:55.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:55.810 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:55.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:56.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:55.812+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.340384+0000 front 2024-04-22T07:26:07.340413+0000 (oldest deadline 2024-04-22T07:26:32.040497+0000) 2024-04-22T07:26:56.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:55.812+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.141374+0000 front 2024-04-22T07:26:16.141387+0000 (oldest deadline 2024-04-22T07:26:37.241517+0000) 2024-04-22T07:26:56.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:55.812+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:20.742453+0000 front 2024-04-22T07:26:20.742600+0000 (oldest deadline 2024-04-22T07:26:44.842552+0000) 2024-04-22T07:26:56.064 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:55 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7[49460]: 2024-04-22T07:26:55.812+0000 7efeadede700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:28.343720+0000 front 2024-04-22T07:26:28.343638+0000 (oldest deadline 2024-04-22T07:26:53.643706+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:55.892+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:55.892+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:55.892+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:55.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:55.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:55.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:56.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:55 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:55.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:56.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:55.685+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:56.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:55.685+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:56.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:55.685+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:56.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:55.685+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:56.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:55 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:55.685+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:26:56.508 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:56 smithi133 podman[68388]: 2024-04-22 07:26:56.199788972 +0000 UTC m=+5.443778535 container died a798f39d6140d4b4d229b942e117f14fac09a74fab9499b9dfabd54c946fb347 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:26:56.508 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:56 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:56.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:56.508 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:56 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:56.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:56.508 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:56 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:56.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:56.508 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:56 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:56.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:56.508 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:56 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:56.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:26:56.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:56 smithi133 podman[68388]: 2024-04-22 07:26:56.601461064 +0000 UTC m=+5.845450627 container cleanup a798f39d6140d4b4d229b942e117f14fac09a74fab9499b9dfabd54c946fb347 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T07:26:56.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:56 smithi133 bash[68388]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7 2024-04-22T07:26:57.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:56.871+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:57.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:56.871+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:57.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:56.871+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:57.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:56.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:57.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:56.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:57.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:56.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:57.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:56 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:56.802+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:57.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:56 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:56.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:57.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:56 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:56.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:57.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:56 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:56.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:57.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:56 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:56.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:57.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:56 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:56.707+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:26:57.116 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:56 smithi133 podman[68418]: 2024-04-22 07:26:56.844191126 +0000 UTC m=+0.645622162 container remove a798f39d6140d4b4d229b942e117f14fac09a74fab9499b9dfabd54c946fb347 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:26:57.402 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:57 smithi133 podman[68511]: 2024-04-22 07:26:57.119968934 +0000 UTC m=+0.097178877 container create 294bfa0f8b60dd728b9582fefbd6af3874e82882f466c8fb5b444f53bc9abf3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T07:26:57.402 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:57 smithi133 podman[68511]: 2024-04-22 07:26:57.040686456 +0000 UTC m=+0.017896369 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:26:57.402 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:57 smithi133 podman[68511]: 2024-04-22 07:26:57.33091483 +0000 UTC m=+0.308124757 container init 294bfa0f8b60dd728b9582fefbd6af3874e82882f466c8fb5b444f53bc9abf3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T07:26:57.403 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:57 smithi133 podman[68511]: 2024-04-22 07:26:57.334559816 +0000 UTC m=+0.311769720 container start 294bfa0f8b60dd728b9582fefbd6af3874e82882f466c8fb5b444f53bc9abf3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5) 2024-04-22T07:26:57.403 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:57 smithi133 podman[68511]: 2024-04-22 07:26:57.405374814 +0000 UTC m=+0.382584737 container attach 294bfa0f8b60dd728b9582fefbd6af3874e82882f466c8fb5b444f53bc9abf3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:26:57.814 INFO:journalctl@ceph.osd.7.smithi133.stdout:Apr 22 07:26:57 smithi133 podman[68511]: 2024-04-22 07:26:57.626650349 +0000 UTC m=+0.603860249 container died 294bfa0f8b60dd728b9582fefbd6af3874e82882f466c8fb5b444f53bc9abf3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:26:57.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:57.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:57.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:57.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:57.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:57.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:57.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:57.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:57.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:57.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:26:58.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:57.902+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:58.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:57.902+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:58.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:57.902+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:58.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:57.810+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:58.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:57.810+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:58.067 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:57.810+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:58.067 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:57.810+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:58.067 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:57 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:57.810+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:26:58.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:57.754+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:58.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:57.754+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:58.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:57.754+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:58.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:57.754+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:58.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:57 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:57.754+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:26:58.261 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.7.service' 2024-04-22T07:26:58.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:26:58.301 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-22T07:26:58.301 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-22T07:26:58.301 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.8 2024-04-22T07:26:58.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:58 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:58.388+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:58.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:58 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:58.388+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:58.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:58 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:58.388+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:58.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:58 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:58.388+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:58.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:58 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:58.388+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:26:58.601 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 systemd[1]: Stopping Ceph osd.8 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.602+0000 7efd6357c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.602+0000 7efd6357c700 -1 osd.8 106 *** Got signal Terminated *** 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.602+0000 7efd6357c700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:58.765+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:58.765+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:58.765+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:58.765+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:26:58.916 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:58.765+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:26:59.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:58 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:58.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:26:59.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:58 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:58.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:26:59.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:58 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:58.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:26:59.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:58 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:58.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:26:59.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:58 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:58.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:26:59.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.917+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:26:59.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.917+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:26:59.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.917+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:26:59.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:58 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:58.917+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.199182+0000 front 2024-04-22T07:26:32.199171+0000 (oldest deadline 2024-04-22T07:26:58.098990+0000) 2024-04-22T07:26:59.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:59.375+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:26:59.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:59.375+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:26:59.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:59.375+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:26:59.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:59.375+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:26:59.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:26:59 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:26:59.375+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:59.935+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:59.935+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:59.935+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:26:59.935+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.199182+0000 front 2024-04-22T07:26:32.199171+0000 (oldest deadline 2024-04-22T07:26:58.098990+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:59.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:59.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:59.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:59.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:00.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:26:59 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:26:59.804+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:00.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:59.745+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:00.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:59.745+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:00.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:59.745+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:00.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:59.745+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:00.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:26:59 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:26:59.745+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:00.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:00 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:00.390+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:00.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:00 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:00.390+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:00.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:00 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:00.390+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:00.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:00 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:00.390+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:00.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:00 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:00.390+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:01.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:00.740+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:01.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:00.740+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:01.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:00.740+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:01.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:00.740+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:01.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:00 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:00.740+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:01.315 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:00.927+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:00.927+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:00.927+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:00.927+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.199182+0000 front 2024-04-22T07:26:32.199171+0000 (oldest deadline 2024-04-22T07:26:58.098990+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:00.842+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:00.842+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:00.842+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:00.842+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:01.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:00 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:00.842+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:01.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:01 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:01.434+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:01.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:01 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:01.434+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:01.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:01 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:01.434+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:01.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:01 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:01.434+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:01.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:01 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:01.434+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:02.065 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:01.894+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:01.894+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:01.894+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:01.894+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.199182+0000 front 2024-04-22T07:26:32.199171+0000 (oldest deadline 2024-04-22T07:26:58.098990+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:01.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:01.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:01.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:01.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:02.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:01 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:01.812+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:02.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:01.762+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:02.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:01.762+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:02.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:01.762+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:02.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:01.762+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:02.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:01 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:01.762+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:02.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:02.409+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:02.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:02.409+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:02.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:02.409+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:02.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:02.409+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:02.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:02.409+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:02.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:02.409+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:02.985 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:02 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:02.715+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:02.986 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:02 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:02.715+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:02.986 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:02 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:02.715+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:02.986 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:02 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:02.715+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:02.986 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:02 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:02.715+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:02.986 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:02 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:02.715+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:02.890+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:57.793637+0000 front 2024-04-22T07:25:57.793572+0000 (oldest deadline 2024-04-22T07:26:23.693671+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:02.890+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.096237+0000 front 2024-04-22T07:26:14.096259+0000 (oldest deadline 2024-04-22T07:26:38.796436+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:02.890+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:24.097650+0000 front 2024-04-22T07:26:24.097623+0000 (oldest deadline 2024-04-22T07:26:45.197453+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:02.890+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.199182+0000 front 2024-04-22T07:26:32.199171+0000 (oldest deadline 2024-04-22T07:26:58.098990+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8[50919]: 2024-04-22T07:27:02.890+0000 7efd5fb70700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.099311+0000 front 2024-04-22T07:26:38.099360+0000 (oldest deadline 2024-04-22T07:27:02.199395+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:02.839+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:02.839+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:02.839+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:02.839+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:03.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:02 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:02.839+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:03.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:03.428+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:03.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:03.428+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:03.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:03.428+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:03.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:03.428+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:03.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:03.428+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:03.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:03 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:03.428+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:04.002 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:03 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:03.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:04.002 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:03 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:03.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:04.002 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:03 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:03.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:04.002 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:03 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:03.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:04.002 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:03 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:03.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:04.002 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:03 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:03.708+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:04.260 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:04 smithi154 podman[70269]: 2024-04-22 07:27:04.019021043 +0000 UTC m=+5.523883767 container died 71855abbbcdee9c2898a2af10983d7dd2118bdd5841159ec9f64b56cb9feb776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-22T07:27:04.260 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:03.852+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:04.260 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:03.852+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:04.260 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:03.852+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:04.261 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:03.852+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:04.261 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:03.852+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:04.261 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:03 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:03.852+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:04.566 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:04 smithi154 podman[70269]: 2024-04-22 07:27:04.345496145 +0000 UTC m=+5.850358876 container cleanup 71855abbbcdee9c2898a2af10983d7dd2118bdd5841159ec9f64b56cb9feb776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS) 2024-04-22T07:27:04.566 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:04 smithi154 bash[70269]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8 2024-04-22T07:27:04.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:04 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:04.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:04.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:04 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:04.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:04.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:04 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:04.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:04.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:04 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:04.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:04.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:04 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:04.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:04.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:04 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:04.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:04 smithi154 podman[70288]: 2024-04-22 07:27:04.572114439 +0000 UTC m=+0.554312291 container remove 71855abbbcdee9c2898a2af10983d7dd2118bdd5841159ec9f64b56cb9feb776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:04.809+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:04.809+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:04.809+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:04.809+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:04.809+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:04.889 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:04 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:04.809+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:05.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:04 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:04.679+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:05.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:04 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:04.679+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:05.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:04 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:04.679+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:05.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:04 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:04.679+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:05.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:04 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:04.679+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:05.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:04 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:04.679+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:05.189 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:04 smithi154 podman[70376]: 2024-04-22 07:27:04.794839169 +0000 UTC m=+0.018127298 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:27:05.189 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:04 smithi154 podman[70376]: 2024-04-22 07:27:04.92488755 +0000 UTC m=+0.148175702 container create 60d15da09c3d81bbce353738b94cf483bf96691f2a8575bc61040cf07448c97e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-22T07:27:05.189 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:05 smithi154 podman[70376]: 2024-04-22 07:27:05.11748069 +0000 UTC m=+0.340768798 container init 60d15da09c3d81bbce353738b94cf483bf96691f2a8575bc61040cf07448c97e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-22T07:27:05.189 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:05 smithi154 podman[70376]: 2024-04-22 07:27:05.120515869 +0000 UTC m=+0.343803981 container start 60d15da09c3d81bbce353738b94cf483bf96691f2a8575bc61040cf07448c97e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-22T07:27:05.189 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:05 smithi154 podman[70376]: 2024-04-22 07:27:05.192379307 +0000 UTC m=+0.415667425 container attach 60d15da09c3d81bbce353738b94cf483bf96691f2a8575bc61040cf07448c97e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:27:05.566 INFO:journalctl@ceph.osd.8.smithi154.stdout:Apr 22 07:27:05 smithi154 podman[70376]: 2024-04-22 07:27:05.416451939 +0000 UTC m=+0.639740059 container died 60d15da09c3d81bbce353738b94cf483bf96691f2a8575bc61040cf07448c97e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T07:27:05.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:05 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:05.405+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:05.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:05 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:05.405+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:05.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:05 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:05.405+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:05.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:05 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:05.405+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:05.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:05 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:05.405+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:05.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:05 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:05.405+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:05.992 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:05 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:05.831+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:05.992 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:05 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:05.831+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:05.993 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:05 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:05.831+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:05.993 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:05 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:05.831+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:05.993 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:05 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:05.831+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:05.993 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:05 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:05.831+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:06.015 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.8.service' 2024-04-22T07:27:06.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:06.056 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-22T07:27:06.056 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-22T07:27:06.056 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.9 2024-04-22T07:27:06.083 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:05.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:06.083 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:05.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:06.083 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:05.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:06.083 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:05.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:06.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:05.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:06.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:05 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:05.706+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:06.334 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 systemd[1]: Stopping Ceph osd.9 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:06.733 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.411+0000 7f8b7af55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:06.733 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.411+0000 7f8b7af55700 -1 osd.9 106 *** Got signal Terminated *** 2024-04-22T07:27:06.733 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.411+0000 7f8b7af55700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:27:06.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:06.439+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:06.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:06.439+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:06.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:06.439+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:06.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:06.439+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:06.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:06.439+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:06.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:06 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:06.439+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:07.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:07.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:07.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:07.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:07.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:07.085 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:06 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:06.736+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:07.201 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:06.844+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:07.202 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:06.844+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:07.202 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:06.844+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:07.202 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:06.844+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:07.202 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:06.844+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:07.202 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:06 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:06.844+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:07.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:07.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:07.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:07.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:07.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:07.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:07.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:07.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:07.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:07.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:07.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:07 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:07.429+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:08.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:07.756+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:08.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:07.756+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:08.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:07.756+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:08.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:07.756+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:08.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:07.756+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:08.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:07 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:07.756+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:08.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:07.894+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:08.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:07.894+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:08.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:07.894+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:08.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:07.894+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:08.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:07.894+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:08.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:07.894+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:08.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:08.419+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:08.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:08.419+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:08.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:08.419+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:08.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:08.419+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:08.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:08.419+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:08.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:08 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:08.419+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:09.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:08.789+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:09.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:08.789+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:09.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:08.789+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:09.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:08.789+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:09.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:08.789+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:09.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:08 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:08.789+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:09.315 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:08.906+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:09.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:08.906+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:09.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:08.906+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:09.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:08.906+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:09.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:08.906+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:09.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:08 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:08.906+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:09.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:09.444+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:09.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:09.444+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:09.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:09.444+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:09.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:09.444+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:09.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:09.444+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:09.815 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:09 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:09.444+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:10.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:09.832+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:10.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:09.832+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:10.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:09.832+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:10.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:09.832+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:10.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:09.832+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:10.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:09 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:09.832+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:10.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:09.944+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:10.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:09.944+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:10.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:09.944+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:10.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:09.944+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:10.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:09.944+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:10.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:09.944+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:10.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:10 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:10.447+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:11.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:10.807+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.119096+0000 front 2024-04-22T07:25:58.119318+0000 (oldest deadline 2024-04-22T07:26:23.419363+0000) 2024-04-22T07:27:11.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:10.807+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.420245+0000 front 2024-04-22T07:26:03.420267+0000 (oldest deadline 2024-04-22T07:26:29.319815+0000) 2024-04-22T07:27:11.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:10.807+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:16.321505+0000 front 2024-04-22T07:26:16.321517+0000 (oldest deadline 2024-04-22T07:26:38.621521+0000) 2024-04-22T07:27:11.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:10.807+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:18.621921+0000 front 2024-04-22T07:26:18.621967+0000 (oldest deadline 2024-04-22T07:26:49.822490+0000) 2024-04-22T07:27:11.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:10.807+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:29.823047+0000 front 2024-04-22T07:26:29.823126+0000 (oldest deadline 2024-04-22T07:26:55.123181+0000) 2024-04-22T07:27:11.084 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:10 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9[60275]: 2024-04-22T07:27:10.807+0000 7f8b73541700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:38.524643+0000 front 2024-04-22T07:26:38.524709+0000 (oldest deadline 2024-04-22T07:27:02.624667+0000) 2024-04-22T07:27:11.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:10.960+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:11.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:10.960+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:11.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:10.960+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:11.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:10.960+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:11.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:10.960+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:11.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:10 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:10.960+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:11.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:11 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:11.487+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:12.022 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:11 smithi113 podman[94300]: 2024-04-22 07:27:11.746525783 +0000 UTC m=+5.495218963 container died e10700e6f1a4df8e41247bb41fde090231b3bd9b25f6a8e839f28c0e37aaa486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-22T07:27:12.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:11.982+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:12.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:11.982+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:12.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:11.982+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:12.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:11.982+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:12.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:11.982+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:12.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:11 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:11.982+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:12.334 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 podman[94300]: 2024-04-22 07:27:12.094241678 +0000 UTC m=+5.842934856 container cleanup e10700e6f1a4df8e41247bb41fde090231b3bd9b25f6a8e839f28c0e37aaa486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-22T07:27:12.334 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 bash[94300]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9 2024-04-22T07:27:12.334 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 podman[94319]: 2024-04-22 07:27:12.261990767 +0000 UTC m=+0.511803499 container remove e10700e6f1a4df8e41247bb41fde090231b3bd9b25f6a8e839f28c0e37aaa486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:27:12.813 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:12.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:12.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:12.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:12.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:12.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:12.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:12 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:12.506+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:12.953 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 podman[94412]: 2024-04-22 07:27:12.704439778 +0000 UTC m=+0.113393657 container create e412a47af079d7811026175da4f9fb12a4d8012cdae9ff237ed7554696a8f547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:27:12.954 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 podman[94412]: 2024-04-22 07:27:12.6094339 +0000 UTC m=+0.018387787 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:27:13.245 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 podman[94412]: 2024-04-22 07:27:12.957084491 +0000 UTC m=+0.366038377 container init e412a47af079d7811026175da4f9fb12a4d8012cdae9ff237ed7554696a8f547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-22T07:27:13.245 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:12 smithi113 podman[94412]: 2024-04-22 07:27:12.960679326 +0000 UTC m=+0.369633217 container start e412a47af079d7811026175da4f9fb12a4d8012cdae9ff237ed7554696a8f547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:27:13.245 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:13 smithi113 podman[94412]: 2024-04-22 07:27:13.03172833 +0000 UTC m=+0.440682213 container attach e412a47af079d7811026175da4f9fb12a4d8012cdae9ff237ed7554696a8f547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-22T07:27:13.315 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:13.011+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:13.315 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:13.011+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:13.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:13.011+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:13.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:13.011+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:13.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:13.011+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:13.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:13.011+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:13.565 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:13 smithi113 podman[94412]: 2024-04-22 07:27:13.248886859 +0000 UTC m=+0.657840749 container died e412a47af079d7811026175da4f9fb12a4d8012cdae9ff237ed7554696a8f547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:13.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:13.460+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:13.817 INFO:journalctl@ceph.osd.9.smithi113.stdout:Apr 22 07:27:13 smithi113 podman[94438]: 2024-04-22 07:27:13.818730614 +0000 UTC m=+0.564814354 container remove e412a47af079d7811026175da4f9fb12a4d8012cdae9ff237ed7554696a8f547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T07:27:13.838 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.9.service' 2024-04-22T07:27:13.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:13.916 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-22T07:27:13.916 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-22T07:27:13.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.10 2024-04-22T07:27:14.241 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:13 smithi133 systemd[1]: Stopping Ceph osd.10 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:14.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:14 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:14.038+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:14.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:14 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:14.038+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:14.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:14 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:14.038+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:14.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:14 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:14.038+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:14.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:14 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:14.038+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:14.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:14 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:14.038+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.243+0000 7f1dc54f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.243+0000 7f1dc54f1700 -1 osd.10 106 *** Got signal Terminated *** 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.243+0000 7f1dc54f1700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:14.564 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:14 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:14.454+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:15.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:15 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:15.057+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:15.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:15 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:15.057+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:15.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:15 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:15.057+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:15.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:15 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:15.057+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:15.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:15 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:15.057+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:15.316 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:15 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:15.057+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:15.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:15 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:15.448+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:16.565 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:16 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:16.100+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:16.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:16 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:16.100+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:16.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:16 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:16.100+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:16.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:16 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:16.100+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:16.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:16 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:16.100+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:16.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:16 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:16.100+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:16.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:16 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:16.491+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:17.565 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:17.109+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:17.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:17.109+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:17.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:17.109+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:17.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:17.109+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:17.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:17.109+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:17.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:17.109+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:17.814 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:17 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:17.475+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:18.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:18 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:18.155+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:18.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:18 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:18.155+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:18.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:18 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:18.155+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:18.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:18 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:18.155+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:18.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:18 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:18.155+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:18.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:18 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:18.155+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:18.817 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.868967+0000 front 2024-04-22T07:25:58.868932+0000 (oldest deadline 2024-04-22T07:26:21.769031+0000) 2024-04-22T07:27:18.817 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:07.670100+0000 front 2024-04-22T07:26:07.670186+0000 (oldest deadline 2024-04-22T07:26:31.770334+0000) 2024-04-22T07:27:18.817 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:15.271615+0000 front 2024-04-22T07:26:15.271628+0000 (oldest deadline 2024-04-22T07:26:41.171701+0000) 2024-04-22T07:27:18.818 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:21.172297+0000 front 2024-04-22T07:26:21.172156+0000 (oldest deadline 2024-04-22T07:26:47.072153+0000) 2024-04-22T07:27:18.818 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.373086+0000 front 2024-04-22T07:26:32.373146+0000 (oldest deadline 2024-04-22T07:26:55.873298+0000) 2024-04-22T07:27:18.818 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:35.873906+0000 front 2024-04-22T07:26:35.873789+0000 (oldest deadline 2024-04-22T07:27:01.773759+0000) 2024-04-22T07:27:18.818 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:18 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10[56389]: 2024-04-22T07:27:18.507+0000 7f1dc1ae5700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.113:6822 osd.6 since back 2024-04-22T07:26:46.975454+0000 front 2024-04-22T07:26:46.975418+0000 (oldest deadline 2024-04-22T07:27:09.875330+0000) 2024-04-22T07:27:19.565 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:19.133+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:19.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:19.133+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:19.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:19.133+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:19.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:19.133+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:19.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:19.133+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:19.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:19 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:19.133+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:20.024 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:19 smithi133 podman[68615]: 2024-04-22 07:27:19.75771069 +0000 UTC m=+5.641906689 container died 10895c805196f8e483efd0ca13f0ef184820253903e0e585a5bd4bc38e00c782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-22T07:27:20.313 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68615]: 2024-04-22 07:27:20.160112724 +0000 UTC m=+6.044308717 container cleanup 10895c805196f8e483efd0ca13f0ef184820253903e0e585a5bd4bc38e00c782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-22T07:27:20.314 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 bash[68615]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10 2024-04-22T07:27:20.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:20 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:20.145+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:20.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:20 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:20.145+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:20.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:20 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:20.145+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:20.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:20 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:20.145+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:20.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:20 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:20.145+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:20.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:20 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:20.145+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:20.676 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68632]: 2024-04-22 07:27:20.369378834 +0000 UTC m=+0.605686456 container remove 10895c805196f8e483efd0ca13f0ef184820253903e0e585a5bd4bc38e00c782 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:27:20.970 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68727]: 2024-04-22 07:27:20.582636532 +0000 UTC m=+0.018171615 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:27:20.970 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68727]: 2024-04-22 07:27:20.686637263 +0000 UTC m=+0.122172332 container create 7c95ceec629bace61ff367d8e1c5ea88fc75e396a5b9e8dd000c167e167a6c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T07:27:20.970 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68727]: 2024-04-22 07:27:20.897662197 +0000 UTC m=+0.333197266 container init 7c95ceec629bace61ff367d8e1c5ea88fc75e396a5b9e8dd000c167e167a6c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T07:27:20.970 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68727]: 2024-04-22 07:27:20.900678174 +0000 UTC m=+0.336213249 container start 7c95ceec629bace61ff367d8e1c5ea88fc75e396a5b9e8dd000c167e167a6c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T07:27:20.970 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:20 smithi133 podman[68727]: 2024-04-22 07:27:20.972465975 +0000 UTC m=+0.408001057 container attach 7c95ceec629bace61ff367d8e1c5ea88fc75e396a5b9e8dd000c167e167a6c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:27:21.314 INFO:journalctl@ceph.osd.10.smithi133.stdout:Apr 22 07:27:21 smithi133 podman[68727]: 2024-04-22 07:27:21.198650146 +0000 UTC m=+0.634185210 container died 7c95ceec629bace61ff367d8e1c5ea88fc75e396a5b9e8dd000c167e167a6c39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-22T07:27:21.565 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:21.136+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:21.565 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:21.136+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:21.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:21.136+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:21.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:21.136+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:21.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:21.136+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:21.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:21.136+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:21.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.10.service' 2024-04-22T07:27:21.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:21.848 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-22T07:27:21.848 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-22T07:27:21.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@osd.11 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:21 smithi154 systemd[1]: Stopping Ceph osd.11 for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:22.179 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.113+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2024-04-22T07:26:54.777311+0000 front 2024-04-22T07:26:54.777234+0000 (oldest deadline 2024-04-22T07:27:21.177436+0000) 2024-04-22T07:27:22.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.181+0000 7f5d9a220700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:22.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.181+0000 7f5d9a220700 -1 osd.11 106 *** Got signal Terminated *** 2024-04-22T07:27:22.180 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:22 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:22.181+0000 7f5d9a220700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:23.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:23 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:23.121+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2024-04-22T07:26:54.777311+0000 front 2024-04-22T07:26:54.777234+0000 (oldest deadline 2024-04-22T07:27:21.177436+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:24.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:24 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:24.105+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2024-04-22T07:26:54.777311+0000 front 2024-04-22T07:26:54.777234+0000 (oldest deadline 2024-04-22T07:27:21.177436+0000) 2024-04-22T07:27:25.565 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:25.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:25.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:25.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:25.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:25.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:25.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:25 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:25.143+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2024-04-22T07:26:54.777311+0000 front 2024-04-22T07:26:54.777234+0000 (oldest deadline 2024-04-22T07:27:21.177436+0000) 2024-04-22T07:27:26.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6806 osd.0 since back 2024-04-22T07:25:58.967834+0000 front 2024-04-22T07:25:58.967854+0000 (oldest deadline 2024-04-22T07:26:23.067830+0000) 2024-04-22T07:27:26.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2024-04-22T07:26:03.068646+0000 front 2024-04-22T07:26:03.068487+0000 (oldest deadline 2024-04-22T07:26:28.968592+0000) 2024-04-22T07:27:26.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6804 osd.2 since back 2024-04-22T07:26:14.869687+0000 front 2024-04-22T07:26:14.869760+0000 (oldest deadline 2024-04-22T07:26:40.169821+0000) 2024-04-22T07:27:26.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.113:6814 osd.3 since back 2024-04-22T07:26:22.470822+0000 front 2024-04-22T07:26:22.470906+0000 (oldest deadline 2024-04-22T07:26:45.970768+0000) 2024-04-22T07:27:26.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2024-04-22T07:26:32.073841+0000 front 2024-04-22T07:26:32.073871+0000 (oldest deadline 2024-04-22T07:26:57.373880+0000) 2024-04-22T07:27:26.566 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.154:6812 osd.5 since back 2024-04-22T07:26:37.874740+0000 front 2024-04-22T07:26:37.874757+0000 (oldest deadline 2024-04-22T07:27:03.774759+0000) 2024-04-22T07:27:26.567 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:26 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11[57928]: 2024-04-22T07:27:26.153+0000 7f5d9280c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2024-04-22T07:26:54.777311+0000 front 2024-04-22T07:26:54.777234+0000 (oldest deadline 2024-04-22T07:27:21.177436+0000) 2024-04-22T07:27:27.754 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:27 smithi154 podman[70496]: 2024-04-22 07:27:27.49066008 +0000 UTC m=+5.445455509 container died 378a4e9451a42c7577f6449a4a5ba2db0084b7a2675d745d8983e182cdf717d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-22T07:27:28.065 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:27 smithi154 podman[70496]: 2024-04-22 07:27:27.856166104 +0000 UTC m=+5.810961514 container cleanup 378a4e9451a42c7577f6449a4a5ba2db0084b7a2675d745d8983e182cdf717d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:27:28.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:27 smithi154 bash[70496]: ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11 2024-04-22T07:27:28.472 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70513]: 2024-04-22 07:27:28.116047794 +0000 UTC m=+0.616456007 container remove 378a4e9451a42c7577f6449a4a5ba2db0084b7a2675d745d8983e182cdf717d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:27:28.766 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70606]: 2024-04-22 07:27:28.475301567 +0000 UTC m=+0.114091733 container create ce3b8c667d89fb17fa2649fd5ce4e89549e1e4d696c5c300bbe682a5a02eecce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-22T07:27:28.766 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70606]: 2024-04-22 07:27:28.378955103 +0000 UTC m=+0.017745296 image pull 1b5e03f260bdcb5b725274fde291494c3b969a1a9de8edaab3dd8caea076bb61 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2 2024-04-22T07:27:28.766 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70606]: 2024-04-22 07:27:28.686277764 +0000 UTC m=+0.325067937 container init ce3b8c667d89fb17fa2649fd5ce4e89549e1e4d696c5c300bbe682a5a02eecce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-22T07:27:28.766 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70606]: 2024-04-22 07:27:28.689738952 +0000 UTC m=+0.328529121 container start ce3b8c667d89fb17fa2649fd5ce4e89549e1e4d696c5c300bbe682a5a02eecce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2) 2024-04-22T07:27:28.766 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70606]: 2024-04-22 07:27:28.769314619 +0000 UTC m=+0.408104795 container attach ce3b8c667d89fb17fa2649fd5ce4e89549e1e4d696c5c300bbe682a5a02eecce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2) 2024-04-22T07:27:29.066 INFO:journalctl@ceph.osd.11.smithi154.stdout:Apr 22 07:27:28 smithi154 podman[70606]: 2024-04-22 07:27:28.984431219 +0000 UTC m=+0.623221396 container died ce3b8c667d89fb17fa2649fd5ce4e89549e1e4d696c5c300bbe682a5a02eecce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-22T07:27:29.578 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@osd.11.service' 2024-04-22T07:27:29.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:29.615 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-22T07:27:29.615 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-22T07:27:29.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.a 2024-04-22T07:27:29.916 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:29 smithi113 systemd[1]: Stopping Ceph mds.a for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:30.334 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:29 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a[67251]: 2024-04-22T07:27:29.918+0000 7f9136429700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:30.334 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:29 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a[67251]: 2024-04-22T07:27:29.918+0000 7f9136429700 -1 mds.a *** got signal Terminated *** 2024-04-22T07:27:34.196 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:33 smithi113 podman[94517]: 2024-04-22 07:27:33.925413041 +0000 UTC m=+4.121950633 container died 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:27:34.525 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:34 smithi113 podman[94517]: 2024-04-22 07:27:34.368195997 +0000 UTC m=+4.564733589 container cleanup 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:27:34.525 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:34 smithi113 bash[94517]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a 2024-04-22T07:27:34.754 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.a.service' 2024-04-22T07:27:34.775 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:34 smithi113 podman[94531]: 2024-04-22 07:27:34.52776832 +0000 UTC m=+0.593101672 container remove 332be3bd9c70589fc96fefd35082e66a4104ebd67250bc4e2bdc80729322b70e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-22T07:27:34.775 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:34 smithi113 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.a.service: Deactivated successfully. 2024-04-22T07:27:34.775 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:34 smithi113 systemd[1]: Stopped Ceph mds.a for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:27:34.775 INFO:journalctl@ceph.mds.a.smithi113.stdout:Apr 22 07:27:34 smithi113 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.a.service: Consumed 1.077s CPU time. 2024-04-22T07:27:34.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:34.789 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-22T07:27:34.790 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-22T07:27:34.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.d 2024-04-22T07:27:35.084 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:27:34 smithi113 systemd[1]: Stopping Ceph mds.d for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:35.584 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:27:35 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d[66290]: 2024-04-22T07:27:35.123+0000 7fbe306a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:35.584 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:27:35 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d[66290]: 2024-04-22T07:27:35.123+0000 7fbe306a9700 -1 mds.d *** got signal Terminated *** 2024-04-22T07:27:39.394 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:27:39 smithi113 podman[94647]: 2024-04-22 07:27:39.132144258 +0000 UTC m=+4.116355204 container died 7ca41c63c4d60f22b02068e77ba86484220edc5860284ac4ae9461b1f22dff30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:27:39.669 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:27:39 smithi113 podman[94647]: 2024-04-22 07:27:39.471580324 +0000 UTC m=+4.455791271 container cleanup 7ca41c63c4d60f22b02068e77ba86484220edc5860284ac4ae9461b1f22dff30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:27:39.669 INFO:journalctl@ceph.mds.d.smithi113.stdout:Apr 22 07:27:39 smithi113 bash[94647]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-d 2024-04-22T07:27:39.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.d.service' 2024-04-22T07:27:39.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:39.925 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-22T07:27:39.925 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-22T07:27:39.925 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.g 2024-04-22T07:27:40.272 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:27:40 smithi113 systemd[1]: Stopping Ceph mds.g for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:40.584 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:27:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g[66891]: 2024-04-22T07:27:40.274+0000 7f3b7902c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:40.584 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:27:40 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g[66891]: 2024-04-22T07:27:40.274+0000 7f3b7902c700 -1 mds.g *** got signal Terminated *** 2024-04-22T07:27:44.584 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:27:44 smithi113 podman[94769]: 2024-04-22 07:27:44.259255951 +0000 UTC m=+4.102776509 container died 674baabee99666ae1f6baffdebdb4db4b26135f9e88764271c14ef0772999ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:27:44.872 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:27:44 smithi113 podman[94769]: 2024-04-22 07:27:44.691114913 +0000 UTC m=+4.534635472 container cleanup 674baabee99666ae1f6baffdebdb4db4b26135f9e88764271c14ef0772999ec5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T07:27:44.872 INFO:journalctl@ceph.mds.g.smithi113.stdout:Apr 22 07:27:44 smithi113 bash[94769]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-g 2024-04-22T07:27:45.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.g.service' 2024-04-22T07:27:45.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:45.129 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-22T07:27:45.129 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-22T07:27:45.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.j 2024-04-22T07:27:45.450 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:45 smithi113 systemd[1]: Stopping Ceph mds.j for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:45.834 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j[65894]: 2024-04-22T07:27:45.452+0000 7fa0e8dce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:45.834 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:45 smithi113 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j[65894]: 2024-04-22T07:27:45.452+0000 7fa0e8dce700 -1 mds.j *** got signal Terminated *** 2024-04-22T07:27:49.192 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:48 smithi113 podman[94888]: 2024-04-22 07:27:48.864444794 +0000 UTC m=+3.508564838 container died 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T07:27:49.459 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:49 smithi113 podman[94888]: 2024-04-22 07:27:49.286184768 +0000 UTC m=+3.930304761 container cleanup 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T07:27:49.459 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:49 smithi113 bash[94888]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j 2024-04-22T07:27:49.713 INFO:journalctl@ceph.mds.j.smithi113.stdout:Apr 22 07:27:49 smithi113 podman[94902]: 2024-04-22 07:27:49.462174174 +0000 UTC m=+0.590862831 container remove 66dd16f85579e6366ea0e66b54de595f2f041c219be83d70fba1caedbb3c64b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:27:49.716 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.j.service' 2024-04-22T07:27:49.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:49.746 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-22T07:27:49.746 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-22T07:27:49.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b 2024-04-22T07:27:50.048 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:27:49 smithi133 systemd[1]: Stopping Ceph mds.b for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:50.314 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:27:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[65475]: 2024-04-22T07:27:50.049+0000 7f1414f1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:50.314 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:27:50 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b[65475]: 2024-04-22T07:27:50.049+0000 7f1414f1f700 -1 mds.b *** got signal Terminated *** 2024-04-22T07:27:53.017 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:27:52 smithi133 podman[68845]: 2024-04-22 07:27:52.73510948 +0000 UTC m=+2.789891932 container died a72bad9e888a3595683237956832f1d16e7776e5be7f45ad8fda8cf8759f0b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=) 2024-04-22T07:27:53.314 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:27:53 smithi133 podman[68845]: 2024-04-22 07:27:53.10849749 +0000 UTC m=+3.163279935 container cleanup a72bad9e888a3595683237956832f1d16e7776e5be7f45ad8fda8cf8759f0b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:27:53.314 INFO:journalctl@ceph.mds.b.smithi133.stdout:Apr 22 07:27:53 smithi133 bash[68845]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-b 2024-04-22T07:27:53.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.b.service' 2024-04-22T07:27:53.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:53.554 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-22T07:27:53.554 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-22T07:27:53.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.e 2024-04-22T07:27:54.064 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:27:53 smithi133 systemd[1]: Stopping Ceph mds.e for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:54.064 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:27:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: 2024-04-22T07:27:53.861+0000 7f16f069d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:54.064 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:27:53 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e[61271]: 2024-04-22T07:27:53.861+0000 7f16f069d700 -1 mds.e *** got signal Terminated *** 2024-04-22T07:27:57.025 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:27:56 smithi133 podman[68965]: 2024-04-22 07:27:56.748675016 +0000 UTC m=+2.978247679 container died f9cf8bfae3a662fc8b8f9c211a8bd498fdd9f48b4fda0dde611dd6c0266937d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T07:27:57.314 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:27:57 smithi133 podman[68965]: 2024-04-22 07:27:57.103062 +0000 UTC m=+3.332634662 container cleanup f9cf8bfae3a662fc8b8f9c211a8bd498fdd9f48b4fda0dde611dd6c0266937d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5) 2024-04-22T07:27:57.314 INFO:journalctl@ceph.mds.e.smithi133.stdout:Apr 22 07:27:57 smithi133 bash[68965]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-e 2024-04-22T07:27:57.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.e.service' 2024-04-22T07:27:57.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:27:57.555 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-22T07:27:57.555 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-22T07:27:57.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.h 2024-04-22T07:27:58.064 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:27:57 smithi133 systemd[1]: Stopping Ceph mds.h for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:27:58.064 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:27:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[64880]: 2024-04-22T07:27:57.889+0000 7f46ff6eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:27:58.064 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:27:57 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h[64880]: 2024-04-22T07:27:57.889+0000 7f46ff6eb700 -1 mds.h *** got signal Terminated *** 2024-04-22T07:28:01.030 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:28:00 smithi133 podman[69085]: 2024-04-22 07:28:00.74454417 +0000 UTC m=+2.967348717 container died 3155cda1a7b142c8bd6801d0e4c952f219479ac21c7f24b463c1424326c9b388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-22T07:28:01.313 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:28:01 smithi133 podman[69085]: 2024-04-22 07:28:01.107000121 +0000 UTC m=+3.329804674 container cleanup 3155cda1a7b142c8bd6801d0e4c952f219479ac21c7f24b463c1424326c9b388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T07:28:01.314 INFO:journalctl@ceph.mds.h.smithi133.stdout:Apr 22 07:28:01 smithi133 bash[69085]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-h 2024-04-22T07:28:01.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.h.service' 2024-04-22T07:28:01.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:28:01.559 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-22T07:28:01.559 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-22T07:28:01.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.k 2024-04-22T07:28:01.813 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:01 smithi133 systemd[1]: Stopping Ceph mds.k for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:28:02.564 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k[60948]: 2024-04-22T07:28:02.110+0000 7fc382fa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:28:02.564 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:02 smithi133 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k[60948]: 2024-04-22T07:28:02.110+0000 7fc382fa0700 -1 mds.k *** got signal Terminated *** 2024-04-22T07:28:06.224 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:05 smithi133 podman[69204]: 2024-04-22 07:28:05.975587145 +0000 UTC m=+4.201139716 container died 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-22T07:28:06.475 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:06 smithi133 podman[69204]: 2024-04-22 07:28:06.301848958 +0000 UTC m=+4.527401535 container cleanup 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=) 2024-04-22T07:28:06.475 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:06 smithi133 bash[69204]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k 2024-04-22T07:28:06.475 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:06 smithi133 podman[69219]: 2024-04-22 07:28:06.4775938 +0000 UTC m=+0.500418461 container remove 5d68cde142841f070bbf2661a397434a73459900158726857ae350b2c26a2aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-22T07:28:06.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.k.service' 2024-04-22T07:28:06.745 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:06 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-22T07:28:06.745 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:06 smithi133 systemd[1]: Stopped Ceph mds.k for 81035712-0076-11ef-bc93-c7b262605968. 2024-04-22T07:28:06.745 INFO:journalctl@ceph.mds.k.smithi133.stdout:Apr 22 07:28:06 smithi133 systemd[1]: ceph-81035712-0076-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.096s CPU time. 2024-04-22T07:28:06.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:28:06.755 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-22T07:28:06.755 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-22T07:28:06.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c 2024-04-22T07:28:07.053 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:28:06 smithi154 systemd[1]: Stopping Ceph mds.c for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:28:07.315 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:28:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[67312]: 2024-04-22T07:28:07.055+0000 7ff528d93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:28:07.315 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:28:07 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c[67312]: 2024-04-22T07:28:07.055+0000 7ff528d93700 -1 mds.c *** got signal Terminated *** 2024-04-22T07:28:09.011 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:28:08 smithi154 podman[70728]: 2024-04-22 07:28:08.733304166 +0000 UTC m=+1.779090563 container died 4ff76a22d8d009aa5380be0bfe879ecd7412dbc780d39cc97fdf15ce7253d81c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T07:28:09.315 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:28:09 smithi154 podman[70728]: 2024-04-22 07:28:09.105635702 +0000 UTC m=+2.151422105 container cleanup 4ff76a22d8d009aa5380be0bfe879ecd7412dbc780d39cc97fdf15ce7253d81c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS) 2024-04-22T07:28:09.315 INFO:journalctl@ceph.mds.c.smithi154.stdout:Apr 22 07:28:09 smithi154 bash[70728]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-c 2024-04-22T07:28:09.454 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.c.service' 2024-04-22T07:28:09.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:28:09.489 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-22T07:28:09.489 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-22T07:28:09.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f 2024-04-22T07:28:09.806 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:28:09 smithi154 systemd[1]: Stopping Ceph mds.f for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:28:10.065 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:28:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[67307]: 2024-04-22T07:28:09.809+0000 7f5c17d2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:28:10.065 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:28:09 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f[67307]: 2024-04-22T07:28:09.809+0000 7f5c17d2e700 -1 mds.f *** got signal Terminated *** 2024-04-22T07:28:12.816 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:28:12 smithi154 podman[70851]: 2024-04-22 07:28:12.731694171 +0000 UTC m=+3.015668776 container died 666b931f150d6cf3a83609fd44634b4a7c9ba1187df6c689a1d07527a18dd2b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-22T07:28:13.315 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:28:13 smithi154 podman[70851]: 2024-04-22 07:28:13.13448728 +0000 UTC m=+3.418461882 container cleanup 666b931f150d6cf3a83609fd44634b4a7c9ba1187df6c689a1d07527a18dd2b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-22T07:28:13.316 INFO:journalctl@ceph.mds.f.smithi154.stdout:Apr 22 07:28:13 smithi154 bash[70851]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-f 2024-04-22T07:28:13.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.f.service' 2024-04-22T07:28:13.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:28:13.518 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-22T07:28:13.518 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-22T07:28:13.518 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.i 2024-04-22T07:28:13.815 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:28:13 smithi154 systemd[1]: Stopping Ceph mds.i for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:28:14.316 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:28:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[66673]: 2024-04-22T07:28:13.837+0000 7fe8acb10700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:28:14.316 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:28:13 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i[66673]: 2024-04-22T07:28:13.837+0000 7fe8acb10700 -1 mds.i *** got signal Terminated *** 2024-04-22T07:28:17.002 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:28:16 smithi154 podman[70973]: 2024-04-22 07:28:16.731152588 +0000 UTC m=+2.991772564 container died d376c651472949bbbe92e52948b5ce8c9e2255b28b44c480cdab77abc96fa5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5) 2024-04-22T07:28:17.315 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:28:17 smithi154 podman[70973]: 2024-04-22 07:28:17.096945594 +0000 UTC m=+3.357565567 container cleanup d376c651472949bbbe92e52948b5ce8c9e2255b28b44c480cdab77abc96fa5c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-22T07:28:17.316 INFO:journalctl@ceph.mds.i.smithi154.stdout:Apr 22 07:28:17 smithi154 bash[70973]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-i 2024-04-22T07:28:17.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.i.service' 2024-04-22T07:28:17.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:28:17.545 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-22T07:28:17.545 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-22T07:28:17.545 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-81035712-0076-11ef-bc93-c7b262605968@mds.l 2024-04-22T07:28:18.065 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:28:17 smithi154 systemd[1]: Stopping Ceph mds.l for 81035712-0076-11ef-bc93-c7b262605968... 2024-04-22T07:28:18.066 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:28:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l[62650]: 2024-04-22T07:28:17.842+0000 7fc615c9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T07:28:18.066 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:28:17 smithi154 ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l[62650]: 2024-04-22T07:28:17.842+0000 7fc615c9d700 -1 mds.l *** got signal Terminated *** 2024-04-22T07:28:19.440 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:28:19 smithi154 podman[71092]: 2024-04-22 07:28:19.153518056 +0000 UTC m=+1.392693471 container died da1eb41d991c6ba8d97cc95b6369388cd883ed55aca5980727f980e1c5ee24d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-86f7587) 2024-04-22T07:28:19.807 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:28:19 smithi154 podman[71092]: 2024-04-22 07:28:19.542979144 +0000 UTC m=+1.782154516 container cleanup da1eb41d991c6ba8d97cc95b6369388cd883ed55aca5980727f980e1c5ee24d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3df98b53968d4f80a5e4a8eef98419a71c20df8a9ae66ad24bfd71b414b692d2, name=ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-86f7587, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T07:28:19.808 INFO:journalctl@ceph.mds.l.smithi154.stdout:Apr 22 07:28:19 smithi154 bash[71092]: ceph-81035712-0076-11ef-bc93-c7b262605968-mds-l 2024-04-22T07:28:19.964 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81035712-0076-11ef-bc93-c7b262605968@mds.l.service' 2024-04-22T07:28:19.994 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T07:28:19.994 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-22T07:28:19.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81035712-0076-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T07:28:20.182 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:28:22.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81035712-0076-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T07:28:22.768 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:28:25.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81035712-0076-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T07:28:25.252 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:28:27.602 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:28:27.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:28:27.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T07:28:27.692 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-22T07:28:27.694 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi113/crash 2024-04-22T07:28:27.695 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash -- . 2024-04-22T07:28:27.722 INFO:teuthology.orchestra.run.smithi113.stderr:tar: /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T07:28:27.723 INFO:teuthology.orchestra.run.smithi113.stderr:tar: Error is not recoverable: exiting now 2024-04-22T07:28:27.724 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi133/crash 2024-04-22T07:28:27.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash -- . 2024-04-22T07:28:27.758 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T07:28:27.758 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-04-22T07:28:27.759 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi154/crash 2024-04-22T07:28:27.760 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash -- . 2024-04-22T07:28:27.795 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/81035712-0076-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T07:28:27.795 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now 2024-04-22T07:28:27.796 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-22T07:28:27.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-22T07:28:27.847 INFO:tasks.cephadm:Compressing logs... 2024-04-22T07:28:27.848 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:28:27.851 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:28:27.854 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:28:27.872 INFO:teuthology.orchestra.run.smithi113.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log 2024-04-22T07:28:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:: No such file or directory 2024-04-22T07:28:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T07:28:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-22T07:28:27.880 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-22T07:28:27.880 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-22T07:28:27.880 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-22T07:28:27.880 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-22T07:28:27.880 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-22T07:28:27.880 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.a.log: 87.3% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr: 90.3% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr: 94.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr: 94.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.a.log: 84.0% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-22T07:28:27.881 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-22T07:28:27.891 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-22T07:28:27.891 INFO:teuthology.orchestra.run.smithi113.stderr: 94.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-22T07:28:27.891 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:: No such file or directory 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-22T07:28:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.h.log: 87.3% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr: 90.4% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T07:28:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T07:28:27.895 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-22T07:28:27.895 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-22T07:28:27.896 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log 2024-04-22T07:28:27.896 INFO:teuthology.orchestra.run.smithi154.stderr:: No such file or directory 2024-04-22T07:28:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-22T07:28:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-22T07:28:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T07:28:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-22T07:28:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-22T07:28:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-22T07:28:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-22T07:28:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.c.log: /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.3% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T07:28:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-22T07:28:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T07:28:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T07:28:27.898 INFO:teuthology.orchestra.run.smithi154.stderr: 91.9% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-22T07:28:27.899 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-22T07:28:27.899 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-22T07:28:27.899 INFO:teuthology.orchestra.run.smithi154.stderr: 83.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T07:28:27.899 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T07:28:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T07:28:27.900 INFO:teuthology.orchestra.run.smithi113.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T07:28:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-22T07:28:27.906 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T07:28:27.911 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T07:28:27.913 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-22T07:28:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T07:28:27.930 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-22T07:28:27.930 INFO:teuthology.orchestra.run.smithi154.stderr: 91.0% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T07:28:27.932 INFO:teuthology.orchestra.run.smithi154.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T07:28:27.941 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-22T07:28:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T07:28:28.125 INFO:teuthology.orchestra.run.smithi113.stderr: 90.3% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-22T07:28:28.323 INFO:teuthology.orchestra.run.smithi154.stderr: 92.0% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-22T07:28:28.365 INFO:teuthology.orchestra.run.smithi133.stderr: 92.0% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-22T07:28:29.022 INFO:teuthology.orchestra.run.smithi154.stderr: 94.4% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-22T07:28:29.128 INFO:teuthology.orchestra.run.smithi113.stderr: 90.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-22T07:28:29.505 INFO:teuthology.orchestra.run.smithi113.stderr: 93.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-22T07:28:29.592 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-22T07:28:29.597 INFO:teuthology.orchestra.run.smithi113.stderr: 93.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-22T07:28:29.669 INFO:teuthology.orchestra.run.smithi154.stderr: 93.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-22T07:28:29.726 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-22T07:28:29.771 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-22T07:28:29.788 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-22T07:28:29.813 INFO:teuthology.orchestra.run.smithi113.stderr: 93.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-22T07:28:29.814 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-22T07:28:29.818 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m1.952s 2024-04-22T07:28:29.818 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m8.458s 2024-04-22T07:28:29.818 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.298s 2024-04-22T07:28:30.024 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-22T07:28:30.113 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-22T07:28:30.225 INFO:teuthology.orchestra.run.smithi133.stderr: 93.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-22T07:28:30.395 INFO:teuthology.orchestra.run.smithi154.stderr: 93.5% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-22T07:28:30.455 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-22T07:28:30.751 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-22T07:28:30.796 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-22T07:28:47.487 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-22T07:28:47.489 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-04-22T07:28:47.489 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m19.621s 2024-04-22T07:28:47.489 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m30.539s 2024-04-22T07:28:47.489 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m1.010s 2024-04-22T07:28:51.604 INFO:teuthology.orchestra.run.smithi133.stderr: 93.3% -- replaced with /var/log/ceph/81035712-0076-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-22T07:28:51.606 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-22T07:28:51.606 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m23.739s 2024-04-22T07:28:51.606 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m37.434s 2024-04-22T07:28:51.606 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m1.169s 2024-04-22T07:28:51.607 INFO:tasks.cephadm:Archiving logs... 2024-04-22T07:28:51.607 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi113/log 2024-04-22T07:28:51.608 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:28:53.009 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi133/log 2024-04-22T07:28:53.010 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:28:58.361 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi154/log 2024-04-22T07:28:58.362 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:29:02.795 INFO:tasks.cephadm:Removing cluster... 2024-04-22T07:29:02.796 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81035712-0076-11ef-bc93-c7b262605968 --force 2024-04-22T07:29:02.983 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:29:03.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81035712-0076-11ef-bc93-c7b262605968 --force 2024-04-22T07:29:03.509 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:29:03.799 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81035712-0076-11ef-bc93-c7b262605968 --force 2024-04-22T07:29:03.991 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 81035712-0076-11ef-bc93-c7b262605968 2024-04-22T07:29:04.282 INFO:tasks.cephadm:Removing cephadm ... 2024-04-22T07:29:04.282 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T07:29:04.299 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T07:29:04.315 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T07:29:04.331 INFO:tasks.cephadm:Teardown complete 2024-04-22T07:29:04.331 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T07:29:04.343 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T07:29:04.343 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:29:04.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:29:04.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:29:04.497 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T07:29:04.509 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T07:29:04.509 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:29:04.511 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:29:04.514 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:29:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:29:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:29:04.527 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:29:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:29:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-04-22T07:29:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:^- hv01.front.sepia.ceph.com 2 6 377 61 +1235us[+1235us] +/- 607ms 2024-04-22T07:29:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 +40us[ +56us] +/- 63ms 2024-04-22T07:29:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -29us[ -13us] +/- 66ms 2024-04-22T07:29:04.528 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:29:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:29:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-04-22T07:29:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:^- hv01.front.sepia.ceph.com 2 6 377 63 +1248us[+1248us] +/- 607ms 2024-04-22T07:29:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 63 +39us[ +39us] +/- 63ms 2024-04-22T07:29:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -53us[ -53us] +/- 66ms 2024-04-22T07:29:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:29:04.532 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:29:04.532 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-04-22T07:29:04.532 INFO:teuthology.orchestra.run.smithi154.stdout:^- hv01.front.sepia.ceph.com 2 6 377 64 +1264us[+1264us] +/- 607ms 2024-04-22T07:29:04.532 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 +51us[ +54us] +/- 63ms 2024-04-22T07:29:04.532 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -40us[ -40us] +/- 66ms 2024-04-22T07:29:04.532 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:29:04.533 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T07:29:04.550 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T07:29:04.551 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T07:29:04.625 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:29:04.702 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:29:04.767 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:29:04.831 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:29:04.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:29:04.859 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:29:04.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:29:04.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:29:04.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:29:04.913 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T07:29:04.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T07:29:04.984 INFO:teuthology.task.internal:Duration was 1894.354108 seconds 2024-04-22T07:29:04.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T07:29:05.059 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T07:29:05.059 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:29:05.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:29:05.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:29:05.097 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:29:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:29:05.101 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:29:05.606 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T07:29:05.606 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-04-22T07:29:05.606 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:29:05.627 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-04-22T07:29:05.628 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:29:05.670 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-04-22T07:29:05.670 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:29:05.698 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T07:29:05.698 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:29:05.701 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:29:05.712 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:29:05.763 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T07:29:05.763 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:29:05.767 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:29:05.778 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:29:06.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T07:29:06.159 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T07:29:06.160 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:29:06.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:29:06.207 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:29:06.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T07:29:06.243 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:29:06.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:29:06.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:29:06.264 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-04-22T07:29:06.268 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-04-22T07:29:06.294 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-04-22T07:29:06.313 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:29:06.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:29:06.336 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:29:06.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:29:06.364 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:29:06.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:29:06.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T07:29:06.389 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T07:29:06.390 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi113 2024-04-22T07:29:06.390 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T07:29:06.431 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi133 2024-04-22T07:29:06.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T07:29:06.466 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160/remote/smithi154 2024-04-22T07:29:06.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T07:29:06.501 INFO:teuthology.task.internal:Removing archive directory... 2024-04-22T07:29:06.502 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T07:29:06.504 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T07:29:06.506 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T07:29:06.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T07:29:06.565 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T07:29:06.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T07:29:06.626 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T07:29:06.626 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:29:06.629 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:29:06.631 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:29:06.640 INFO:teuthology.orchestra.run.smithi113.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:29 /home/ubuntu/cephtest 2024-04-22T07:29:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:29 /home/ubuntu/cephtest 2024-04-22T07:29:06.645 INFO:teuthology.orchestra.run.smithi154.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:29 /home/ubuntu/cephtest 2024-04-22T07:29:06.646 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T07:29:06.703 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T07:29:06.862 INFO:teuthology.nuke:Checking targets against current locks 2024-04-22T07:29:06.898 DEBUG:teuthology.nuke:shortname: smithi113 2024-04-22T07:29:06.898 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:29:06.939 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.658753', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6g0JTthriIoXN5kBBGseqoqWy1JOXp45uI8u2MFVYG9zizTzKOY+zEkhKBbMelmYFy26mSKGWmrXvYDlSEp+E='} 2024-04-22T07:29:06.981 DEBUG:teuthology.nuke:shortname: smithi133 2024-04-22T07:29:06.981 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:29:07.052 DEBUG:teuthology.nuke:shortname: smithi154 2024-04-22T07:29:07.052 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:29:07.095 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.662081', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPk9A0djFdO5+ax7FyFdebc7JerxPkuam0fVNNPzK3lJD3HZmwBMDE9OTT8HGce9NbeAQWWiQB2nHL/7E0NuJTg='} 2024-04-22T07:29:07.175 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668160', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:47.660459', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGBpNK4PdUCor6qTZfXUX28WjERiFuxHXPSv/o2ZVN9dqn1K1O5AnqNlRUduwt3ESyYoKq1ZOOQW9Sin5xhXWsA='} 2024-04-22T07:29:07.181 INFO:teuthology.orchestra.console.smithi113:Power off 2024-04-22T07:29:07.182 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:29:07.240 INFO:teuthology.orchestra.console.smithi133:Power off 2024-04-22T07:29:07.240 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:29:07.248 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-04-22T07:29:07.249 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:07.286 INFO:teuthology.orchestra.console.smithi154:Power off 2024-04-22T07:29:07.286 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:29:07.298 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2024-04-22T07:29:07.298 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:07.306 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-04-22T07:29:07.336 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-04-22T07:29:07.337 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:07.347 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-22T07:29:07.385 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-22T07:29:11.308 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:11.348 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:11.387 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:11.419 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-04-22T07:29:11.459 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-22T07:29:11.497 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-04-22T07:29:15.422 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:15.460 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:15.498 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:29:15.534 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-04-22T07:29:15.571 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2024-04-22T07:29:15.611 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-04-22T07:29:15.636 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-04-22T07:29:15.672 INFO:teuthology.orchestra.console.smithi133:Power off completed 2024-04-22T07:29:15.712 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-04-22T07:29:15.817 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/norstats}} duration: 1894.3541078567505 failure_reason: 'error during quiesce thrashing: Expecting value: line 1 column 1 (char 0)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-22T07:29:15.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T07:29:15.927 INFO:teuthology.run:FAIL